Project Malmo uses Minecraft for AI research

That's key to creating systems that can augment human intelligence—and eventually help us with everything from cooking and doing laundry to driving and performing lifesaving tasks in an operating room.

Katja Hofmann, a researcher in Microsoft's Cambridge, UK, research lab, who leads the development of Project Malmo, said the system will help researchers develop new techniques and approaches to reinforcement learning.

That's an area of AI in which agents learn how to complete a task by being given a lot of room for trial and error and then being rewarded when they make the right decision.

"We're trying to put out the tools that will allow people to make progress on those really, really hard research questions," Hofmann said. 

For example, computer scientists have gotten exceptionally good at creating tools that can understand the words we say, whether we're asking a gadget for directions or navigating an automated customer service line.

But when it comes to actually comprehending the meaning of those audio waves – well, in most cases a baby could do better.

"We've trained the AI to identify patterns in the dictation, but the underlying technology doesn't have any understanding of what those words mean," Hofmann said. "They're just statistical patterns, and there's no connection to any experience."

Beyond understanding to comprehension

Teaching AI agents to comprehend humans in the same way we comprehend each other is one of the core goals of advanced AI research. With Project Malmo's public launch, the team has added functionality that will let computer scientists create bots that can learn to talk to each other, and to people.

Project Malmo also can be used to teach AI to do crafting – using tools and resources to build things like a table or a sword – and to learn how to get around on their own without falling down a hill or into a lava pit. They also can learn to build with blocks, navigate mazes and do any number of other tasks that mimic the types of things we might want AI to one day do in real life.

The researchers who have been part of Project Malmo's private preview say Minecraft, with its rich, immersive world and endless possibilities for collaboration and exploration, is ideally suited for general AI research.

"Minecraft is very close to the real world in many ways," said Jose Hernandez-Orallo, a professor at the Technical University of Valencia, Spain, who has been part of the private preview. "There are so many possibilities."

Doing this kind of research requires a lot of trial and error, with small and incremental victories along the way. That's why, when Project Malmo launches publicly, it also will have another new feature: Overclocking, or the ability to run experiments faster than the usual pace of Minecraft's world.

Evelyne Viegas, director of AI outreach at Microsoft Research, said that will allow researchers to get results, and make adjustments, more quickly. "It's accelerating the pace of those experiments," she said.

A standard for measuring progress

The AI researchers who have gotten a sneak peek at Project Malmo say another key advantage to the system is that it will let researchers compare their progress against the work of others, by seeing how well their theories perform in the same environment.

Hernandez-Orallo said AI researchers are often developing their own systems for testing their theories and algorithms. That allows them to solve isolated problems, but it can be tough to know how those results compare to, or would complement, the work of others.

With a system like Project Malmo, he said researchers can test their systems in the same Minecraft setting. The ability to use the same testing ground "is music to my ears," said Hernandez-Orallo, who has a particular interest in AI evaluation and is spending the summer at Microsoft's UK lab so he can work directly with the Project Malmo researchers.

The open-source environment also allows researchers to much more easily collaborate, sharing research insights and bringing their findings together. "There's no question that it vastly speeds up the research process," said Matthew Johnson, the development lead on Project Malmo, who also works in Microsoft's Cambridge, UK, lab.

All coders welcome

Hofmann and her team created Project Malmo to help seasoned AI researchers conduct their research. But they've been pleasantly surprised to find that everyone from tweens with an early passion for programming to professors trying to train the next generation of AI researchers want to work with it as well.

Viegas said more novice coders can experience the system. "You need to know how to program, but you don't need to be an advanced programmer," she said.

The Project Malmo platform consists of a mod for the Java version and code that helps AI agents sense and act within the Minecraft environment. The two components can run on Windows, Linux or Mac OS, and programmers can use any programming language they are comfortable with.

The team also has heard from several professors who want to incorporate Project Malmo into their lesson plans. 

That makes sense. Hernandez-Orallo said his students – who may well spend their free time playing Minecraft – are going to be a lot more excited by an assignment using Project Malmo than by one that asks them to work with a more generic algorithm pulled from a research paper.

"This is going to have an impact in education, at least at the university level," he said. Johnson said they are already seeing people produce academic research based on Project Malmo, and that's the core reason for doing a project like this. But he concedes that it's also fun to imagine that a more mainstream audience might want to check it out.

"If I come across some YouTube video showing off some exciting new functionality enabled by our mod, that would make my day," he said.