Zuckerberg’s AI projects unlock the power of the metaverse

The metaverse, often regarded as the next iteration of the Internet, combines physical and digital worlds to allow virtual communities to come together and interact in a variety of settings. It allows users to become entirely different people and experience things they couldn’t do in the physical world.

The biggest user of the metaverse so far is the gaming industry, but there are many more possibilities starting to emerge, such as virtual workspaces, networking, and live entertainment.

In 2021, Zuckerberg announced the rebranding of Facebook to Meta, giving the company a focus on bringing the metaverse to life and helping people connect, find communities and grow businesses. In his Founder’s letter, Zuckerberg stated: “The defining quality of the metaverse will be a feeling of presence — like you are right there with another person or in another place. Feeling truly present with another person is the ultimate dream of social technology. That is why we are focused on building this.”

At a recent Meta AI: Inside the Lab event, the company focused on AI’s role in building for the metaverse and showcased some of its work:

Project CAIRaoke is a new approach to AI that powers chatbots and assistants. It is a fully end-to-end neural model for building on-device assistance and is designed to deliver better dialogue capabilities – allowing people to have more fluid conversations with virtual assistants.

Zuckerberg said: “To support true world creation and exploration, we need to advance beyond the current state-of-the-art for smart assistance.”

Part of Project CAIRaoke, BuilderBot is a new tool designed to encourage creativity in the metaverse. Using just voice commands, users can generate and import things into a virtual world. During a livestreamed demonstration, Zuckerberg created a simple virtual world using his voice – asking the AI to add numerous features like trees, a beach, clouds and a table.

The technology is still in its infancy, but Zuckerberg says: “As the technology advances, you’re going to be able to create nuanced worlds to explore and share experiences with just your voice.”

However, there are a lot of challenges still to overcome. For example, did you know that nearly half of the world can’t access the Internet in their own language?

To help tackle this problem, Meta has built an open-source AI model that can translate directly between 100 languages without going to English as an intermediate step. It’s Universal Speech Translator is an AI system that provides instantaneous speech-to-speech translation across all languages and No Language Left Behind is a single system capable of translating between all written languages.

“[We’re] going to keep building technology that enables more people to access the Internet in their own language. In the future, we hope to extend that to content and experiences in the metaverse,” said Zuckerberg. “The ability to communicate with anyone in any language, that’s a superpower that people have dreamed of forever. And AI is going to deliver that within our lifetimes.”

Meta has also introduced the first self-supervised algorithm for speech, vision and text. Self-supervised learning allows machines to understand the world around them without relying exclusively on labelled data. Meta states that: “[This} brings us closer to a world where AI might use videos, articles, and audio recordings to learn about complicated subjects, such as the game of soccer or different ways to bake bread.”

This is just a brief snapshot of the types of AI-based projects that Meta is currently developing. Overall, Zuckerberg wants to “build the metaverse for everyone, so that people around the world have access to tools and technologies to realise their own vision for the future”.

You can watch the full Meta AI: Inside the Lab event here: https://www.facebook.com/MetaAI