Today I’m going to talk about the integration of AI with VR. There has been a lot of buzz and anticipation surrounding recent advancements in AI, specifically chatbots/large language models and generative image applications, over the past few months. It’s understandable that people are curious and excited about these tools, as they have often proven to streamline tasks and save time. Of course, concerns about the negative implications of AI also exist, ranging from legitimate worries about job displacement to more far-fetched notions about the potential downfall of humanity.
While I don’t intend to delve into speculation on those matters here, I would like to address a peculiar false equivalence that I have seen appear in several articles. Some commentators, seemingly revelling in schadenfreude when it comes to Mark Zuckerberg, assert that Meta has “backed the wrong horse” in investing in VR research and would have been better off directing those resources toward AI research.
I believe this perspective is misguided for several reasons, primarily because VR and AI should not be pitted against each other as an either/or choice. They are distinct technologies with numerous ways in which they can complement each other. In fact, AI assisted technologies have quietly found applications in immersive technologies since long before the current AI frenzy.
Those of us involved in creating immersive technology apps have long anticipated and speculated about the developments in AI and their integration into VR. Therefore, it appears rather nonsensical to portray AI as the victor in an imaginary race against other dissimilar (though sometimes complementary) technologies.
Enhancing Realism: AI-Driven Rendering and Interaction
Here are a few examples of how we have been implementing AI in our workflow. NeRF, short for Neural Radiance Fields, is a technology that aims to capture and represent complex 3D scenes or objects by learning their appearance and geometry from a set of images or video footage. It leverages deep learning techniques and neural networks to construct a volumetric representation of the scene or object, allowing for more realistic rendering and interaction.
While NeRF technology is still in its early stages and generally cannot currently produce fully usable 3D assets for VR, it serves as a valuable tool in the production process. It enables us to quickly and accurately capture the scale and structure of real-world environments, providing a reference for our creative developers to build upon when creating 3D models.
Immersive Voice Experiences: AI-Generated Voices in VR
In addition to NeRF, the use of AI-generated voices has become prevalent in various applications, including VR. AI-generated voices can simulate human-like speech patterns and intonations, enhancing the immersive experience for users. Currently, the scripts for these AI-generated voices need to be written by humans. However, it is evident that with advancements in AI technology, it would be relatively straightforward to develop a VR avatar capable of answering a wide range of questions in real-time.
Natural Language Processing (NLP) – AI-powered language processing technology will make interacting with virtual characters (including but not limited to within a VR environment) more natural and believable than ever before.
These are just two examples of AI we currently use, and an idea of how they will develop in future. There will be many more usages of AI as technologies develop. For example AI will be used to personalise a user’s VR experience. AI algorithms can analyse user behaviour and preferences to give the user a completely tailored experience. By understanding user preferences, AI can dynamically adjust the content, visuals, and interactions to suit individual needs and interests, creating a more tailored and enjoyable virtual experience.
Personalising the VR Experience with AI
AI also has the potential to greatly enhance the immersive experience in VR. By leveraging AI algorithms, VR applications can provide more realistic and interactive environments, incorporating elements such as intelligent NPCs (non-player characters) that respond dynamically to user interactions, flexible and adaptive environments that adjust based on user behaviour, and personalised content recommendations tailored to individual users’ preferences and interests.
AI will assist in content creation for VR applications. Generative AI models can generate realistic 3D assets, environments, and characters, reducing the time and effort required for manual content creation. AI algorithms can also assist in automating aspects of level design, object placement, and scene optimisation, making the process more efficient and allowing developers to focus on the creative aspects.
Advancing Immersion: AI-Enabled NPCs and Adaptive Environments
AI also has the potential to greatly enhance the immersive experience in VR. By leveraging AI algorithms, VR applications can provide more realistic and interactive environments, incorporating elements such as intelligent NPCs (non-player characters) that respond dynamically to user interactions, flexible and adaptive environments that adjust based on user behaviour, and personalised content recommendations tailored to individual users’ preferences and interests.
AI in Content Creation: Streamlining Development for VR
AI will assist in content creation for VR applications. Generative AI models can generate realistic 3D assets, environments, and characters, reducing the time and effort required for manual content creation. AI algorithms can also assist in automating aspects of level design, object placement, and scene optimisation, making the process more efficient and allowing developers to focus on the creative aspects.
Real-Time Data Integration: AI’s Impact on Dynamic VR Experiences
AI-enabled technologies will assist in real-time Data Integration. AI algorithms can analyse real-time data from various sources, such as sensors or user inputs, to dynamically adapt VR experiences. For example, AI could analyse biometric data to adjust the virtual environment based on the user’s emotional state or physiological responses, creating more immersive and personalised experiences.
Embracing the Future: AI and VR Pushing Boundaries
Advancements in AI and VR technologies present a thrilling and promising era of innovation. Rather than viewing AI and VR as competing technologies, it is essential to recognise their unique strengths and their potential to complement each other. AI contributes to the production process, enhances the immersive experience, assists in content creation, and enables personalised and adaptive VR environments.
We are constantly striving to weave new AI technologies into our Avatar Academy platform where appropriate. Despite challenges in data management and privacy, the opportunities for creating captivating and impactful technologies are abundant. By embracing the integration of AI and VR, we can unlock new possibilities and push the boundaries of human experiences in entertainment, education, communication, and beyond.
-Fiona Moran, Mersus Technologies
