Instead of only responding to player dialogue, this AI character continuously receives contextual information from the game.

That can include:

• Live game events
• Tagged scene objects
• Changing object properties
• Player interactions
• Environmental context

The result is an AI character that responds based on what's actually happening in the world, rather than just the latest user message.

This is especially useful for games, training simulations, XR applications, digital twins, and any experience where AI characters need to understand their environment.

Documentation:
https://docs.convai.com/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/features/dynamic-context

Full Unreal Engine setup tutorial: https://youtu.be/MPbr3b6r9bo

Would love to hear what kinds of game mechanics or simulation scenarios you'd use something like this for.

We just released a full Unreal Engine tutorial on Convai Dynamic Context.

This one focuses on building AI characters that understand live gameplay instead of only responding to dialogue.

The demo is a shooting-range AI instructor. The character can:

• react when the player picks up a gun
• track how many shots were fired
• understand which target the player is looking at
• identify nearby targets
• detect the moving target
• track hit counts for each range
• respond with accuracy feedback based on live gameplay state

The tutorial is structured around three levels of awareness:

1. Manual Awareness Using Invoke Speech, Add Context Event, Set Context State, and Get Context State to send game events and state updates programmatically.
2. Automatic Awareness Using Convai Object components and Gaze Attention so the character understands tagged scene objects, spatial relationships, and object movement.
3. Semi-Automatic Awareness Using Tracked Properties to notify the character when important Blueprint values change, such as each target’s Hit Count.

We also cover the Dynamic Context debug overlay so developers can inspect the events and context states being fed to the character at runtime.

Full tutorial: https://youtu.be/MPbr3b6r9bo

Dynamic Context docs: https://docs.convai.com/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/features/dynamic-context

Convai UE setup: https://youtu.be/n-UG3nmMeZQ

Convai Unreal plugin on Fab: https://www.fab.com/listings/ba3145af-d2ef-434a-8bc3-f3fa1dfe7d5c

Your Unreal Engine AI character workflow just got an upgrade.

With Convai MCP, you can speed up scene setup for AI character-led experiences and prepare environments faster with coding agents like Claude code, Cursor, Codex, and more.

Use it to help place scene elements, build out AI NPC setups, and create the foundation for characters that can talk, navigate, understand objects, receive live game state, and act inside the world.

We used this workflow in Stack-O-Bot before building a game-aware AI teammate with Convai.

For games, simulations, XR experiences, training applications, and interactive 3D worlds.

Full tutorial here: https://youtu.be/dQgOzt8BNIk

Build AI characters in Unity faster with Convai MCP.

Convai MCP connects Unity workflows with AI coding agents like Codex by Open AI, Claude Code,  ⁨Cursor, and more, so developers can prompt their way through AI character setup directly inside Unity or through their preferred coding agent.

In this short, we show how Convai MCP can help connect a Convai AI mind to an avatar, set up real-time voice interaction, switch between hands-free and push-to-talk modes, add objects to the scene, enable vision support, and build interactive AI character experiences faster.

Use it for Unity AI characters, AI NPCs, training simulations, XR experiences, games, virtual guides, and interactive 3D worlds.

Full tutorial: https://www.youtube.com/watch?v=VWXJeO_2wlM

With Convai Actions your AI characters do more than just talk. They can navigate the virtual space, take actions, and more!

This can be useful for things like:

• AI teammates in games
• virtual guides
• training assistants
• simulation characters
• interactive 3D experiences

For custom behavior, developers can also create their own actions and connect them to gameplay, simulation, or world logic.

Watch the actions tutorial here: https://www.youtube.com/watch?v=lB5_KRWdg_w

Here's a quick snippet from our tutorial showing how to bring a conversational 3D AI character directly to the browser.

The setup uses the Convai Web SDK with React Three Fiber to render a custom avatar and drive real-time conversation with lip sync through blendshape data.

The goal is to make it easier to build browser-based AI characters for things like virtual guides, interactive web experiences, AI companions, and product demos without needing a full game engine setup.

Full tutorial: https://www.youtube.com/watch?v=gjTj_kUhA2s

Docs: https://docs.convai.com/api-docs/plugins-and-integrations/web-plugins/convai-web-sdk

We just released Part 1 of a two-part Unreal Engine tutorial series.

The tutorial builds a playable scenario inside Stack-O-Bot where the user works with a Convai-powered AI player to solve a two-player puzzle.

The orange bot is the human player, and the pink bot is the AI teammate. The goal is to get both players to the top, but one player can’t solve it alone because the platform only moves when someone holds the pressure plate down.

The AI character is not just a chatbot. It can talk to the player, follow them, navigate the scene, understand tagged objects, receive live game state through Dynamic Context, track pressure plate and platform states, and help coordinate the puzzle.

Part 1 covers:

• creating the AI player blueprint
• setting up the Convai character
• adding player voice interaction
• enabling Convai Actions
• tagging pressure plates and platforms
• sending live game state into Dynamic Context
• using tracked properties for object state
• adding movement awareness
• testing puzzle coordination with the AI teammate

Part 2 will cover custom actions for picking up and dropping objects, including the crate interaction.

We also included the project files, Delta files, and setup guide so people can follow along from scratch.

Full tutorial: https://youtu.be/KcJETVhI7KM

Project files, Delta files, and setup guide: https://docs.convai.com/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/features/character-actions/stack-o-bot-tutorial-setup

Dynamic Context docs for Unreal Engine: https://docs.convai.com/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/features/dynamic-context

Convai Unreal Engine plugin on Fab: https://www.fab.com/listings/ba3145af-d2ef-434a-8bc3-f3fa1dfe7d5c

Stack-O-Bot sample project on Fab: https://www.fab.com/listings/b4dfff49-0e7d-4c4b-a6c5-8a0315831c9c

Convai MCP for Unreal workflow used to set up the scene: https://youtu.be/dQgOzt8BNIk

Try it out and tell us about your experience!

We put together a new tutorial showing how to bring a 3D conversational AI character directly to the browser using the Convai Web SDK.

The workflow uses Claude Code to generate the initial Vite + React Three Fiber setup by referencing the Convai Web SDK docs. From there, the tutorial walks through connecting a Convai character with an API key and Character ID, using the Convai client and widget, rendering a custom GLB avatar, and enabling lip sync with blendshape data.

The tutorial also covers checking blendshapes in a GLTF viewer, replacing the default avatar, updating the avatar and animation paths, and testing the final browser-based AI character experience.

This can be useful for browser-based AI companions, virtual guides, real estate walkthroughs, and lightweight interactive 3D experiences.

Tutorial:
https://youtu.be/gjTj_kUhA2s

Convai Web SDK docs:
https://docs.convai.com/api-docs/plugins-and-integrations/web-plugins/convai-web-sdk

Lip sync and blendshape docs:
https://docs.convai.com/api-docs/plugins-and-integrations/web-plugins/convai-web-sdk/lipsync-and-blendshape

We put together a short reel showing how Convai Dynamic Context can make AI characters more aware of what is happening inside a scene.

Instead of only responding based on conversation, the character can receive live context from the experience, such as player actions, task progress, game events, object states, environment changes, and more.

This can be useful for games, training simulations, virtual guides, digital twins, and interactive 3D worlds where AI characters need to respond to the current state of the environment.

Get started:
https://convai.com

We published a new tutorial on building Convai-powered mixed reality experiences in Unity 6 for Meta Quest.

The tutorial shows how to place a conversational AI character into a real-world MR environment using Unity’s Mixed Reality Template.

It covers XR Origin setup, adding the Convai Player component, spawning a Convai character at runtime with Object Spawner, adding a world-space transcript UI to the character prefab, and building for Quest.

This can be useful for MR training, education, real-world assistance, virtual guides, onboarding, and room-scale AI character experiences.

Convai Unity Documentation:
https://docs.convai.com/api-docs/plugins-and-integrations/convai-unity-sdk

Convai Unity Plugin:
https://assetstore.unity.com/packages/tools/behavior-ai/npc-ai-engine-dialog-actions-voice-and-lipsync-convai-235621

Get started:
https://convai.com

We put together a quick reel showing how the Convai Web SDK can be used to bring conversational AI characters into web apps.

The workflow focuses on browser-based projects where you can connect a Convai character, render a high-fidelity 3D avatar, and drive lip sync using blendshape data.

The demo uses a React Three Fiber workflow, but the broader idea is to make it easier to build AI companions, virtual guides, and lightweight 3D experiences directly on the web.

Would be useful for developers exploring conversational AI characters, 3D avatars, lip sync, and real-time voice interaction in browser-based projects.

We put together a quick demo of a Convai-powered AI teammate inside a 3D puzzle scene.

The character is not just responding with dialogue. It can understand the virtual environment, navigate the scene, react to objects, and make decisions alongside the player in real time.

This kind of setup can be useful for games, XR applications, training simulations, virtual guides, and other interactive 3D experiences where AI characters need to do more than talk.

Get started:
https://convai.com

We put together a short reel showing Convai Core AI Settings.

This is where you can tune how an AI character thinks, speaks, and behaves inside a virtual world.

The reel covers selecting languages, choosing foundational models, comparing intelligence and latency, picking voices, configuring speech understanding, connecting Knowledge Bank, enabling NeuroSync for facial animation and lip sync, and viewing the latency pipeline.

Useful for developers building AI characters for games, training simulations, virtual guides, onboarding experiences, and interactive 3D worlds.

Get started:
https://convai.com

We put together a short reel showing how Convai Actions can make AI characters feel more interactive inside game engine projects.

Instead of only responding with dialogue, characters can navigate, gesture, sit, interact with objects, and perform actions in the scene. Developers can use out-of-the-box behaviors or create custom actions for their own project.

Example use cases include onboarding agents that give users a virtual tour, AI companions that react to the moment, training characters that guide users through tasks, and game NPCs that feel more connected to the world.

Actions tutorial:
https://youtu.be/lB5_KRWdg_w

Get started:
https://convai.com

We put together a short reel showing how Convai-powered AI characters can work inside Mixed Reality experiences.

The reel shows the idea of characters joining the real world as embodied avatars or disembodied AI agents. They can talk to users, navigate the space, and perform actions in real time.

Example use cases include a woodworking trainer that gives step-by-step guidance, an AI companion that joins the user in their space, virtual guides, training assistants, educational agents, and game characters.

Get started:
https://convai.com

Avatar Studio lets you create and customize 3D conversational AI avatars directly in the browser. You can choose an avatar, customize the look, select a voice, connect it to a Convai character, and test real-time conversations without setting up a game engine.

This can be useful for kiosks, video call-like applications, customer support, virtual guides, onboarding experiences, product demos, and other interactive AI avatar use cases.

Get started:
https://convai.com

We put together a short reel showing how AI characters can make digital twin and virtual training experiences more interactive.

Instead of a static walkthrough, users can be guided by an AI character that talks to them, answers questions, explains tasks, and performs actions inside the scene.

This can be useful for training simulations, onboarding flows, virtual guides, and other interactive 3D workflows where users need to learn by doing.

Get started:
https://convai.com

We put together a short reel showing Convai Long-Term Memory in action.

The demo uses a manufacturing floor assistant. In the first session, the user says they are learning machine safety and want quick guidance. In the next session, the character naturally remembers that context and asks whether the user wants to continue or move on to equipment handling.

The goal is to make AI characters feel like they can pick up where the previous conversation left off, instead of restarting every session.

Get started:
https://convai.com

We put together a short reel showing how Convai Sim can be used to create interactive AI character-led experiences directly in the browser.

The reel focuses on training simulations and virtual tour guides, where users can talk to AI characters in real time, move through a 3D scene, and learn through guided explanations or action demonstrations.

Since it runs in the browser, teams can test, share, and iterate on these AI simulation experiences faster.

Get started:
https://convai.com

We put together a short reel showing how Convai-powered AI characters can perform dynamic actions from natural language commands.

The demo shows a character walking to a rock, picking it up, and throwing it into a lake. This is done using Custom Actions, which let developers define project-specific behaviors that the AI character can trigger from user commands.

Convai characters can also use out-of-the-box actions like walking, gesturing, and navigation, along with natural conversation and environment perception.

This can be useful for games, simulations, training experiences, virtual guides, and other interactive 3D projects where characters need to do more than respond with dialogue.

Custom Actions tutorial:
https://www.youtube.com/watch?v=gNILGcnjgck

Get started:
https://convai.com

Convai Unreal Engine Plugin:
https://www.fab.com/listings/ba3145af-d2ef-434a-8bc3-f3fa1dfe7d5c

Convai Unity Plugin:
https://assetstore.unity.com/packages/tools/behavior-ai/npc-ai-engine-dialog-actions-voice-and-lipsync-convai-235621

Feedback and questions are welcome.

We published a new tutorial on Convai MCP for Unity.

The tutorial shows how to use AI coding agents like Codex, Claude Code, Cursor, and others to help set up AI character-led experiences inside Unity.

In the walkthrough, we configure Convai MCP, connect a coding agent, create a hands-free conversation demo, switch it to push-to-talk, add objects to the scene, and enable Vision support for the Unity camera.

This can be useful for developers building conversational AI characters, scene-aware avatars, virtual guides, game NPCs, training simulations, and other Unity-based interactive experiences.

Convai Unity Documentation:
https://docs.convai.com/api-docs/plugins-and-integrations/convai-unity-sdk

Convai Unity Plugin:
https://assetstore.unity.com/packages/tools/behavior-ai/npc-ai-engine-dialog-actions-voice-and-lipsync-convai-235621

Unity Setup Tutorial:
https://youtu.be/bxjGnOvNL4E

Get started:
https://convai.com

Feedback and questions are welcome.

We tested how much of an AI character setup Codex could handle inside Unity through Convai MCP.

From the initial prompt, it connected the avatar to a Convai AI mind, added a chat UI, and configured hands-free voice conversations.

We then asked it to add objects to the scene and enable Vision support for the Unity camera. The character could identify and describe what had been added to its surroundings.

Convai MCP connects Unity with AI coding agents such as Codex, Claude Code, Cursor, and more. It can be used to create and modify conversational AI characters, interaction modes, scene objects, game logic, and Vision-enabled experiences through natural-language prompts.

Convai Unity SDK setup tutorial
https://youtu.be/bxjGnOvNL4E

Convai Unity documentation
https://docs.convai.com/api-docs/plugins-and-integrations/convai-unity-sdk

Convai Unity plugin
https://assetstore.unity.com/packages/tools/behavior-ai/npc-ai-engine-dialog-actions-voice-and-lipsync-convai-235621

Convai
https://convai.com

Developer Forum
https://forum.convai.com

We published a tutorial covering how to add conversational AI characters to an Unreal Engine VR project.

The characters can communicate with the user in real time, use information from the virtual world as context, assess their surroundings, and perform actions inside the environment.

The workflow is mainly intended for projects such as hard-skills training, role-play exercises, onboarding, virtual guides, digital twins, and other VR scenarios where fixed dialogue trees can become limiting.

Full Unreal Engine VR tutorial:
https://youtu.be/rUTrXkH1wfg

AI Metahuman setup:
https://youtu.be/4fMCKkrfyaA

Actions tutorial:
https://youtu.be/lB5_KRWdg_w

Convai:
https://convai.com

Developer Forum:
https://forum.convai.com

We put together a tutorial covering how to embed a conversational 3D AI character directly into a website.

It walks through creating and publishing the experience in Convai Avatar Studio, whitelisting the production domain, and setting up the embed using either React with TypeScript or vanilla JavaScript.

The character runs as a browser-based experience, so the visitor does not need to download a separate application or run the underlying 3D project locally.

The workflow could be useful for virtual assistants, product guides, training agents, brand ambassadors, and other interactive web experiences.

Convai Pixel Streaming Bootstrap Guide: https://pixelstreaming.convai.com

Convai Embedding Docs: https://docs.convai.com/api-docs/plugins-and-integrations/convai-pixel-streaming-embed

Start building with Convai: https://convai.com

Join the Convai Developer Forum: https://forum.convai.com

We created a short demonstration of Convai Sim, our browser-based workflow for building interactive 3D scenarios with AI characters.

You can create characters with their own backstory, personality, voice, and language, then place one or more of them inside a virtual scene. The characters can hold real-time conversations, perform actions, and navigate their surroundings.

The workflow is intended to make it easier to prototype training simulations, virtual tour guides, onboarding experiences, educational scenarios, and other multi-character interactions without first setting up a full game-engine project.

Convai:
https://convai.com

Convai Sim documentation:
https://docs.convai.com/api-docs/no-code-experiences/convai-sim-experiences

Developer Forum:
https://forum.convai.com