Hire Me →
AI · LLM · REALTIME
// 04

AI and LLM Integration for Unity, XR and Web Applications

Intelligent interactive experiences powered by large language models and real-time AI. From virtual AI assistants, to adaptive training simulations that respond to user behaviour.

ABOUT

AI can be used to create virtual patients that actually understand what you say or product assistants that answer questions dynamically. AI integration takes interactive experiences from scripted and predictable to genuinely responsive: training simulations that adapt in real time, virtual assistants that hold a real conversation, product tools that answer questions dynamically.

Large language models and real-time AI embedded directly into Unity or web applications. The shift from scripted to conversational experiences is as significant as any platform change in the last decade.

DELIVERABLES

AI-powered virtual assistants and NPCs in Unity
LLM-integrated product demonstration tools
Adaptive training simulations with real-time feedback
Conversational event booth experiences
AI-driven data visualisation and reporting tools

THE PROCESS

01

AI Architecture & Safety Planning

Define the AI model, prompt strategy, content guardrails and fallback handling appropriate to the context. Particularly important for live event and healthcare deployments.

02

Integration & Prompt Engineering

LLM integration into build tools with latency management, streaming responses and graceful error handling. Prompt engineering refined through iterative testing.

03

Testing & Deployment

Thorough testing under realistic conditions including edge cases, adversarial inputs and high-traffic scenarios before any live deployment.

WHERE AI ACTUALLY HELPS

Most of the AI briefs I see want a chatbot bolted onto something. Sometimes that is right, but the cases that work well in this space tend to be narrower than that.

The first is answering questions against a fixed body of approved content. A congress poster, a set of study data, a product monograph. The content is already written and already approved, and the AI is really just a better index into it.

The second is conversation practice. A rep or a clinician rehearsing a difficult conversation with a character that responds, rather than clicking through a branching tree with four options at each step. Branching scripts are cheaper to build, but people can feel the rails.

The third is adaptive difficulty inside training, where the simulation notices somebody is struggling and changes what it asks of them.

What those have in common is a bounded problem with an approved source behind it. That is the part that makes AI viable in a regulated environment at all really.

SELECTED BUILDS

IBM Watson · Web · ISMPP 2019 · via CMC Connect

Chatbot for a congress poster

A QR code on a printed poster took delegates to a chatbot that could answer questions about the authors, the methodology, the data and the conclusions. It ran in any browser, so there was nothing to install on the stand and it worked on whatever device somebody already had. We retrained the question and answer set across the show as real questions came in. A fair bit of the project was teaching the client how to write content for an AI framework, and that turned out to be most of the value. The poster won best in show.

Unity · Meta Quest · WebGL · iPad · via McCann Health

Conversational VR with a virtual doctor

A doctor’s office where the user is the patient and holds an actual conversation, rather than picking lines from a menu. We used real-time 3D characters with dynamic lip sync instead of pre-baked facial animation. The quality is lower that way, but the team could iterate on the script without going back to animation every time, and on a demo that mattered more than the fidelity did. It ported to Quest, WebGL and iPad off the same build.

THE CONSTRAINTS NOBODY MENTIONS AT BRIEF STAGE

Three things decide whether an AI build is viable, and they rarely come up in the brief.

Approval. In pharma, anything a model says to a physician is promotional material. That usually means constraining it to approved source content and logging every exchange, and it means medical and legal need to see it early rather than at sign-off.

Latency. Two or three seconds is fine in a chat window and unbearable in VR, where it reads as the character having frozen. If the AI sits inside an immersive experience, plan around what can be pre-generated and what genuinely has to be live.

Cost per session. API calls are cheap individually and less cheap across three days of congress with a queue at the stand. Worth modelling before the build rather than after it.

None of that makes AI a bad fit. It just means the fun part is not the hard part.

COMMON QUESTIONS

Can it run without an internet connection?

Not if it is calling a hosted model. On a congress stand that matters, because stand wifi is unreliable. The options are a local model on a machine you control, or pre-generating the responses you know you will need and falling back to those.

How do you stop it saying something it shouldn’t?

Constrain it to approved source content, log every exchange, and test it adversarially before sign-off. In a regulated setting you should assume somebody will try to break it, because at a congress somebody will.

What does an AI-integrated build cost?

Usually £10,000 to £15,000 over nine to sixteen weeks, including the integration work. Ongoing API costs sit on top of that and depend on usage. Full ranges are on the pricing page.

Which model do you use?

Depends on the job, and it changes fairly often at the moment. I build the integration so the model can be swapped out without rewriting the experience around it.

PLATFORMS & DELIVERY

LLM integrations
Various large language models integrated into Unity or bespoke web tools.
Voice Input
Speech-to-text integration for hands-free conversational AI.
Custom APIs
Integration with custom AI services to create fully dynamic experiences.
// OTHER SERVICES
Gamification & Game Experiences→XR Simulators→Web Interactive→AR Sales Aids→Interactive Learning→Immersive Experiences→Holograms & Spatial Computing→Touchscreen Apps→

INTERESTED IN THIS SERVICE?

Get in touch to discuss your project. I will aim to respond within 24-48 hours with an honest assessment of what's possible, how long it will take and what it will cost.