sequenceDiagram
participant V as Visitor
participant M as Microphone
participant W as Whisper ASR
participant Q as Qdrant Vector DB
participant G as GPT-4o (Groq)
participant E as ElevenLabs TTS
participant A as Avatar Display
V->>M: Speaks question
M->>W: Audio stream
W->>Q: Query embedding
Q->>G: Retrieved context + query
G->>E: Response text
E->>A: Synthesized speech + morph
A->>V: Persona response
| Layer | Component | Spec | Latency (ms) |
|---|---|---|---|
| Capture | ClearOne BMA-CT | 12° beam, −48 dB noise floor | <10 |
| Vision | Intel RealSense D435 | 90 fps depth | <10 |
| ASR | OpenAI Whisper-tiny | Local processing | 120 |
| Inference | GPT-4o 32k on Groq LPU | RAG prompt | 250–400 |
| TTS | ElevenLabs | 22050 Hz synthesis | 350 |
| Avatar mouth | HeyGen live-talk | Real-time sync | 60 |
| Total | target < 1.5s |
Visitors select a persona (Hoosier Oracle, Vonnegut, etc.) which triggers the idle state → transition animation → persona state sequence.
The system transcribes visitor speech, processes the query through GPT-4o (or other LLM), then converts the response to text-to-speech.
Audio streams feed into facial animation systems (Audio2Face or Speech Graphics SGX) to generate lip-synced movement in real time.
Raw facial animation is transformed through a stylization layer:
The stylized video with alpha channel is:
Our AI system uses ElevenLabs’ voice cloning technology to recreate the authentic speech patterns of each Indiana persona. Historical recordings, when available, are used to train persona-specific voice models. For figures without recorded speech, we employ linguistic analysis of their written works combined with regional accent modeling to create plausible voice representations.
Latency Management: Target response under 1.5 seconds during typical load. When processing exceeds this threshold, particle animation loops maintain engagement while displaying “The Oracle is considering…” status. Local fallback models activate during cloud service interruptions.
The Oracle operates through a carefully orchestrated state system designed for smooth visitor interaction:
stateDiagram-v2
[*] --> IDLE
IDLE --> SUMMON : Motion Detected
SUMMON --> LIVE_TALK : Voice Input
LIVE_TALK --> LIVE_TALK : Conversation
LIVE_TALK --> OUTRO : Silence (10s)
LIVE_TALK --> GLITCH : Latency > 700ms
GLITCH --> LIVE_TALK : Recovery
OUTRO --> IDLE : Complete
SUMMON --> IDLE : Timeout (30s)
State Descriptions:
Real-Time Rendering Stack:
Visual Layer Management:
Failsafe Systems: When cloud services lag, the system transitions to cached responses with “thinking” animations until connectivity restores. Local TTS provides basic functionality during extended outages.
Our primary display approach uses Pepper’s Ghost illusion technology, creating convincing 3D presence without requiring special glasses:
Light-Field Technology: Looking Glass displays provide true parallax viewing for multiple simultaneous viewers (32”-65” units, $15K-$45K hardware cost).
Transparent OLED: Layered glass configuration allows see-through effects with particle depth layers (55” FHD panels, ~$16K per unit).
Volumetric Projection: Proto hologram units for reliable 3D presence in full installations.
Download the complete Bill of Materials and technical specifications for implementation.
📄 Download BOM.xlsx
Promotional Visual Language — Art Deco meets digital futurism
While the Oracle entities will appear as monochromatic, spectral forms — smoky particles and digital glitches bridging past and future — the promotional materials work in a related but distinct style. They lean on Art Deco geometry, vintage electrical motifs, and kinetic motion typography to shape an “old-meets-new” aesthetic for campaign pieces and adjacent projects.
Letterforms assemble and dismantle with machine-like precision, a gesture toward temporal echoes and a nod to Indiana’s manufacturing and craft traditions — from RCA’s years in Bloomington to the makers and tradespeople of the Showers district. The imagery borrows from moments when technology felt theatrical and full of promise — the polish of Deco, the spark of early electronics, and the speculative tone of classic sci-fi — and ties that sensibility to the state’s universities, notably IU in Bloomington, as well as Notre Dame and other research centers. Together, these elements create a visual language that connects industrial history, design heritage, and a sense of forward momentum.