Selected work at ART+COM Telekom
Telekom
The system behind the conversation.
AI platform technical concept
AI Platform
Simplified technical conceptUse-case setup
Agent context and
available actions
Guide interface
Controls, state
and transcripts
The coordinating layer
AI orchestrator
- Agent sessions
- Realtime transport
- Function-call dispatch
Janus audio gateway
Microphone input
Response playback
OpenAI Realtime
Speech, responses
and function calls
Exhibition control
Configured action lists
via MQTT to displays and media
About the project
An AI platform developed for Deutsche Telekom’s T Gallery. At ART+COM, I developed its core technical concept and built the realtime orchestration layer connecting configurable agents, guide conversations, and exhibition actions.
- Client / project
- Deutsche Telekom · T Gallery
- Studio
- ART+COM
- My work
- Technical concept & software development
- My contribution
- January–July 2026
Technologies used
- AI platform & guide interface
- TypeScript
- Node.js
- OpenAI Realtime API
- WebRTC / Werift
- Janus / RTP
- MQTT
- MCP
- React
The setting
From a conversation
to an exhibition action.
T Gallery is Deutsche Telekom’s showroom in Bonn. The AI platform connects a guide’s conversation with the surrounding exhibition: a spoken request can invoke a configured action, such as changing the content on a separate display.
The work shown here covers technical concept, implementation, and local demonstrations. Public information establishes the showroom context, but does not confirm the release of this particular platform.
01 / Technical concept
One platform.
Different stories.
Each use case supplies its context and available actions. The orchestrator brings together the agent session, realtime speech, guide controls, and the tool services that connect to the exhibition.
My implementation
Clear boundaries,
one coordinated experience.
I established a dedicated orchestration layer and separated the agent and realtime transport from MQTT messaging. The orchestrator validates guide commands, manages sessions and audio-stream selection, and publishes state and transcripts.
I also built the bridge from the installation’s RTP audio streams into the realtime speech transport, and worked on function-call handling and tool integration. Separate MCP and exhibition-control services execute the configured actions.
An interruption should mean stop
Cancelling a model response does not remove sound already waiting to play. I changed interruption handling to cancel generation and clear buffered output together, keeping the agent’s turn-taking consistent with the audio path.
02 / Component architecture
Inside the
AI platform.
Two interfaces connect to an orchestration service and the exhibition’s shared infrastructure. Controls, live audio and action requests follow separate paths.
Two frontends, one orchestration service and the shared services around it. The connections separate guide controls, live audio and tool-driven exhibition actions.
Guide and admin commands pass through MQTT to the session controller. It coordinates the agent and stream selection, then publishes state and transcripts back to the interfaces.
Janus supplies the selected input stream. Inside the orchestrator, the input adapter passes its audio into the realtime transport. Model responses return directly from that transport to Janus for playback.
The agent session sends a tool call to the MCP action service. That service publishes a configured action request through MQTT to exhibition control; a tool result returns to the conversation.
Frontends
Guide and operations interfaces
Guide interface
Push-to-talk, transcript and playback
Admin interface
Session controls and stream monitoring
AI Orchestrator
One Node.js backend service
Session controller
Commands, lifecycle and shared state
Agent session
Conversation, context and tool calls
Audio input adapter
Selected Janus stream and RTP input
Realtime transport
Speech, events and response audio
Exhibition services
Tool and action execution
MCP action service
Tools for configured action lists
Exhibition control
Configured displays and media
Shared infrastructure
Use-case settings
Session context and available tools
MQTT broker
Commands, state and transcripts
Janus audio gateway
Input streams and response playback
OpenAI Realtime
Speech and model responses
Read the connections
- Controls: guide/admin interfaces exchange commands, state and transcripts with the session controller through MQTT.
- Configuration: shared use-case settings supply session context and the available tool connections.
- Input: interfaces connect to Janus over WebRTC. The orchestrator’s input adapter receives the selected stream, and the coordinating layers forward its RTP packets into the realtime transport.
- Conversation: the transport exchanges speech and events with OpenAI Realtime over WebRTC. Response RTP/UDP audio goes directly back to Janus, bypassing the input adapter.
- Actions: the local agent session calls the MCP service over HTTP. That service publishes configured action requests through MQTT to exhibition control and returns tool results. A returned result does not verify that a display or physical action has finished.
03 / The guide interface
Speak. Listen.
Follow the exchange.
I also worked on the interfaces that make the platform usable: transcripts, listening feedback, and push-to-talk controls that release reliably when disabled or removed. They let a guide follow the exchange and take over.
Running assistant interface · Simulated local services; no audio streamed