Selected work at ART+COM Telekom
Telekom
The system behind the conversation.
AI platform technical concept
AI Platform
Simplified technical conceptMy work at ART+COM: the core technical concept and orchestration. Read the central layer as the coordinator connecting a guide’s conversation with separate speech, audio and exhibition services.
Use-case setup
Agent context and
available actions
Guide interface
Controls, state
and transcripts
The coordinating layer
AI orchestrator
- Agent sessions
- Realtime transport
- Function-call dispatch
Janus audio gateway
Microphone input
Response playback
OpenAI Realtime
Speech, responses
and function calls
Exhibition control
Configured action lists
via MQTT to displays and media
About the project
An AI platform developed for Deutsche Telekom’s T Gallery. At ART+COM, I developed its core technical concept and built the realtime orchestration layer connecting configurable agents, guide conversations, and exhibition actions.
My contribution
I developed the platform’s core technical concept and realtime orchestration at ART+COM, separating conversation and audio transport from exhibition messaging. For interruptions, I made response cancellation clear queued audio as well: stopping generation alone would leave sound waiting to play. The model, audio gateway and exhibition-control services remain separate from the orchestrator.
Read about orchestration & interruption- Client / project
- Deutsche Telekom · T Gallery
- Studio
- ART+COM
- My work
- Technical concept & software development
- Contribution period
- January–July 2026
Technologies used
- AI platform & guide interface
- TypeScript
- Node.js
- OpenAI Realtime API
- WebRTC / Werift
- Janus / RTP
- MQTT
- MCP
- React
The setting
From a conversation
to an exhibition action.
T Gallery is Deutsche Telekom’s showroom in Bonn. The AI platform connects a guide’s conversation with the surrounding exhibition: a spoken request can invoke a configured action, such as changing the content on a separate display.
The work shown here covers technical concept, implementation, and local demonstrations. Public information establishes the showroom context, but does not confirm the release of this particular platform.
01 / Technical concept
One platform.
Different stories.
Each use case supplies its context and available actions. The orchestrator brings together the agent session, realtime speech, guide controls, and the tool services that connect to the exhibition.
My implementation
Clear boundaries,
one coordinated experience.
I established a dedicated orchestration layer and separated the agent and realtime transport from MQTT messaging. The orchestrator validates guide commands, manages sessions and audio-stream selection, and publishes state and transcripts.
I also built the bridge from the installation’s RTP audio streams into the realtime speech transport, and worked on function-call handling and tool integration. Separate MCP and exhibition-control services execute the configured actions.
An interruption should mean stop
Cancelling a model response does not remove sound already waiting to play. I changed interruption handling to cancel generation and clear buffered output together, keeping the agent’s turn-taking consistent with the audio path.
02 / Component architecture
Inside the
AI platform.
Two interfaces connect to an orchestration service and the exhibition’s shared infrastructure. Controls, live audio and action requests follow separate paths.
Start with the four responsibilities inside one AI Orchestrator backend. The surrounding interfaces and services exchange controls, audio and action requests with it. Highlighting shows a connection path, not who authored each component.
Guide and admin commands pass through MQTT to the session controller. It coordinates the agent and stream selection, then publishes state and transcripts back to the interfaces.
The interfaces send audio to Janus over WebRTC. The selected input reaches the orchestrator’s input adapter, then its realtime transport and OpenAI Realtime. Response audio goes directly from the transport to Janus for playback, bypassing the input adapter.
The local agent session calls the MCP action service over HTTP. The service publishes a configured request through MQTT to exhibition control and returns a tool result. This confirms publication of the request, not completion of a display or physical action.
Follow an exhibition action
- Request. The conversation produces a tool call for a configured exhibition action.
- Dispatch. The local agent session calls the MCP (Model Context Protocol) service over HTTP. The model does not contact it directly.
- Publish. The MCP service checks the action list and sends its request through the MQTT message broker.
- Receive. Exhibition control receives the request. A returned tool result confirms request publication; it does not establish that the display or physical action has finished.
Frontends
Guide and operations interfaces
Guide interface
Push-to-talk, transcript and playback
Admin interface
Session controls and stream monitoring
AI Orchestrator
One Node.js backend service
Session controller
Commands, lifecycle and shared state
Agent session
Conversation, context and tool calls
Audio input adapter
Selected Janus stream and RTP input
Realtime transport
Speech, events and response audio
Exhibition services
Tool and action execution
MCP action service
Tools for configured action lists
Exhibition control
Configured displays and media
Shared infrastructure
Use-case settings
Session context and available tools
MQTT broker
Commands, state and transcripts
Janus audio gateway
Input streams and response playback
OpenAI Realtime
Speech and model responses
Read the connections
- Controls: guide/admin interfaces exchange commands, state and transcripts with the session controller through MQTT.
- Configuration: shared use-case settings supply session context and the available tool connections.
- Input: interfaces connect to Janus over WebRTC. The orchestrator’s input adapter receives the selected stream, and the coordinating layers forward its RTP packets into the realtime transport.
- Conversation: the transport exchanges speech and events with OpenAI Realtime over WebRTC. Response RTP/UDP audio goes directly back to Janus, bypassing the input adapter.
- Actions: the local agent session calls the MCP service over HTTP. That service publishes configured action requests through MQTT to exhibition control and returns tool results. A returned result does not verify that a display or physical action has finished.
03 / The guide interface
Speak. Listen.
Follow the exchange.
I also worked on the interfaces that make the platform usable: transcripts, listening feedback, and push-to-talk controls that release reliably when disabled or removed. They let a guide follow the exchange and take over.
Running assistant interface · Simulated local services; no audio streamed