Principal-speaker gating for service robots
Your robot hears everyone. Now it knows who it is supposed to answer.
Service robots operate in lobbies, hospital corridors, restaurant floors, and warehouses where voices come from every direction.
Selective Auditory Attention (SAA) is the hosted, pre-ASR layer that decides which person is addressing the robot, in real time, before dialogue management runs, so your robot engages only the person actually addressing it.
The failure
In a shared space with multiple people, a robot cannot tell who it is supposed to answer.
A service robot is positioned in a hospital corridor. A nurse approaches to request a supply delivery. Two visitors walk past mid-conversation, and a PA announcement plays from the ceiling. The robot hears all four audio sources simultaneously and has no principled way to determine which one is addressing it.
A bystander's aside gets executed as a command. The nurse who actually needed help is ignored. In a regulated environment, that is not a UX failure: it is an accountability gap. And it is reproducible on demand in any room with more than one person.
ghost activation
The robot answers the bystander
A visitor two feet away finishes a sentence near the robot. The robot executes it as a command. The person who actually needed assistance is ignored, trust collapses, and in a regulated setting, staff have to override manually.
missed command
The real operator gets no response
The nurse's instruction is buried in simultaneous ambient audio. The robot's ASR receives three overlapping sources and cannot determine priority. The person with the actual need waits, repeats, or escalates to a human, erasing the deployment's productivity case.
The fix
One hosted layer that decides who the robot is addressing.
SAA is a hosted, pre-ASR engagement-control layer that decides, in real time before speech-to-text, whether speech in the robot's environment is addressed to the device or to another person in the room.
It routes only addressed audio to the ASR and dialogue manager you already run, fails closed when it is unsure, and needs no wake word and no GPU.
Research foundation
Built on our published technical paper.
SAA reliably classifies which person is addressing the robot across held-out multi-party sessions, on the noisy, far-field floor your robot operates on, whether using audio and video fusion or audio alone (arXiv:2604.08412).
Two caveats apply either way: (a) the system fails closed under distribution shift; (b) cross-lingual recall is a known limitation under active work.
performance
High accuracy with audio and video fusion
Held-out multi-party sessions. Published at arXiv:2604.08412. Fails closed under distribution shift; cross-lingual recall is a known limitation under active work.
performance
High accuracy with audio only
Real multi-speaker audio, no video stream. Same two caveats apply: fails closed under distribution shift; cross-lingual recall is a known limitation.
latency
Real-time decision
The routing decision is returned in real time, before ASR begins. No GPU required on the robot hardware. Fits inside a natural conversational exchange without perceptible delay.
design principle
Fails closed by design
Ambiguous speech is suppressed, not forwarded. Under uncertainty the default is silence, not a false command. In an environment where a wrong action can have physical consequences, that property matters.
coverage
Up to 4 simultaneous speakers
Addressee detection in multi-party environments with up to four speakers active at once, matching the group sizes a service robot faces in a lobby or corridor.
signal model
Acoustic and prosodic; no wake word
No trigger phrase required. Language-agnostic at the acoustic level. The principal speaker is identified by engagement signal, not by vocabulary. Cross-lingual recall parity is a known open item under active work.
How it integrates
Where SAA sits in your robot voice stack.
SAA runs between the robot's microphone array and ASR. Raw audio streams to the hosted service, SAA returns an addressee decision in real time, and only audio addressed to the robot is forwarded to the recognizer and dialogue manager.
No wake word. No GPU. No change to your downstream NLU, action planner, or ROS stack.
Audio streams from the microphone array
Raw audio from the robot's microphone array streams to the SAA hosted service before any ASR or dialogue management processes it. No change to the robot's hardware, microphone, or mechanical platform.
SAA decides who is addressing the robot
SAA analyzes acoustic and prosodic signals across all voices in range. No wake word required. No GPU required. In real time it classifies whether speech is addressed to the robot or directed elsewhere, including group scenarios with up to four simultaneous speakers.
Only addressed audio reaches the dialogue manager
Audio classified as device-directed is forwarded to your ASR and dialogue manager. Bystander speech, side conversations, PA announcements, and ambient noise are suppressed before ASR runs. Under uncertainty SAA fails closed and forwards nothing.
Real time, behind any dialogue stack
The decision returns in real time, fitting inside a natural conversational exchange. SAA runs as a hosted service on GCP and is licensed per device. A DPA is available for regulated deployments. You keep your ASR, your dialogue manager, your ROS stack, and your robot hardware unchanged.
architecture
Pre-ASR, hosted, behind any dialogue manager
Client SDKs in JavaScript/TypeScript and Python stream microphone audio to server.attentionlabs.ai and receive the addressee decision before transcription begins. SAA is model-agnostic: it works in front of any ASR and any dialogue manager you already license, whether that is a custom ROS controller, a cloud language model, or an embedded action planner. No wake word. No GPU. A DPA is available for regulated deployments.
access
Eval tier is free. Licensing is per device.
Book a call to scope the integration with your stack, or request an eval to run SAA on audio from your own robot deployment and measure addressee decisions.
Book a callWhy existing tools miss it
Beamforming and wake words do not decide who is addressing the robot.
Each technique below solves a real problem in a robot voice stack. None of them answers the question the robot needs answered before it responds: is this person addressing me right now?
That decision has to happen before ASR, or every voice in range reaches the part of the pipeline that acts on it.
| Technique | What it answers | When it runs | Decides who is addressed? |
|---|---|---|---|
| Noise cancellation and beamforming | Is the audio cleaner? Where is the source spatially? | Pre-ASR | No |
| Voice activity detection (VAD) | Is someone speaking? | Pre-ASR | No |
| Speaker diarization | Who spoke, and when? | Post-ASR | No |
| Wake word | Was a trigger phrase said? | Pre-ASR | Phrase only, not intent |
| Speaker identification | Is this a known, enrolled voice? | Post-ASR | No |
| SAA addressee detection | Is the robot being addressed right now? | Pre-ASR, real time | Yes |
Robot OEM and platform teams who need the right person to get a response, every time, across every shared environment where the robot is deployed.
Common questions
What robot teams ask before integrating.
How does a robot know which person in a group is addressing it?
Without a dedicated addressee layer, it cannot reliably determine this. Voice activity detection detects that someone is speaking. Direction of arrival locates the nearest sound source. Wake words trigger on a specific phrase. None of these answer whether the person speaking right now is addressing the robot or a nearby bystander. SAA adds that decision in real time, before ASR or dialogue management runs, using acoustic and prosodic signals. When it is uncertain, it fails closed and the robot stays silent.
How does SAA handle multiple people approaching the robot at the same time?
SAA evaluates each audio segment for addressee signals, not just loudness or proximity. When two or more people speak simultaneously near the robot, SAA gates on the segment showing addressee-consistent acoustic and prosodic features directed toward the device. Non-addressed speech is suppressed. In high-ambiguity situations the system fails closed, keeping the robot silent rather than acting on an utterance that was not clearly addressed to it.
Is SAA hosted or local, and what is the latency?
SAA is a hosted cloud service running on GCP. Audio from the robot's microphone array streams to server.attentionlabs.ai via a thin client SDK in JavaScript/TypeScript or Python. The addressee decision returns in real time. No GPU is required on the robot hardware. A Data Processing Agreement is available for regulated deployments. Local or embedded deployment is a separate enterprise license and is not the standard hosted path.
How does SAA integrate with a dialogue manager or a ROS-based voice stack?
SAA sits between the microphone array and ASR as a pre-ASR stage. The client SDK streams raw audio to the hosted service and receives a routing decision. Only audio classified as device-directed is forwarded to your ASR and then to your dialogue manager, whether that is a custom ROS controller, a cloud language model, or an embedded action planner. No changes to the downstream dialogue layer are required. The integration adds one pre-ASR stage.
Does SAA work in far-field, noisy environments like hospital corridors or busy lobbies?
SAA is trained and evaluated on real multi-party audio in noisy, far-field environments with up to four simultaneous speakers. On held-out multi-party sessions it reliably identifies whether speech is addressed to the robot, whether using audio and video fusion or audio alone (arXiv:2604.08412). Two caveats apply either way: (a) it fails closed under distribution shift; (b) cross-lingual recall is a known limitation under active work. The right next step is an evaluation on audio from your own deployment environment. The evaluation tier is free.
Does SAA log which utterances were addressed to the robot?
SAA logs the addressee decision (device-directed vs not) and its confidence per routed utterance. SAA does not perform speaker identification or diarization, and does not name or enroll individuals.
Get in touch
Book a call or request an eval.
Tell us about your robot deployment: the environment it operates in, the voice stack it runs, and the addressee failure you are seeing.
We will scope the integration with your stack and set up a free evaluation on audio from your own deployment. Licensing is per device.
Last updated June 26, 2026