Documentation

attention labs SDKs stream your mic and webcam to the hosted server and return typed events: silent, human-directed, or device-directed. No wake word. You choose what reaches your language model.

Overview

Two clients share the same surface, @attenlabs/saa-js for the browser and attenlabs-saa for Python. Both stream to a managed, versioned model and return typed events; routing to your language model is up to you.

Install

# JavaScript
npm install @attenlabs/saa-js

# Python
pip install attenlabs-saa

View the SDK on GitHub

The JavaScript client needs a browser (getUserMedia, AudioContext, AudioWorklet, Canvas, WebSocket). The Python client needs 3.10+ and installs sounddevice and opencv-python automatically.

Authentication and connect

Sign up for the dashboard, create an API key, and pass it to the client as the token option. Keys are self-serve and available immediately, scoped to your account. Keep yours secret: load it from an environment variable rather than hardcoding it.

// JavaScript
const client = new AttentionClient({ token: "your-token" });

# Python
client = AttentionClient(token=os.environ["SAA_API_KEY"])

When you call start(), the SDK resolves a connection in two steps:

1. Allocate. The client sends a POST to the broker's /allocate endpoint (the broker base defaults to https://broker.attentionlabs.ai; override it with the url option). The token is sent in the Authorization: Bearer <token> header. The broker picks a backend and responds with JSON containing a url field, the WebSocket URL for your session.
2. Connect. The client opens a WebSocket to that returned url, passing the token again as the WebSocket subprotocol. Mic and webcam media stream over this socket and typed events come back on it.

The allocate step runs once per connect, so each reconnect requests a fresh backend. If you pass a ws:// or wss:// URL as the url option, the SDK treats it as a direct backend and skips the broker step.

Core concepts

Four ideas explain everything else in these docs.

Hosted, not local

Both clients are thin. They capture mic and webcam, stream frames to the hosted server through the broker, and hand you typed events back. No model runs in your process, and there's nothing to version or update yourself.

Three classes, one threshold

Every prediction resolves to a class: 0 silent, 1 human-directed, 2 device-directed. initialThreshold (default 0.7) sets how confident the model must be before it commits to class 2. Call setThreshold() (JavaScript) or set_threshold() (Python) to tune it while the client is running.

Prediction and turn events

prediction fires on every frame while the model classifies live audio and video; it's what you gate a live stream on. turnReady fires once per completed utterance, with the assembled clip ready to send. Most integrations use both: prediction to hold a gate open or closed, turnReady to grab the audio that goes to the LLM.

Fail closed

Only class 2 should ever reach your language model. If the socket drops, an error fires, or a prediction is inconclusive, treat it the same as human-directed: keep the gate closed rather than forward audio you're not sure about. Missing an occasional device-directed turn is safer than leaking a side conversation to your model.

Quickstart

Create a client, listen for prediction and turnReady, and start streaming. cls tells you who is speaking: 0 silent, 1 human-directed, 2 device-directed. Only 2 should reach your model.

// JavaScript
import { AttentionClient } from "@attenlabs/saa-js";

const client = new AttentionClient({ token: "your-token" });

client.on("prediction", (e) => {
  const label = { 0: "silent", 1: "human", 2: "device" }[e.cls] ?? "?";
  console.log(`${label}  ${(e.confidence * 100).toFixed(0)}%  faces=${e.numFaces}`);
});

client.on("turnReady", (e) => {
  console.log(`turn ready (${e.durationSec.toFixed(2)}s)`);
  // e.audioBase64 is PCM16 @ 16 kHz, ready to forward to your LLM
});

const video = document.getElementById("preview");
await client.start({ videoElement: video });

# Python
import os, time
from saa import AttentionClient

client = AttentionClient(token=os.environ["SAA_API_KEY"])

@client.on_prediction
def _(event):
    label = {0: "silent", 1: "human", 2: "device"}.get(event.cls, "?")
    print(f"{label}  {event.confidence:.0%}  faces={event.num_faces}")

@client.on_turn_ready
def _(event):
    print(f"turn ready ({event.duration_sec:.2f}s)")
    # event.audio_base64 is ready to forward to your LLM

client.start()
try:
    while True:
        time.sleep(0.1)
except KeyboardInterrupt:
    client.stop()

From here, follow the full references for the complete constructor options, the shared method surface, every event and its payload type, and the LLM mute pattern: JavaScript SDK and Python SDK.

Integrations

The Cloud SDK hands each device-directed turn to common voice-agent frameworks through an adapter, in one of two shapes:

• Existing pipeline. The adapter gates it in place: device-directed speech passes through, side conversation is held back.
• New agent. The adapter hands each completed turn to your onTurn callback as a TurnReadyEvent.

engine.on("prediction", (p) => {
  // enable input only for device-directed speech (class 2)
  session.input.setAudioEnabled(p.alignedClass === 2);
});

engine.on("interrupt", () => session.interrupt());

engine.on("prediction", (p) => {
  // hold back person-directed speech before it reaches STT
  addresseeGate.suppressed = p.alignedClass === 1;
});

const attn = new SAAFeedAudioInterface(baseInterface, saa, { gate: true });

saa.on("prediction", (e) => {
  attn.setGateOpen(e.cls === 2); // only device-directed speech reaches the agent
});

engine.on("prediction", (p) => {
  // gate input before it reaches the realtime model
  session.input.setAudioEnabled(p.alignedClass === 2);
});

Every adapter reports the active speaker's aligned class the same way the core SDK does: 0 silent, 1 person-directed, 2 device-directed. Only class 2 should reach your language model.

Errors and limits

Both clients emit a typed error event, error in JavaScript and @client.on_error in Python, carrying a title, message, and the WebSocket close code when applicable. The most common title is "Auth Failed"; generate a fresh key in the dashboard if yours is missing, expired, or revoked.

• Audio: 16 kHz mono, streamed in 100 ms (1600-sample) chunks.
• Video: JPEG frames at about 4 fps; frames drop under backpressure (WebSocket send buffer over 1 MB) rather than blocking audio.
• Reconnects: every reconnect re-runs the broker's allocate step, so a dropped connection always lands on a fresh backend, never a stale one.

Full detail lives on the client references: JavaScript errors and the shared event-type reference.

Need help integrating? Write to [email protected].