spaethf/javis

0 Fork 0

Files

T

JavisandCopilot 1fa75e58fc End recording when the user stops, not the music

Silero VAD answers "is this voice-like?", which music, TV and podcasts
satisfy just as well as the user does. With Spotify playing through the
device's own speakers the VAD never went quiet, so record_utterance never
saw the end of the command and ran to the 20 s cap on master and slave
alike -- and music alone was enough to defeat the "no speech during the
grace window" early return.

Wrap the VAD in a SpeechEndpointer that tracks the background loudness of
the recent window and only reports speech while a frame is clearly louder
than it. Both wake-word loops feed their idle audio through observe(), so
the gate is already calibrated when a turn starts, and only non-voice
frames update the background so a long sentence cannot raise the gate onto
itself. Onset/release hysteresis keeps trailing quiet syllables.

The master owns the knob (wake_word.voice_gate_ratio, 0 = off) and pushes
it to satellites in the wake-word block.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

2026-10-10 13:04:13 +02:00

4.8 KiB

Raw Blame History

javis-slave

Satellite app for Javis. Runs on a tablet (designed for a Surface Pro 7) or headless on a Raspberry Pi, listens for the wake word locally, and sends recorded utterances to the Javis master over the LAN for transcription, LLM reasoning and speech synthesis. The reply is shown on screen (GUI mode) and played back through the device's speakers.

→ Raspberry Pi guide: RASPBERRY_PI.md

How it works

Pairing — On the master, open the Devices tab and click Generate pairing code. On the slave, enter the master's IP, port and the 6-digit code (setup screen, or --pair HOST:PORT CODE). The master issues a permanent per-device API key and the device appears in its Devices list.
Room assignment — In the master's Devices tab, type a room name for the device (e.g. Küche) and click Save room. The slave picks it up on its next config refresh; every utterance from this device is tagged [Room: Küche] for the LLM.
Wake word — The slave uses the same openWakeWord model name the master is configured with (sent during pairing / config refresh) and detects the wake word fully locally. Built-in models ship with openWakeWord; a custom model (e.g. rufus, see docs/WAKE_WORD.md) is downloaded from the master on first use. A short rising chime confirms detection.
Turns — After the wake word, the utterance is recorded locally (Silero VAD or RMS end-pointing), uploaded as WAV to POST /api/v1/turn, and the master's reply text + synthesized speech are returned and played. Recording ends when your voice stops: Silero rates music and TV as speech, so a loudness gate (wake_word.voice_gate_ratio, pushed by the master) only counts audio that is clearly louder than the background the wake-word loop measured. Without it, music from this device's own speakers kept the recorder running until the hard length cap. When the master marks a reply as expecting a follow-up, the slave starts listening immediately after playback without requiring another wake word. If the master's turn fails it returns its spoken error phrase instead, so a screenless satellite still gets audible feedback; connection problems play a low falling tone.

Setup

This folder is fully standalone — it can be copied on its own to a satellite device without the rest of the Javis repo. It vendors its own small copy of the audio stack (javis_slave/audio_io/), so there's no cross-project dependency.

On the master (config.yaml):

api:
  enabled: true
  host: 0.0.0.0
  port: 8765

On the slave:

run.bat                       :: Windows tablet (installs the GUI extra)

./run.sh                      # Linux with display
JAVIS_SLAVE_HEADLESS=1 ./run.sh --headless   # Linux without display
sudo ./setup-pi.sh            # Raspberry Pi: apt deps + venv + systemd service

The first run creates .venv and installs the dependencies declared in pyproject.toml. Optional extras: gui (PySide6/qasync, needed for the window) and vad (silero-vad + PyTorch; without it an RMS energy gate is used).

Command line

javis-slave                                      touch GUI
javis-slave --fullscreen | --kiosk               fullscreen / fullscreen + hidden cursor
javis-slave --headless                           no window (services, Pi)
javis-slave --pair 192.168.1.10:8765 123456 --name Kueche
javis-slave --list-devices                       show mic/speaker indices
javis-slave --input-device 1 --output-device 0 --input-gain 1.5 --api-port 8766
javis-slave --show-control-token                 print the control-API token
javis-slave --control-token <TOKEN>              set your own token (≥ 16 chars)

Settings are stored in %LOCALAPPDATA%\JavisSlave\slave_config.json (Windows) or ~/.javis-slave/slave_config.json (Linux); override the folder with JAVIS_SLAVE_CONFIG_DIR. A rotating log is written to <config dir>/logs/slave.log.

Notes

Transport is plain HTTP — use on a trusted LAN only.
The control API on port 8766 (volume, power, mic, …) requires authentication: Authorization: Bearer <token> from other machines, or the header X-Javis-Control: 1 from the same machine (loopback). The token is generated on first start and stored in slave_config.json (mode 0600 on Linux); show it with --show-control-token or in the GUI settings. Only GET /mcp/v1/health is public.
Microphones/speakers that don't support 16 kHz are opened at their native rate and resampled automatically (common for USB audio on Linux).
System control endpoints (volume / brightness / power, see javis-slave-mcp) work on Windows (pycaw, screen-brightness-control) and Linux (pactl/amixer, /sys/class/backlight, systemctl).

4.8 KiB Raw Blame History