javis/docs/WAKE_WORD.md

# Custom wake word ("Rufus")

Javis uses [openWakeWord](https://github.com/dscripka/openWakeWord). The built-in
models are English phrases (`hey_jarvis`, `alexa`, `hey_mycroft`, `hey_rhasspy`,
`timer`, `weather`). Anything else — e.g. **Rufus** — is a small custom `.onnx`
classifier that you train once on synthetic speech (no recordings of your own
voice needed) and drop into Javis. Slaves download it from the master
automatically.

## 1. How Javis loads wake-word models

`wake_word.model` in `config.yaml` accepts three forms:

| Value | Meaning |
|-------|---------|
| `hey_jarvis` | built-in openWakeWord model |
| `rufus` | custom model at `<paths.base_dir>/models/wakeword/rufus.onnx` (default `~/.javis/models/wakeword/rufus.onnx`) |
| `D:/models/rufus.onnx` | explicit path to a custom model |

The detector keys predictions by the file stem, so the file **must** be named
`<name>.onnx`. Slaves receive the bare name (`rufus`) from the master and fetch
the file via `GET /api/v1/wake_word/model` on first use, caching it under
`<slave config dir>/models/rufus.onnx`.

## 2. Train the model

Training runs on **Linux** (WSL2 on Windows works) or free Google Colab; it needs
a GPU for the ~10 min synthetic-speech step but finishes in well under an hour.

### Option A — Colab (no setup)

Open openWakeWord's
[automatic_model_training.ipynb](https://github.com/dscripka/openWakeWord/blob/main/notebooks/automatic_model_training.ipynb)
in Colab, run the cells top to bottom, and change only the config cell:

```python
config["target_phrase"] = ["rufus", "hey rufus", "roofus", "hey roofus"]
config["model_name"]    = "rufus"
config["n_samples"]     = 5000      # positives; more = better, 20k+ for production
config["n_samples_val"] = 1000
config["steps"]         = 20000
```

Download `my_custom_model/rufus.onnx` when it finishes.

### Option B — locally (WSL2 / Linux, reproducible)

```bash
# one-time environment
python3 -m venv ~/oww && source ~/oww/bin/activate
git clone https://github.com/dscripka/openwakeword && pip install -e ./openwakeword
git clone https://github.com/rhasspy/piper-sample-generator
wget -O piper-sample-generator/models/en_US-libritts_r-medium.pt \
  https://github.com/rhasspy/piper-sample-generator/releases/download/v2.0.0/en_US-libritts_r-medium.pt
pip install piper-phonemize webrtcvad mutagen torchinfo torchmetrics speechbrain \
            audiomentations torch-audiomentations acoustics pronouncing datasets deep-phonemizer

# data: room impulse responses, background noise, pre-computed negative features
# (copy the "Download Data" cells from the notebook — ~3 GB total)

# config
cp openwakeword/examples/custom_model.yml rufus.yml
#   target_phrase: ["rufus", "hey rufus", "roofus", "hey roofus"]
#   model_name: rufus
#   n_samples: 5000  n_samples_val: 1000  steps: 20000
#   background_paths: ["./audioset_16k", "./fma"]
#   false_positive_validation_data_path: validation_set_features.npy
#   feature_data_files: {ACAV100M_sample: openwakeword_features_ACAV100M_2000_hrs_16bit.npy}

python openwakeword/openwakeword/train.py --training_config rufus.yml --generate_clips
python openwakeword/openwakeword/train.py --training_config rufus.yml --augment_clips
python openwakeword/openwakeword/train.py --training_config rufus.yml --train_model
# → my_custom_model/rufus.onnx
```

### Tips that matter for a German household

- **Spell out the German pronunciation.** The synthetic voices are English; add
  phonetic variants to `target_phrase` so the model hears the "u" you actually
  say: `"roofus"`, `"ruufus"`. Include both the bare name and `"hey <name>"`.
- **Add adversarial negatives.** `custom_negative_phrases` in the YAML (e.g.
  `["rufen", "ruf mal", "ruhe", "bus", "roof"]`) sharply cuts false wakes on
  similar-sounding words.
- **Two-syllable names with a hard consonant** ("Rufus", "Jarvis") work far
  better than short or vowel-heavy ones.
- Short names false-trigger more: expect to run `threshold: 0.6–0.7` rather
  than the default 0.5. Tune in **Settings → Wake Word**; the Status tab shows
  the live score.

## 3. Install in Javis

```powershell
mkdir $env:USERPROFILE\.javis\models\wakeword
copy rufus.onnx $env:USERPROFILE\.javis\models\wakeword\
```

`config.yaml`:

```yaml
wake_word:
  model: rufus
  threshold: 0.6
```

Restart Javis. The log shows `openWakeWord loaded (…\rufus.onnx, key=rufus, …)`.
Paired slaves pick up the new name on their next config poll (≤ 60 s), download
the model from the master and reload the detector — nothing to copy by hand.
For a slave that cannot reach the master at that moment, place the file at
`~/.javis-slave/models/rufus.onnx` (or `%LOCALAPPDATA%\JavisSlave\models\`).

## 4. Verify

- Say the name at normal speaking distance: the wake chime plays and the
  Status tab's *Wake score* bar jumps. Watch for a minute of normal
  conversation without triggers; if it fires spuriously, raise the threshold or
  retrain with more negatives.
- Quick offline check without Javis:

  ```powershell
  python -c "from openwakeword.model import Model; m=Model(wakeword_models=[r'%USERPROFILE%\.javis\models\wakeword\rufus.onnx'], inference_framework='onnx'); print(m.predict_clip(r'test.wav'))"
  ```

## Troubleshooting

| Symptom | Cause / fix |
|---------|-------------|
| `wake word model 'rufus' is neither a built-in … nor a custom .onnx` | File missing or misnamed — it must be `rufus.onnx` in `models/wakeword/`. |
| Slave says `Wake-Word-Modell 'rufus' konnte nicht geladen werden` | Master unreachable during download or master itself has no custom file; check `GET http://master:8765/api/v1/wake_word/model` returns 200 with slave credentials. |
| Fires on TV / music | Raise `threshold`; retrain with more `background_paths` hours and `custom_negative_phrases`. |
| Misses when spoken quickly / far away | Add phrase variants, more `n_samples` (20k+), and set `augmentation_rounds` higher in the YAML. |