# Javis — Local Voice AI Agent

Wake-word activated, Ollama-backed, MCP-enabled voice assistant with audiobook-driven voice cloning.

## Architecture

```
microphone ─▶ VAD / wake word ─▶ STT (faster-whisper)
                                          │
                                          ▼
                                Ollama LLM (llama3.1:8b) ◀─▶ MCP servers (filesystem, fetch, time, …)
                                          │
                                          ▼
                                  TTS (XTTS v2, cloned voice)
                                          │
                                          ▼
                                       speaker
```

- **STT** — faster-whisper, CUDA float16, default `large-v3`.
- **TTS** — Coqui XTTS v2 (community fork). Zero-shot cloning plus optional fine-tune from audiobook snippets.
- **LLM** — local Ollama at `http://localhost:11434`, `llama3.1:8b` (native tool calling).
- **MCP** — host/client only, reads `mcp_servers.json` (VS Code-compatible).
- **Wake word** — openWakeWord, default `hey_jarvis` (custom "Javis" model trainable later).
- **GUI** — PySide6 + qasync.

## Prerequisites

- Windows 10/11
- Python 3.10 or 3.11 (XTTS does not yet support 3.12)
- NVIDIA GPU + recent CUDA drivers (CPU works but slow)
- A running Ollama instance with the chosen model pulled:
  ```powershell
  ollama pull llama3.1:8b
  ```
- For MCP servers: Node.js (for `npx`) and/or `uv` (`uvx`).

## Install

```powershell
cd Javis
python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -e .           # all Python deps (a CPU torch may be pulled in as a fallback)

# Detects your GPU / driver and swaps in the matching PyTorch wheel (skip on CPU-only)
python -m javis --install-deps

copy config.example.yaml config.yaml
copy mcp_servers.example.json mcp_servers.json

# Verify: Ollama reachable, model pulled, CUDA/torch loadable, audio libs present
python -m javis --doctor
```

<details>
<summary>Manual install (if you prefer to pick the torch wheel yourself)</summary>

```powershell
# CUDA torch first (skip if CPU-only) — pick the index matching your driver
pip install torch torchaudio --index-url https://download.pytorch.org/whl/cu124
pip install -r requirements.txt
pip install -e .
```

If torch later fails to import (DLL errors), run `python -m javis --repair`.
</details>

## Run

```powershell
.\run.bat            # GUI
python -m javis      # GUI
python -m javis --headless   # no GUI (tool calls with approval "ask" are denied — set "allow" per server)
python -m javis --doctor     # check environment
```

First launch opens a configuration dialog (models, microphone with live level meter,
persona). Untick *Show this dialog on every start* to skip it next time; reopen it via
**Tools → Startup configuration…**. Closing the window hides Javis to the system tray so
wake word and timers keep running — quit via **File → Quit** or the tray menu.
Push-to-talk: `Ctrl+Space` in the window, `Ctrl+Alt+Space` system-wide (Windows).

## Voice profiles

1. Open the **Voices** panel.
2. Drag an audiobook file (mp3/wav/flac) in.
3. Javis will VAD-split, transcribe, and build a clean dataset.
4. Test with zero-shot cloning (instant) or click **Fine-tune** for higher quality.

## MCP servers

Edit `mcp_servers.json`. Schema matches VS Code's `mcp.json`, so you can copy/paste server entries.
First-time tool calls require approval (configurable per server).

## Custom wake word

The default wake phrase is *hey Jarvis*. To use your own name (e.g. **Rufus**), train a
small openWakeWord model on synthetic speech (< 1 h, Colab or WSL2), drop `rufus.onnx`
into `~/.javis/models/wakeword/` and set `wake_word.model: rufus`. Slaves fetch the model
from the master automatically. Step-by-step: [docs/WAKE_WORD.md](docs/WAKE_WORD.md).

## Satellites (multi-room)

[javis-slave](javis-slave/README.md) turns a Windows tablet or a **Raspberry Pi** into a
satellite: local wake word, audio streamed to this master over the LAN, spoken reply
played back locally. Enable `api.enabled: true` here, generate a pairing code in the
**Devices** tab, then follow [javis-slave/RASPBERRY_PI.md](javis-slave/RASPBERRY_PI.md)
for a headless Pi (`sudo ./setup-pi.sh`) or run `run.bat` on a tablet.
[javis-slave-mcp](javis-slave-mcp/README.md) exposes the satellites' volume/mute/power
controls as tools so you can say "mach die Küche leiser".

## Project layout

```
Javis/
  src/javis/
    core/        config, logger, state machine, event bus
    audio_io/    devices, mic stream, VAD, wake word
    stt/         faster-whisper
    tts/         backend interface, XTTS implementation
    voices/      profiles, audiobook ingest, fine-tune
    llm/         Ollama client, conversation memory
    mcp_client/  host, schema bridge
    agent/       orchestrator, system prompts
    gui/         PySide6 main window + panels
```

## Status

Initial scaffold. End-to-end voice loop works once dependencies and a voice reference clip are provided.