v1.0
This commit is contained in:
1 parent
0246cbb6ba
commit
0ca4b4bf7a
89 files changed
+7042
-2480
No files matched your search
@@ -5,21 +5,22 @@ Wake-word activated, Ollama-backed, MCP-enabled voice assistant with audiobook-d
|
||||
## Architecture
|
||||
|
||||
```
|
||||
microphone ─▶ VAD / wake word ─▶ STT (faster-whisper)
|
||||
microphone ─▶ VAD / wake word ─▶ STT (faster-whisper) + speaker ID (ECAPA)
|
||||
│
|
||||
▼
|
||||
Ollama LLM (llama3.1:8b) ◀─▶ MCP servers (filesystem, fetch, time, …)
|
||||
quick commands ◀── Ollama LLM ◀─▶ MCP servers (filesystem, fetch, time, …)
|
||||
│
|
||||
▼
|
||||
TTS (XTTS v2, cloned voice)
|
||||
TTS (Piper VITS, or F5-TTS cloning)
|
||||
│
|
||||
▼
|
||||
speaker
|
||||
```
|
||||
|
||||
- **STT** — faster-whisper, CUDA float16, default `large-v3`.
|
||||
- **TTS** — Coqui XTTS v2 (community fork). Zero-shot cloning plus optional fine-tune from audiobook snippets.
|
||||
- **LLM** — local Ollama at `http://localhost:11434`, `llama3.1:8b` (native tool calling).
|
||||
- **TTS** — Piper (VITS, ONNX) by default, fine-tunable overnight from audiobook snippets;
|
||||
optional F5-TTS zero-shot cloning (`tts.backend: f5`, non-commercial weights).
|
||||
- **LLM** — local Ollama at `http://localhost:11434` (native tool calling).
|
||||
- **MCP** — host/client only, reads `mcp_servers.json` (VS Code-compatible).
|
||||
- **Wake word** — openWakeWord, default `hey_jarvis` (custom "Javis" model trainable later).
|
||||
- **GUI** — PySide6 + qasync.
|
||||
@@ -27,7 +28,7 @@ microphone ─▶ VAD / wake word ─▶ STT (faster-whisper)
|
||||
## Prerequisites
|
||||
|
||||
- Windows 10/11
|
||||
- Python 3.10 or 3.11 (XTTS does not yet support 3.12)
|
||||
- Python 3.10 or 3.11 (tested on 3.10; `piper-tts` is pinned to the 1.x line)
|
||||
- NVIDIA GPU + recent CUDA drivers (CPU works but slow)
|
||||
- A running Ollama instance with the chosen model pulled:
|
||||
```powershell
|
||||
@@ -118,8 +119,8 @@ Javis/
|
||||
core/ config, logger, state machine, event bus
|
||||
audio_io/ devices, mic stream, VAD, wake word
|
||||
stt/ faster-whisper
|
||||
tts/ backend interface, XTTS implementation
|
||||
voices/ profiles, audiobook ingest, fine-tune
|
||||
tts/ backend interface, Piper + F5 implementations
|
||||
voices/ profiles, audiobook ingest, Piper training + scheduler
|
||||
llm/ Ollama client, conversation memory
|
||||
mcp_client/ host, schema bridge
|
||||
agent/ orchestrator, system prompts
|
||||
@@ -128,4 +129,13 @@ Javis/
|
||||
|
||||
## Status
|
||||
|
||||
Initial scaffold. End-to-end voice loop works once dependencies and a voice reference clip are provided.
|
||||
End-to-end voice loop works once dependencies and a voice model/reference clip are provided.
|
||||
See [docs/AUDIT.md](docs/AUDIT.md) for the latest code audit.
|
||||
|
||||
## Tests
|
||||
|
||||
```powershell
|
||||
pip install -e .[dev]
|
||||
python -m pytest tests -q --ignore=tests/_retheme_check.py
|
||||
$env:QT_QPA_PLATFORM="offscreen"; python tests\_retheme_check.py # GUI smoke test
|
||||
```
|
||||
Reference in new issue
Block a user