This commit is contained in:
Javis committed 2026-10-10 13:20:35 +02:00
1 parent 4c16f87a3e
commit 2493c5c532
27 files changed
+8008 -568

No files matched your search

+6 -3
View File
@@ -494,9 +494,12 @@ The overnight scheduler runs `run_pipeline()` via `asyncio.to_thread`; the threa
on the training subprocess for hours. Cancelling the scheduler task (quit) released
the training lock immediately while the thread kept training, and at interpreter exit
Python joins executor threads — so Javis hung on quit until training finished.
**Fix:** cancellation calls `piper_train.request_stop()`, which terminates the child
process (`_run` now polls instead of blocking in `wait()`); the lock is only released
once the worker has actually finished.
**Fix:** cancellation calls `piper_train.request_stop()`: the training process finishes
its step, saves its progress — within the epoch too: the next run continues with the
next batch — and exits (it is killed if that takes longer than
`STOP_GRACE_SECONDS`), and the scheduler waits for that (`_run` polls instead of
blocking in `wait()`). The lock (`training/.training.lock`, held by `run_pipeline`)
names the training processes and is only released once they have exited.
#### VOI-01
**Imported voice archives may carry pickle checkpoints** — *Medium (trust)*