ji
This commit is contained in:
1 parent
4c16f87a3e
commit
2493c5c532
27 files changed
+8008
-568
No files matched your search
+6
-3
@@ -494,9 +494,12 @@ The overnight scheduler runs `run_pipeline()` via `asyncio.to_thread`; the threa
|
||||
on the training subprocess for hours. Cancelling the scheduler task (quit) released
|
||||
the training lock immediately while the thread kept training, and at interpreter exit
|
||||
Python joins executor threads — so Javis hung on quit until training finished.
|
||||
**Fix:** cancellation calls `piper_train.request_stop()`, which terminates the child
|
||||
process (`_run` now polls instead of blocking in `wait()`); the lock is only released
|
||||
once the worker has actually finished.
|
||||
**Fix:** cancellation calls `piper_train.request_stop()`: the training process finishes
|
||||
its step, saves its progress — within the epoch too: the next run continues with the
|
||||
next batch — and exits (it is killed if that takes longer than
|
||||
`STOP_GRACE_SECONDS`), and the scheduler waits for that (`_run` polls instead of
|
||||
blocking in `wait()`). The lock (`training/.training.lock`, held by `run_pipeline`)
|
||||
names the training processes and is only released once they have exited.
|
||||
|
||||
#### VOI-01
|
||||
**Imported voice archives may carry pickle checkpoints** — *Medium (trust)*
|
||||
|
||||
Reference in new issue
Block a user