docs: Add comprehensive FFI architecture documentation (#2648)
* docs: reorganize architecture docs for FFI and legacy runtimes
Restructure architecture documentation to clearly separate the two runtime
implementations (legacy Python/FastAPI and experimental Rust/PyO3 FFI).
Changes:
- Move existing runtime docs to architecture/legacy/
- Create new architecture/ffi/ with comprehensive FFI documentation
- Add README files for both runtime implementations
- Update architecture/00-overview.md to explain both runtimes
- Document FFI components: PredictionService, Supervisor, PermitPool
- Document IPC protocol: control channel and slot channels
- Document health state machine and prediction flows
- Add migration notes and performance comparisons
- Update docs/http.md and README.md to mention USE_COGLET option
The FFI runtime provides significant improvements: faster HTTP layer,
better worker isolation, slot-based concurrency, and automatic
cancellation on connection drops.
* docs: fix FFI architecture docs to match crate READMEs
- Fix cancellation propagation to show sync (SIGUSR1 + CancelableGuard) vs
async (future.cancel()) mechanisms instead of incorrect cancel_token
- Fix control channel protocol messages and fields
- Fix file structure (worker.rs not worker/ subdirectory)
- Remove incorrect cancel_token from component ownership
* chore: add rust to mise
* chore: default profile mise?
Signed-off-by: Mark Phelps <mphelps@cloudflare.com>
* docs: fix health-check response documentation per PR review
- Health-check endpoint always returns HTTP 200 with status in JSON body
- Clarify that 503/409 responses only apply to prediction endpoints
- Mark worker restart as future potential behavior (not yet implemented)
- Update mermaid diagram notes to be more accurate
* chore: update AGENTS.md
Signed-off-by: Mark Phelps <mphelps@cloudflare.com>
* chore: remove dupl rust install in ci
Signed-off-by: Mark Phelps <mphelps@cloudflare.com>
* fix: explicitly install rustfmt and clippy components in mise
The 'default' profile doesn't reliably include rustfmt in CI, causing
'cargo-fmt is not installed' errors. Explicitly specifying the components
ensures they are always available.
---------
Signed-off-by: Mark Phelps <mphelps@cloudflare.com>
