llama.cpp
75f92859 - llama : add copy-on-write mode for mmap session cache

Commit
329 days ago
llama : add copy-on-write mode for mmap session cache Add llama_state_mmap_open_cow() which reads session data from file but writes to anonymous memory (not persisted to disk). This is useful when you want to: - Load a base session cache and make temporary modifications - Process multiple requests from the same base session in parallel - Avoid modifying the original session file Also adds: - llama_mmap_mode enum to track mapping mode (READ_ONLY, READ_WRITE, COW) - mode() accessor to query the mapping mode - Support for larger-than-file writable buffers in COW mode
Author
Parents
Loading