llama.cpp
20d05e19 - llama : add memory-mapped session cache support

Commit
247 days ago
llama : add memory-mapped session cache support Add support for memory-mapped session caches, both read-only and read-write. This provides efficient access to session files through memory mapping, which can reduce memory copies and improve performance for large session files. New features: - llama_state_mmap_open() - Open session file for read-only mmap access - llama_state_mmap_open_rw() - Open/create session file for read-write - llama_state_mmap_load() - Load state from mmap into context - llama_state_mmap_save() - Save state from context to mmap (rw only) - llama_state_mmap_sync() - Sync changes to disk (rw maps) - llama_state_mmap_free() - Free the mmap handle - llama_state_mmap_supported() - Check platform support Implementation details: - Adds llama_mmap_rw class supporting both read-only and read-write mmap - Uses POSIX mmap on Linux/macOS and CreateFileMapping on Windows - Read-write maps automatically resize when saving larger states - Changes are synced to disk on save and when freeing the handle
Author
Parents
Loading