llama.cpp server adds multi-model management
Original title:New in llama.cpp: Model Management
AI summary
The llama.cpp server has added multi-model management similar to Ollama. The feature uses a multi-process architecture, with each model running independently to ensure that a crash in one model does not affect other services. The system supports automatic discovery of local GGUF model files, on-demand loading, and by default uses an LRU mechanism to manage up to four models loaded simultaneously. Users can route to a specific model via the model field in requests and can use the API to load, unload, and list models. All loaded models can inherit the router's unified settings, and parameters can also be configured individually for each model via preset files. The built-in Web UI also supports model switching.
Key Takeaways
- The llama.
- cpp server has added multi-model management similar to Ollama.
- The feature uses a multi-process architecture,with each model running independently to ensure that…
- The system supports automatic discovery of local GGUF model files,on-demand loading,and by default…
- Users can route to a specific model via the model field in requests and can use the API to…
Quick Facts
- Published
- Updated
- Source
- Hugging Face 社区博客(混合发现)
- Category
- Other
- AI score
- 76
本地跑模型终于能像 Ollama 一样热切换,开发调试效率大幅提升
Related entities
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: llama.cpp server adds multi-model management. December 11, 2025. https://new.nat6.net/en/news/cmoegbhak00a4slxx4drrrsd4/