llama-cpp

Run local GGUF model inference by using this skill to discover models from the Hugging Face Hub and generate exact commands for server deployment. It automates quantization selection and command reconstruction for llama.cpp.

Choosing the right quantization for local LLMs is handled automatically by evaluating your hardware constraints. The skill selects the optimal Q4, Q5, or Q6 quantization based on your specific CPU and GPU architecture to maximize performance.

Yes, you can use llama-cpp-python to deploy an OpenAI-compatible server locally. The skill provides automated workflows and exact command reconstruction to launch the server with your selected GGUF models.

The best way to find compatible GGUF models on Hugging Face Hub is using this skill's model discovery feature. It searches the Hub, traverses model trees, and identifies models compatible with your local hardware constraints for llama.cpp.

Yes, this skill supports hardware optimization across different CPU and GPU architectures. It provides hardware-aware quantization selection and performance optimization specifically tailored to your machine's capabilities.

Yes, you need llama-cpp-python installed to use this skill for local inference. It is a required dependency for integrating the model discovery, quantization selection, and server deployment workflows into your local environment.