source: kdnuggets: run the mythos enhanced coding model locally with llama.cpp and pi
level: technical
qwythos-9b-claude-mythos-5-1m is a 9b reasoning and coding model based on qwen3.5, designed for local coding workflows and agentic development. it is small enough to run on consumer hardware while handling practical coding tasks. this guide shows how to run the model locally with llama.cpp and connect it to pi, a coding agent. the setup uses an rtx 4070 ti super with 16gb vram and the q6_k mtp quantization. for 8gb gpus, the q4_k_m variant is recommended.
first, install llama.cpp cli and set a hugging face cache directory. then start the model with llama serve, using flags like --n-gpu-layers all, --ctx-size 100000, and --flash-attn on. the command enables mtp speculative decoding with up to six tokens per step. once running, the model provides an openai-compatible api at localhost:8910/v1. on the test gpu, it achieved about 81.74 tokens per second. the model can reason through tasks and call tools within agent workflows.
next, install pi and the pi-llama plugin to connect to the local server. set the llama_base_url environment variable to point to the api. inside pi, select the model alias to use it as a coding agent. testing included building a browser game and a csv-to-excel python cli. pi created a complete html game with timer, score, and multiple question types, and a python script that converted csv to xlsx with error handling. both tasks ran successfully, showing the model's ability to handle front-end and scripting tasks locally without external apis.
why it matters: this setup lets developers run capable coding models on local hardware, reducing reliance on cloud apis and enabling private, cost-free agentic workflows.
source: kdnuggets: run the mythos enhanced coding model locally with llama.cpp and pi