source: kdnuggets: 5 cool things i did with local language models
level: technical
running language models on your own machine means no api keys, no billing, and no data leaving your computer. the author used ollama to run models like llama 3.2 and mistral 7b on a laptop with 16 gb of ram. the first project built a private document brain with anythingllm, a local rag tool. it ingested research papers and answered cross-document questions without uploading files. the model cited sources and found a methodological disagreement the author had missed. this setup keeps sensitive documents like contracts or medical records completely private.
the second project used qwen2.5-coder 7b as a local code reviewer. the model caught sql injection, data exposure, and missing error handling in a sample function. it ran entirely offline, so proprietary code never left the machine. the author also integrated it with the continue plugin for vs code to get inline suggestions. the third project was an offline ai assistant for a long flight. by downloading mistral 7b before boarding, the author drafted emails, worked through a system design problem, and outlined an article at 35,000 feet with no internet. the model ran at 25 to 35 tokens per second on an m2 macbook pro.
the fourth project created a persistent thinking partner using a modelfile. this baked in context about the author's work, preferences, and current projects, so every session started with full awareness. the personalized model gave more direct and relevant responses than a context-free one. the fifth project explored building a local ai agent that uses tools, moving beyond text generation to planning and acting. these examples show local models can handle sensitive data, work offline, and adapt to personal workflows without cloud dependencies.
why it matters: local language models let ai and data science practitioners process sensitive data, work offline, and customize assistants without third-party risks or costs.
source: kdnuggets: 5 cool things i did with local language models