Every week, I organize 400 to 700 links into roughly 60 categories as part of my ongoing effort to learn about AI. This is my personal notebook, which I enjoy sharing with friends… a hobby and a labor of love, rather than a commercial publication or product.
If you arrived here through a search or shared link, this page collects the links I found for Locally Run for the week ending September 4, 2026.
As part of my learning process, I like to automate the category covers. It gives me a chance to learn Python and APIs.
This week’s cover prompt was written using Claude Opus 4.7, and the image was generated using Nano Banana 2.1.
Category Cover Image Prompt: A psychedelic screen-print gig poster of a personal desktop computer tower cracked open to reveal a glowing GPU radiating swirling op-art feedback patterns, cables morphing into vines and cooling fans into mandalas, a cozy bedroom window with harvest leaves drifting in the background, bold hand-lettered title Local arcing across the top in Fillmore-era psychedelic letters, electric magenta acid orange deep teal and cream palette with lime green accents, flat ink layers with slight registration offset and visible paper texture, ornate decorative border.
This Week in Locally Run News
Summary by Claude Sonnet 5.5 based on this week’s links:
- Big open models shrink to fit: GLM-5.3 was the local-run headline. Unsloth said its 2-bit version drops from 1.51TB to 239GB while keeping roughly 81% accuracy, small enough for a 256GB Mac. Another post estimated a 2-bit setup at about 230 to 250GB, versus 10 to 12 H100s for FP8.
- Nvidia pushes local AI hardware: Nvidia had several local items. Wired covered the RTX Spark AI PCs, Nvidia promoted a Personal AI Router for local inference, and its Jetson tutorial covers fine-tuning with Unsloth and QLoRA, then running quantized GGUF files in llama.cpp.
- Tooling and practical lessons: Support for local models keeps spreading. Hermes added a local backend for several Unsloth quants, and Ollama added a launcher for the Muse Code harness. One user warned that a turboquant kv cache hurt results and recommended BF16, with CPU offload to save VRAM.
This summary was generated by Claude Sonnet 5.5 to help you explore the links below. Rest assured, I select, organize, and check the links by hand in Google Sheets, and write the introduction and personal commentary in The Main Newsletters myself each week as a labor of love.
This week's links related to Locally Run
⚡ Fine-tune and quantize AI models directly on NVIDIA Jetson. Our new Jetson AI Lab tutorial shows how to use Unsloth and memory-efficient QLoRA to customize models, export quantized GGUF files, and run them locally with llama.cpp. Follow hands-on examples for: 🔹 NVIDIA
https://x.com/NVIDIARobotics/status/2094480283135316182
Nvidia RTX Spark ‘Superchip’: The First AI PCs Are Here | WIRED
https://www.wired.com/story/nvidia-rtx-spark-laptops-first-look/
Personal AI Router for Local Inference | NVIDIA PAIR
https://www.nvidia.com/en-us/ai-on-rtx/personal-ai-router/
GLM-5.3 can now be run locally! The 2-bit model retains ~81% accuracy after we shrunk it from 1.51TB to 239GB (-83% size). Run on a 256GB Mac or RAM/VRAM setups. GLM-5.3 is the strongest open model to date. Guide:
https://t.co/NLgb3CMB6A GGUF:
https://x.com/UnslothAI/status/2093397494889890050
Hermes now has a local backend! There’s support for Unsloth UD-Q4_K_XL and UD-Q4_K_M quants for DeepSeek-V4-Flash, Qwen3.8-27B, 3.6-35B-A3B and Qwen3.8-Flash-Next in Hermes!
https://x.com/danielhanchen/status/2095623899979600152
Two months ago, we started the mission to “build OpenClaw with OpenClaw,” and bit by bit, we moved everyone from using their local coding harness to using
https://t.co/mXvAd5PJFA – our shared agent that knows what everyone’s working on and orchestrates it all. Multiplayer coding
https://x.com/steipete/status/2094290652649636173
Codex voice (or is it ChatGPT Work in Codex voice?) is magical when it works well, but has a lot of weird loose edges that make it hard to use, like it seems to constantly switch threads it is working on, multitasks only sometimes, accesses the rest of Codex inconsistently, etc.
https://x.com/emollick/status/2095340390731940317
@TransluceAI AI evals used to ask: did the model give the right answer to a prompt? That’s nowhere near enough. AI interactions increasingly span days or months. Agents operate autonomously and act in the world. Evals will need to simulate users, local networks, and the internet… and run in
https://x.com/woj_zaremba/status/2094469674453111004
Kimi Code New Feature: Remote Control Kimi Code 0.39.0 add Remote Control as an experimental feature for accessing a local web session remotely. Enable it with KIMI_CODE_EXPERIMENTAL_REMOTE_CONTROL=1, then run kimi rc, kimi web –remote-control , or /remote-control to start
https://x.com/KimiDevs/status/2093184808419746164
dang im getting pilled that cloud agents, not local cli, is the future it also needs shared context, GOOD memory, connected to ALL your services (especially logs), etc
https://x.com/jjacky/status/2093174321157947822
Ollama already support the Muse Code harness out of the box? You can try it with models via Ollama (local and cloud): ollama launch muse
https://x.com/ollama/status/2094622506720391454
I’ve been using GPT-6 Astra for a few weeks now. It’s an incredible model with capabilities I’ve never seen before. It can do things I never thought an LLM could do. It has quirks and rough edges. But it’s the smartest model ever released. At times, it feels like a taste of AGI
https://x.com/theo/status/2095596855367455047
LESSON LEARNED: Always use BF16 kv cache. I was using turboquant. Yeah the VRAM consumption sucks – so another trick is to CPU Offload the kv cache. Implemented in SlimServe for 3090 and Qwen (it requires extensive testing to get it right) I have further work to universalize
https://x.com/QuixiAI/status/2093405502181179422
We are rolling out GLM-5.3 on Ollama. Private. Fast. US and Europe hosted. No data retention. Try Ollama’s GLM-5.3-Flash as we bring it GLM-5.3 online. Super fast. Claude Code: ollama launch claude –model glm-5.3-flash:cloud OpenCode: ollama launch opencode –model
https://x.com/ollama/status/2093356025084797176
Optimizing On-Device Inference for Apple Silicon
https://www.perplexity.ai/hub/blog/optimizing-on-device-inference-for-apple-silicon
Cohere is on-prem so if your AI goes down that’s on you.
https://x.com/cohere/status/2095592717317201984
GLM-5.3 is officially live on Hugging Face. What should run it locally: – FP8: 10-12× H100 or 8× H200 – 4-bit/NVFP4, roughly 390-430GB: one 512GB Mac Studio or 4× DGX Spark – Aggressive 2-bit, roughly 230-250GB: one 256GB Mac Studio or 2× DGX Spark, with quality and context
https://x.com/kimmonismus/status/2093354978534477956
Introducing @huggingface/kernels: 200+ WebGPU Kernels for Local AI
https://huggingface.co/blog/webgpu-kernels
Leave a Reply