Which local AI model should I run? The question sounds as if it should have one answer. It does not.
A model that is comfortable on a 128 GB Apple-silicon workstation may be a bad choice for a 16 GB laptop. A quantization that fits in storage may still leave too little memory for context, the operating system, and the rest of the work. The model name alone also does not tell you which publisher produced the file, which runtime tag resolved locally, or whether two computers use the same artifact.
I built synthesis-local-model-runtime to make those decisions repeatable. It
profiles the computer, compares the local runtime choices, validates a dated
model catalog, applies local policy, produces a dry-run plan, installs only
after explicit authorization, verifies what the runtime registered, and keeps
a private model map for that specific machine.
The skill is open source in the
synthesis-skills repository.
Its deterministic Python program does the operational work. The
language model helps interpret goals and tradeoffs, but it does not guess how
much memory the computer has or improvise a download URL.
Try it without downloading a model
The easiest path is to install the native Synthesis Skills plugin for the AI coding agent you already use. In a terminal, run the two commands for that client:
# Codex or ChatGPT desktop
codex plugin marketplace add synthesisengineering/synthesis-skills
codex plugin add synthesis-skills@synthesis-engineering
# Claude Code
claude plugin marketplace add synthesisengineering/synthesis-skills
claude plugin install synthesis-skills@synthesis-engineeringStart a new session after installation, then ask the agent:
Use synthesis-local-model-runtime. Profile this computer and recommend local open-weight models for coding, code review, test-failure analysis, and professional writing. Compare the runtime choices available on this machine, but use Ollama unless another choice fits a stated need better. Reserve enough memory and storage for ordinary work. Do not download anything. Show me the complete dry-run plan, including exact artifacts, quantizations, estimated disk use, runtime requirements, and any blockers.
That request profiles the machine and produces a recommendation without authorizing a model download. Review the proposed artifacts and total storage first. If the plan fits your needs, authorize only those named downloads, then ask the agent to verify each installed artifact and benchmark the models one at a time.
The repository also supports a direct command-line path when no plugin-capable agent is available:
git clone https://github.com/synthesisengineering/synthesis-skills.git
cd synthesis-skills/skills/synthesis-local-model-runtime
python3 scripts/local_model_runtime.py profile
python3 scripts/local_model_runtime.py runtimes
python3 scripts/local_model_runtime.py catalog
python3 scripts/local_model_runtime.py recommend \
--policy assets/policy.example.json
python3 scripts/local_model_runtime.py install \
--policy assets/policy.example.jsonThe final command is still a dry run. It installs only when repeated with
--yes after you have inspected the plan.
Ollama is the default, not the only choice
There are several substantially used ways to run open-weight models on a Mac, but they do not expose the same lifecycle controls.
- Ollama is the default managed runtime. The skill can plan downloads, install, inventory, verify content digests, benchmark, configure the service, and update installed model references.
- LM Studio is an optional managed alternative.
Its
lmscommand supports exact, noninteractive downloads and JSON inventory. The skill can install and verify cataloged LM Studio targets, but it does not claim to perform verified model-content updates because the CLI does not expose a stable content identity for that proof. - llama.cpp is a direct GGUF execution and serving runtime. The skill detects it and reports what it can do without pretending that llama.cpp owns a managed model library.
- MLX-LM is a direct Apple-silicon-native execution and serving path. The skill detects it separately from any models in the Hugging Face cache.
Run runtimes to see which are installed and the capabilities each adapter can
actually prove. If you prefer LM Studio, select it explicitly:
python3 scripts/local_model_runtime.py recommend \
--runtime lm_studio \
--policy assets/policy.example.json
python3 scripts/local_model_runtime.py install \
--runtime lm_studio \
--artifact qwen3.8-27b-q8-0The LM Studio installation is also a dry run until --yes is added. llama.cpp
and MLX-LM remain valid direct choices for people who want their lower-level
execution or serving paths; the skill does not attribute Ollama-style download,
inventory, or update semantics to them.
What the skill measures
On macOS, the profiler records selection-relevant facts such as Apple chip family, CPU and GPU core counts, unified memory, architecture, free storage, the local runtime version, and the resolved model-store location. It deliberately omits serial numbers, hardware UUIDs, hostnames, account names, and provisioning identifiers.
That distinction matters. A machine inventory should be useful without becoming a device-tracking database.
The planner then applies several hard gates:
- The model plus declared operating and context headroom must fit in memory.
- The whole installation plan must fit while preserving a storage reserve.
- The runtime must meet the artifact’s minimum supported version.
- Provider, organization, lineage, and artifact exclusions are applied before ranking.
- An explicit artifact request cannot bypass those policy exclusions.
Only after those gates does quality ranking choose among the candidates that fit.
Why the dry-run boundary matters
The install command without --yes prints the selected artifact
IDs, runtime names, quantizations, estimated disk use, memory-fit status,
remaining storage, and any blockers. Repeating the command with --yes is the
separate execution step.
This is more than a courtesy prompt. Model downloads can be tens or hundreds of gigabytes. The plan is an inspectable contract between the requested workload, the machine, the catalog, and the files that will arrive.
Updating a model is a verified state transition
An installed tag can keep the same name while the artifact behind it changes. The skill therefore treats an update as a before-and-after identity check, not as a successful progress animation.
For Ollama, copy the exact installed references from ollama list and inspect
the dry run. For example:
python3 scripts/local_model_runtime.py update \
--model glm-4.7-flash:q4_K_M \
--model deepseek-r1:8b-0528-qwen3-q8_0Add --yes and a receipt directory only after confirming the scope. The tool
pulls the same references, enumerates them again, and records digest and size
before and after. An unchanged digest is reported as already-current; a
changed digest is updated. A failed pull or a missing post-pull model fails
the receipt.
There is also an explicit --all mode, but it is intentionally not the
default. Use it only when every model currently registered in Ollama belongs
in the update. LM Studio updates remain blocked until its scriptable interface
provides enough content identity to distinguish an update from a re-download.
A local model is one layer of an on-prem SDLC stack
A local model endpoint does not, by itself, connect to issue tracking, Git hosting, team chat, source code, test systems, or deployment environments. A practical architecture separates those responsibilities:
Issue tracker + Git service + team chat
|
controlled, scoped connectors
|
agent orchestration + identity + policy + audit
|
verified local endpoint from the machine map
|
managed or direct runtime + selected model artifact
|
isolated code/test execution + recorded QA evidencesynthesis-local-model-runtime owns the model-runtime boundary: it measures
the computer, chooses artifacts that fit, installs only after authorization,
verifies their runtime identity, and records the endpoint selected for that
machine. The orchestration layer owns tool permissions, connector credentials,
audit trails, repository rules, code execution, test evidence, and human
approval before external writes, merges, messages, or deployments.
For an initial SDLC deployment, start connectors read-only. Test concrete work such as reviewing a small change, explaining a known test failure, proposing a new test, and finding a documented defect. Keep code execution isolated, keep the evidence produced by the test runner, and compare the result with a task- specific acceptance set. A token-speed benchmark proves that a model can run; it does not prove that the model can review your codebase accurately.
Three hypothetical examples
The following scenarios are made up, but the constraints and outcomes are realistic.
A 16 GB travel laptop
Maya wants an offline assistant for rewriting short notes during flights. Her laptop has 16 GB of unified memory and 180 GB free. Her policy reserves 6 GB for macOS and applications, keeps 60 GB of storage free, and asks for an 8,192-token planning context.
The planner rejects a 27 billion parameter Q8 artifact even though it could be downloaded. It recommends a smaller quantized model that fits with headroom. Maya gets the offline workflow she asked for without turning ordinary multitasking into memory pressure.
A 64 GB engineering workstation
Theo uses a 64 GB machine for code review, test failure analysis, and local documentation search. His policy requires coding and professional-writing roles, allows several model families, and reserves 16 GB of working memory.
The skill selects a stronger mid-sized model for daily work and records its
runtime identity, including an exact content digest when Ollama provides one.
A repository script asks the resolver for the installed coding model instead
of hard-coding a tag. When Theo updates the catalog later, the resolver still
returns only an artifact selected and verified for that exact machine.
A 128 GB model-evaluation Mac
Anika compares several model families on a 128 GB Mac. She wants high-quality writing, coding, long-context reasoning, and a second reasoning model for cross-checks. Her policy names four families and exact quantizations, reserves memory for a 32,768-token planning context, and preserves 500 GB of free disk.
The resulting plan can include four large artifacts because they are stored together but loaded and benchmarked one at a time. Each benchmark receipt contains the catalog ID, resolved runtime model and digest, bounded generation settings, timing, and output path. A later comparison can distinguish a model change from a prompt change or a machine change.
The machine map is part of the system
People who work across several computers need more than a list of favorite models. They need a mapping from each computer to the artifacts that were selected, installed, and verified there. The record also names the runtime, because the same model family can have different identities and lifecycle guarantees across Ollama and LM Studio.
The skill stores that mapping outside source repositories. Each machine gets a random local identifier rather than a hardware identifier. A resolver can then answer a narrow question such as: “Which installed Qwen artifact is approved for this computer?” It fails closed if the artifact was selected but never installed, if the machine is unknown, or if the runtime no longer reports the model.
That makes the inventory useful to synthesis engineering and synthesis coding workflows. A task can request a role or family and receive the correct local runtime name without assuming that every computer has the same memory, storage, or model files.
What a live installation taught me about recovery
One test installation downloaded the large GGUF layers successfully but timed out while retrieving a tiny final registry object. Repeating a 30 GB download would have been wasteful, and trusting a completed progress bar would have been unsafe.
The recovery path accepts only layers pinned in the public catalog. It checks the full SHA-256 digest and exact byte size of every cached model and projector layer. It then uses Ollama’s supported local import and verifies the resulting runtime identity before updating inventory.
The live run exposed another detail worth recording. Hard links eliminate a separate staging copy, but Ollama may normalize the GGUF into a new model-sized runtime layer and retain the original registry cache. The recovery receipt therefore reports two different numbers: zero network download and the worst-case additional runtime materialization. It does not silently delete an unreferenced cache blob from a runtime-owned store.
Operational tools need receipts from real runs as well as clean unit tests.
Installation is not the same as runnable
A second live failure exposed a different gap. Ollama had registered a Kimi
Linear artifact, and its digest and size matched the inventory. The first
generation request still failed. The Homebrew service was configured to use an
8-bit key-value cache, while that model’s value-head dimensions required an
f16 cache.
Ollama documents KV-cache type as a global service setting in its FAQ, and the exact Kimi Linear incompatibility is preserved in Ollama issue 16245.
That is a runtime-fit problem. More memory or another model quantization would not make the existing service setting compatible.
The skill now reads the effective Ollama service configuration and compares it with artifact requirements during planning. On macOS, a separate dry-run-first command can change the standard current-user Homebrew service:
python3 scripts/local_model_runtime.py configure-ollama \
--kv-cache-type f16Adding --yes writes a private backup, changes the allowlisted setting,
reloads the service, and waits for its local API to become healthy. A failed
reload restores the prior configuration. After that controlled change, the
same Kimi artifact completed the bounded prompt at roughly 100 output tokens
per second on the 128 GB evaluation Mac.
The benchmark gate caught another distinction. A DeepSeek reasoning model ignored a request to suppress thinking, emitted reasoning markup, and used the entire small token budget before reaching a final answer. The raw output was preserved, but the receipt did not pass. A later run explicitly included reasoning and supplied enough bounded tokens to reach a stop-completed final response. A benchmark that merely receives bytes is not a functional acceptance test.
The catalog is dated evidence, not eternal truth
Model names, licenses, runtime support, quantizations, and official tags change. The catalog records when each entry was verified, its upstream owner, artifact publisher, distribution channel, license, parameter counts, memory thresholds, disk estimate, and runtime requirement.
Provider origin is procurement metadata. It is not proof that a model is safe, trustworthy, free of text watermarking, or suitable for a particular use. Those claims require separate evidence and evaluation.
The same boundary applies to benchmarks. A bounded local prompt can prove that the model loads and produces output under recorded settings. It cannot prove authorship, watermark absence, factual reliability, or general quality.
Why this helps beyond one computer
The immediate benefit is avoiding a bad download. The larger benefit is a clean boundary between policy and mechanics.
The public catalog and profiler can serve many people. A personal policy can express which providers, licenses, lineages, roles, quantizations, and storage reserves someone accepts. The machine inventory remains local. Teams can choose Ollama for a complete managed lifecycle, LM Studio for its managed desktop and CLI workflow, or a direct llama.cpp or MLX-LM path without conflating their capabilities. People with several Macs can run the same process on each one and get different, defensible answers.
Local models are infrastructure. A measured, versioned, and verified process turns “which model should I run?” into a decision you can reproduce, audit, and change deliberately.