Open-weight / local models

Context

Context length, file access, retrieval, multimodal input, and tool support depend on the model, quantization, runtime, and harness placed around it.

Actions

A bare model generates output only. Tool calling, repository editing, browsing, and automation require a compatible runtime and agent harness.

Execution

Inference can run on CPUs, Apple silicon, GPUs, workstations, or self-managed servers. Performance and supported model size depend on memory, quantization, and backend support.

Memory & rules

The model has no automatic cross-session memory. Applications may add prompt templates, conversation storage, retrieval, profiles, or project instruction files.

Permissions

Running locally can keep weights and some data on controlled hardware, but downloaded models, plugins, telemetry, network calls, APIs, and host filesystem permissions still need review.

Verification

Evaluation and testing are supplied by the operator or surrounding harness. Model quality, tool syntax, and structured output must be tested for the exact model/runtime combination.

Best for

• Offline or controlled-environment inference
• Model and runtime experimentation
• Workloads where hardware ownership and provider choice matter

Limitations

• Expecting one universal capability or safety profile
• Editorial judgment: users who need frontier performance with no setup, hardware planning, or model evaluation