Reliability
How to Route Local LLM Requests Across Multiple Computers
Build a personal or team local-LLM pool without putting laptop IDs in application code. Covers readiness, artifact matching, reassignment, and overrides.
Why ordinary load balancing is not enough
Two computers can both be online while only one has the required model, memory, or capability. A useful local-LLM router needs model-aware readiness, not merely a successful heartbeat.
Optimaiz represents the requested workload with a stable key and matches that key to verified model artifacts advertised by eligible machines. This avoids sending a request to a computer that is connected but cannot actually run it.
The routing sequence
A managed request is filtered by application authorization, execution policy, capability, artifact, and current availability. Ready targets rank ahead of warm targets. One target receives a lease; the response records what actually ran.
- Filter out offline, unauthorized, incompatible, or stale machines.
- Prefer a ready machine with the required artifact already installed.
- Lease the request so two computers do not both execute it.
- Record assignment, start, result, and any pre-output reassignment.
What happens when a laptop sleeps
If the assigned machine becomes unreachable before output begins, Optimaiz can return the work to the eligible pool and choose another computer with the same acceptable artifact. If none exists, a local-required request fails clearly. A local-preferred request may use a configured hosted fallback.
If output may already have started, the gateway does not blindly replay the request. That conservative boundary prevents duplicate tool actions and conflicting answers.
Automatic UX with an expert override
Normal users should choose a workload and execution policy, not a laptop. Operators still need overrides for debugging, stateful workloads, thermal constraints, or a machine with a specialized model.
A practical interface offers Automatic, Prefer this computer, and Pin to this computer. The override is policy, not an application dependency: code continues to use the same stable model key.
The checks that keep the pool accurate
Machine heartbeats should carry a short-lived readiness view and verified installed-model inventory. Uninstalling a model or stopping the runtime must remove eligibility quickly. Success and failure history can help rank equivalent targets, but stale history must never override current artifact and authorization checks.
The seamless experience comes from making these checks continuous and invisible while keeping the selected target and every reassignment visible to the operator.
Frequently asked questions
Does the application need to know every machine ID?+
No. It should request a stable workload key and policy. The router selects a compatible machine; operators can override selection separately.
Can a request move after generation starts?+
Automatic replay after output begins is disabled by default because it can duplicate effects or return competing answers.
Do all machines need the exact same model?+
They need an artifact accepted by the workload profile. Exact artifact matching is the safest configuration for seamless reassignment.