Free local LLM API · OpenAI compatible
Run an LLM locally. Call it like OpenAI.
Turn laptops and workstations you already own into an approved AI compute pool. Optimaiz exposes one OpenAI-compatible endpoint, chooses a ready machine, records what ran, and uses a hosted fallback only when you allow it.
Five-minute integration
Keep the SDK. Make local execution explicit.
- 1. Install. Add Optimaiz Local to one computer and choose a model that fits its memory.
- 2. Approve. Pair that computer with an Optimaiz app. The runtime connects outbound; no inbound port is opened.
- 3. Call. Use the OpenAI SDK with the Optimaiz base URL and explicit local provider.
- 4. Expand. Approve more computers without changing application code.
import os
from openai import OpenAI
client = OpenAI(
api_key=os.environ["OPTIMAIZ_APP_TOKEN"],
base_url="https://optimaiz.endlessriver.ai/v1",
default_headers={"x-optimaiz-trace-mode": "metadata"},
)
response = client.chat.completions.create(
model="local",
messages=[{"role": "user", "content": "Classify this support ticket."}],
extra_body={"provider": "optimaiz.local"},
)Use the right meaning of free
No hosted token fee for local runs. No fake zero-cost claim.
| Approach | Hosted token fee | Capacity | Best for |
|---|---|---|---|
| Hosted free tier | Free within a changing limit | Provider controlled | Prototypes and low-volume tests |
| Direct local runtime | None | One device you manage | Device-only, single-machine work |
| Optimaiz Local | None for successful local runs | Approved machine pool | Applications needing routing and traces |
| Hosted paid API | Usage based | Provider controlled | Hard models, burst traffic, large context |
Local electricity, hardware wear, storage, networking, and operator time remain outside the hosted comparison. Read the full free LLM API cost guide.
From one machine to five
Applications request a model. Optimaiz chooses the computer.
Automatic
Select a ready authorized machine with an accepted model artifact. This is the normal path.
Prefer
Try a chosen computer first, then return to the eligible pool before output starts.
Pin
Require one computer for stateful or specialized work and fail clearly when it is unavailable.
Reassignment is deliberately bounded. Optimaiz can choose another eligible computer before output begins. It does not blindly replay a request after generation may have started.
Local LLM API FAQ
Questions, answered directly
What is a local LLM API?+
A local LLM API lets software send prompts to a language model running on a computer you control. Optimaiz provides an OpenAI-compatible cloud endpoint that routes explicit local requests to an approved computer.
Is the Optimaiz local LLM API free?+
You can start Optimaiz free, and a request that runs locally has no hosted model per-token charge. Your hardware, electricity, network, and operating effort are still real costs.
Can I use the OpenAI SDK with a local model?+
Yes. Change the base URL to the Optimaiz /v1 endpoint, use your Optimaiz app token, and send provider optimaiz.local with the local alias or a stable model key.
Does local mean the prompt never enters the cloud?+
Not for the managed route. Managed local requests transit the Optimaiz cloud gateway and execute on your machine. Use direct Helm loopback for data that must remain entirely on the device.
What if the selected laptop is unavailable?+
Before output begins, Optimaiz can reassign the request to another eligible computer with an accepted model artifact. Hosted fallback happens only when the workload policy explicitly permits it.
Can an operator choose a particular computer?+
Yes. Automatic selection is the normal UX, while Prefer and Pin overrides remain available for debugging, stateful work, or specialized hardware.
Your computer is already paid for.
Start with one bounded workload. Keep harder calls hosted. See both routes in one place.
Start free →