Our own private LLM, on four NVIDIA GB10 systems.
Our internal agents run on an open-weight DeepSeek model that we serve on our own hardware: four NVIDIA GB10 systems working as one, using tensor parallelism. Prompts stay on hardware we control, and the monthly bill does not grow with every token.
The problem.
- Agents that work all day (coding helpers, summarizing long sessions, document work) burn through tokens, and per-token cloud bills grow with usage.
- Some internal work should not leave hardware we control.
- A model too large for one machine has to be split across several without becoming fragile.
What we built.
4-node cluster
Four NVIDIA GB10 (Grace Blackwell) systems serve a single model. Each one holds a quarter of the weights, with tensor parallelism across all four.
Dedicated interconnect
The nodes talk over a dedicated high-speed RDMA network, kept separate from the office LAN.
Open-weight model
An open-weight DeepSeek Flash-class model, quantized to fit the cluster's memory with room for long contexts and a prompt cache.
OpenAI-compatible API
Agents call it like any hosted model, with an OpenAI-compatible API reachable only on our private network. There is no public endpoint.
Prompt cache
Long, repeated agent contexts are cached, so follow-up turns don't re-read the whole conversation.
Guard and autostart
A watchdog checks health, temperatures and network every few minutes. The service comes back on its own after a reboot.
How it fits together.
Alongside: a watchdog checks health, temperatures and network on every node and alerts the operations agent.
Simplified diagram. Hostnames, keys and internal addresses are left out on purpose.
The stack.
- 4 × NVIDIA GB10 systems (Grace Blackwell, unified memory)
- Open-weight DeepSeek model, quantized
- Tensor-parallel inference engine across 4 nodes
- Dedicated RDMA network between nodes
- OpenAI-compatible endpoint on a private network
- Scheduled health and thermal watchdog
How we test and roll back.
- Every engine or model update is benchmarked against the running version on the same prompts (prose, code, structured output, concurrency) before it is adopted.
- Cutover scripts swap engine versions in under a minute and roll back automatically if health checks fail. We have rejected an engine update that brought no gain for our load and kept the previous version.
- OS and driver updates run node by node in a maintenance window, with the old kernel kept for rollback and a real request verified after each node.
- Only one runtime runs at a time, so experiments never compete with production for memory.
Where people stay in control.
- Hardware changes, OS upgrades and model swaps need the owner's explicit approval and happen in a maintenance window.
- The cluster serves internal workloads. Our client-facing agents use the same model family through a hosted endpoint, so client traffic never depends on one office's power or network.
- Alerts go to an operations agent first. A person decides on anything that changes the setup.
What the client gets.
- A sizing and cost analysis for running an open-weight model on hardware you control
- A cluster setup with an OpenAI-compatible API on your private network
- Benchmarks against your real prompts before and after every change
- Runbooks for updates, rollback and monitoring
How this is scoped
$1,500 USD
100% credited to the build within 14 days
We size hardware, model and cost against your workload.
From $5,000 USD
Written scope
Setup, integration and runbooks; Managed Operations $750–3,000/mo.
Prices from our public US price list. Telephony, messaging, AI usage and payment fees are passed through at cost. Final scope in writing.
Status and what we don't publish.
Verified November 2026: the cluster serves our internal agents every day. We don't publish throughput figures on this page because they depend on prompt length and concurrency. We share them, measured on your workload, during the diagnostic.
Want this for your business?
Book a 20-minute call. We look at your channels, your systems and where an agent pays off, and you leave with a plan.