Three machines, three jobs, zero rent

We built an AI bullpen instead of one big server

We built three different computers to run AI locally, not one. Here is why the mismatch between them is the entire point, and what it means for how much marketing work we can do without paying a model provider for the privilege.

Mariano Rivera pitched one inning at a time, almost always the ninth, almost always with a lead to protect. He threw one pitch, the cutter, thousands of times, and he didn't need a second one. Move him to the sixth inning of a blowout and you are wasting the best reliever in history on work a rookie could do just as well for less money.

Bullpens don't run on one arm. They run on a closer, a long reliever who can eat four innings when the starter gets knocked out early, and a specialist who comes in for one batter and one batter only. Three roles, three different skills, and the manager who tries to use his closer for long relief burns out the one asset he cannot replace.

We just built the marketing equivalent of a full bullpen, and it is already running our content generation, our analysis jobs, our crons, and our agents.

The problem with one machine

For the past year, our AI workhorse has been one Ubuntu server: an RTX 4090 paired with an RTX 5090, close to 56GB of fast VRAM between them. That machine is our closer. It is fast, it fits most models we ask it to run, and for anything latency-sensitive, it is still the right arm to call on.

But a closer can't do everything. Ask that same machine to load a 70-billion-parameter model and it runs out of room before it runs out of ideas. The VRAM ceiling is not a performance problem; it is a hard wall. No amount of optimization gets a model that needs 90GB of memory onto a card with 56.

For a year, our answer to that wall was the same answer every marketing team defaults to: send it to a cloud provider and pay per token. That works. It is also the opposite of independence, and every token we send out is a token we are paying someone else's margin on.

Three machines, three roles

So we built two more nodes, and we built them different on purpose.

The second machine is a Framework Desktop running an AMD Ryzen AI Max+ 395, 128GB of unified memory, ordered at $3,420. It doesn't have a discrete GPU in the traditional sense; its GPU and its system memory are the same pool. That is a strange trade if you are thinking like a gamer and a very good one if you are thinking like an infrastructure planner, because it means this machine can load models that would never fit in 56GB of VRAM, just slower. It is our long reliever: not the fastest arm on the roster, but the one that can go deep into a large-context job the closer cannot even attempt.

The third machine is a Threadripper 3970X, 32 cores, 256GB of system RAM, with 41.88TB of NVMe storage sitting behind it. We are adding a GPU to it now, but its real asset was never the graphics card. It is the specialist: the node we send the largest, least time-sensitive jobs to, the ones we can queue overnight and let run in batch rather than wait on. With that much fast local storage, it can also hold a genuine model library on disk, rather than re-downloading multi-gigabyte weights from Hugging Face every time we want to test something.

None of these three machines is a better version of the others; that isn't the design. Each one is the right answer to a question the other two cannot answer well.

Matching the job to the arm

The mistake we were careful not to make was mirroring the same model catalog across all three and calling it redundancy. That is not a bullpen; that is three closers standing around waiting for the ninth inning.

Instead, each node runs a distinct catalog, and our routing logic keys off model identity and job type, not load balancing. A content-generation request that needs a fast turnaround goes to the closer. A large-context analysis job that a 24GB card cannot even load goes to the long reliever or the specialist. A batch of crons that can run overnight without anyone waiting on them goes wherever the largest model that fits is sitting, cost be damned, because the marginal cost of a self-hosted token is zero.

That zero is the whole argument. Every request we can route to hardware we already own is a request we are not paying a provider for, and it is a request no provider can throttle, deprecate, or reprice on us. Model independence is not a slogan. It is a routing table.

What this actually buys us

We are not trying to replace every cloud model we use. Some jobs still belong with a frontier provider, and we will keep sending them there. But the categories of async marketing work that do not need frontier reasoning, the content drafts, the recurring analysis, the scheduled crons, the agent loops that run all day in the background, are exactly the workloads a well-matched local bullpen was built for.

The cost curve on this compounds the way a bullpen's value compounds over a season. One saved API call barely moves anything. A year of routed, self-hosted, async marketing work moves quite a lot.

What we are actually testing

None of the work we are routing to this bullpen is work a user is watching happen in real time. That is the whole precondition. Signal analysis on incoming documents, enrichment agents that go find and attach context nobody asked for on the spot, written content drafts, image and video generation, transcription, and the content planning and strategy work that feeds ICP definitions, personas, and ABM campaigns: none of it needs to return in the two seconds a person will wait at a keyboard. It needs to be right, and it needs to be cheap, and it can take its time getting there.

That is a different question than "which model is fastest." It is "which node fits the largest, best model for this specific job," and the three roles map cleanly onto it. Written content and long-document analysis want the largest model we can fit in memory, which points at the long reliever. Transcription wants a dedicated speech model, not a general one, and can run almost anywhere. Image and video generation want raw GPU throughput more than memory headroom, which points back at the closer or, once its upgrade lands, the specialist.

Every one of these nodes has to survive a power blip or a dropped connection without a human walking over to press a button. Restart-always is not a nice-to-have on a headless server; it is the entire point of running one.

We are treating this as a live experiment, not a finished system. We do not yet know which model earns a permanent seat on which node, and we are not going to pretend we do. What we do know is the shape of the problem: three machines, six distinct job types, and a cost floor of zero once the routing is right. As the app takes on more users, we will expand capacity on whichever node the data says needs it, which is a far better problem to have than guessing at scale before we have earned the right to guess.

Most marketing teams are running every job through the same arm, whether it needs a closer or not. Go count your own bullpen. If you cannot name three different jobs your compute is doing, you probably only have one machine doing all of it, and you are paying for that mismatch every single day.