Building CyberTron — Running LLMs Locally

Testing Qwen models, GPU inference, and the software stack that finally gave CyberTron a brain.

Author: Roggie D/Tuesday, August 25, 2026/Categories: Building CyberTron, Local AI & Models, Knowledge Base & RAG

Rate this article:

No rating

At this point, CyberTron had become a very different machine from the $150 computer I originally brought home.

It now had 32 GB of RAM.

The old spinning system drive had been replaced with a Patriot P210 512 GB SSD.

The original GTX 1060 6 GB had been replaced with a Zotac GeForce RTX 3060 Twin Edge OC with 12 GB of VRAM.

The original 500-watt power supply had been replaced with a be quiet! Pure Power 12 650W 80 Plus Gold unit.

And after totaling everything up in the previous article, I had $592.33 invested in the complete machine.

That was the hardware.

Now came the question that actually mattered:

What could this thing run?

Because putting an RTX 3060 into an old AMD FX-era desktop makes for an entertaining project.

Actually getting useful artificial intelligence out of it is something else entirely.

The Entire Point of CyberTron

CyberTron was never supposed to be a gaming PC.

It wasn’t supposed to be a benchmark monster.

And I certainly wasn’t trying to build something that could compete with a modern multi-GPU AI server.

The goal was much simpler.

I wanted to see how much practical local AI I could run on inexpensive hardware I actually owned.

More importantly, I wanted an AI environment where I controlled the machine, the models, the data, and the infrastructure.

That goes directly back to one of the reasons this project started in the first place.

I use ChatGPT constantly, but one of my biggest frustrations has always been losing useful information inside hundreds of old conversations.

I wanted something different.

Something local.

Something I could eventually connect to my own knowledge bases, my own infrastructure documentation, my own search tools, and my own applications.

CyberTron was supposed to become that platform.

But first it needed a brain.

Ollama Becomes the Engine

The first major component of the local AI stack was Ollama.

If you’re not familiar with Ollama, think of it as the engine room.

You give Ollama a supported AI model, and it handles the mechanics required to run that model locally.

Instead of sending a prompt across the Internet to somebody else’s AI server, the request goes to CyberTron.

The model runs on CyberTron.

The GPU does the work.

And the response comes back from a machine sitting on my own network.

That was exactly what I wanted.

CyberTron’s Ollama service was configured so it could also be accessed by other systems on my network.

That meant CyberTron wasn’t just becoming an AI workstation.

It was becoming an AI server.

Other applications, Docker containers, automation systems, and eventually other machines could send requests to it.

Now things were starting to get interesting.

Then Came Open WebUI

Running models from a command line is useful for testing.

It isn’t how I wanted to interact with the system every day.

That’s where Open WebUI came in.

Open WebUI provides a browser-based interface for local AI models.

In simple terms, it gives Ollama something that feels much more like ChatGPT.

You open a browser.

Select a model.

Start a conversation.

Behind the interface, however, the processing is happening on CyberTron.

That distinction is important.

The browser interface may feel familiar, but the architecture is completely different.

Browser → Internet → Cloud AI Provider

became:

Browser → CyberTron → Ollama → Local AI Model → RTX 3060

The prompts could stay on my network.

The model was under my control.

And suddenly this ridiculous old AMD FX computer was starting to look considerably less ridiculous.

Twelve Gigabytes Changes the Game

This is where the RTX 3060 upgrade from Episode 3 started paying for itself.

The GTX 1060 had only 6 GB of VRAM.

That is enough to experiment with smaller models, but VRAM becomes one of the major constraints when running local AI.

The RTX 3060 doubled that to 12 GB.

And twelve gigabytes opened the door to a much more useful class of models.

CyberTron wasn’t suddenly going to run gigantic frontier models.

That was never the expectation.

But models in roughly the 7-billion to 14-billion parameter range became realistic targets depending on quantization, context size, and model architecture.

That was exactly the territory I wanted to explore.

First Stop: Qwen

One of the model families I spent considerable time testing was Qwen.

CyberTron successfully ran models including Qwen3 8B and Qwen3 14B.

The important part wasn’t simply getting them to load.

I wanted to know whether they could actually run on the RTX 3060 without constantly spilling work back onto system memory and turning the whole experiment into a slow-motion demonstration.

The answer was better than I expected.

Qwen3 8B could run completely on the GPU.

Then I started increasing the context window.

4096.

8192.

16384.

CyberTron kept going.

That matters because context is effectively part of the AI model’s working memory for a conversation.

A larger context window allows the model to work with more information at once, but it also consumes additional memory.

With the RTX 3060, CyberTron finally had enough VRAM to experiment instead of merely surviving.

Then I Tried Qwen3 14B

Naturally, once the 8B model worked, I had to try something bigger.

Because apparently successfully completing one experiment is just an invitation to make the next one more questionable.

So I loaded Qwen3 14B.

This model used roughly 9.6 GB and still ran entirely on the RTX 3060 during our testing.

I tested it with increasing context sizes as well:

4096.

8192.

12288.

And it worked.

That was a significant moment for the project.

Remember what this machine actually is.

Underneath all of these upgrades is still an AMD FX-era platform that started life as a $150 used computer.

Yet here it was running a 14-billion-parameter local language model completely on a consumer RTX 3060.

No cloud GPU rental.

No enterprise AI server.

No $5,000 workstation.

Just CyberTron.

Bigger Isn’t Automatically Better

One thing becomes obvious very quickly when experimenting with local AI models:

Model size isn’t everything.

A larger model can produce better results in some workloads, but it also consumes more VRAM, takes longer to load, and leaves less room for context.

That creates a balancing act.

Do I want a larger model?

A larger context window?

More speed?

Vision capability?

Tool support?

Better reasoning?

There isn’t one perfect model for everything.

That realization eventually influenced the entire design of CyberTron.

Instead of trying to find one enormous model that could somehow perform every job, I started thinking about specialized models and specialized assistants.

One model might be better for general conversation.

Another might be better for technical work.

Another might handle images.

Another might eventually interact with tools or automation.

That idea becomes very important later in the project.

Other Models Enter the Rabbit Hole

Qwen wasn’t the only thing I tested.

CyberTron became a rotating laboratory of local models.

Different versions of Qwen came and went.

Gemma models were tested.

Llama models were tested.

Vision-capable models were tested.

Some models looked promising and stayed.

Others were downloaded, tested, compared, and eventually removed.

That’s part of running your own AI infrastructure.

Cloud AI mostly hides model deployment from the user.

Local AI puts you in charge of it.

You start paying attention to things like:

Model size.

Quantization.

VRAM requirements.

Context length.

GPU utilization.

Inference speed.

Vision support.

Tool calling.

And whether the supposedly brilliant new model you just downloaded is actually any better at the task you care about.

I started treating AI models much like I would any other infrastructure component.

Test it.

Measure it.

Keep what works.

Remove what doesn’t.

Watching the GPU

One of the most satisfying parts of the entire project was watching the RTX 3060 actually do what I bought it to do.

During our Qwen testing, models could show 100 percent GPU utilization for inference rather than falling back heavily onto the CPU.

That was important because CyberTron’s CPU is still the oldest major component in the machine.

The AMD FX platform can keep the system running, but I don’t want it doing the heavy lifting for modern AI inference.

The GPU is where that work belongs.

And the RTX 3060 was doing it.

That $285.04 GPU purchase suddenly made a lot more sense.

The Old Computer Was Becoming Infrastructure

There was another shift happening that I hadn’t completely appreciated when this project started.

CyberTron was no longer just a computer I happened to be running AI software on.

It was becoming part of my infrastructure.

Ollama exposed models as a service.

Open WebUI provided the user interface.

Docker provided the application platform.

Other systems on the network could communicate with CyberTron.

That meant I could begin building things around the models instead of simply chatting with them.

And that’s really where local AI becomes useful.

The model itself is only one component.

The interesting part is what happens when the model gains access to information and tools.

The Problem I Actually Wanted to Solve

Remember the original frustration behind this entire project?

Information gets buried.

You have a useful conversation.

You solve a problem.

You document something important.

Three months later, you know the answer exists somewhere, but finding it becomes another project.

Simply running a local LLM doesn’t solve that problem.

If anything, it just gives me another place to create conversations.

CyberTron needed something more.

I wanted the AI to be able to work with information I had already collected.

Server documentation.

Infrastructure notes.

Technical procedures.

Previous AI conversations.

Business information.

Project documentation.

Eventually, potentially thousands of documents.

That means the next challenge isn’t really about the model at all.

It’s about knowledge.

CyberTron Has a Brain. Now It Needs a Memory.

At this point, CyberTron could run real local language models.

The RTX 3060 had proven it could handle useful 8B-class models and even a 14B model completely on the GPU under the configurations I tested.

Ollama provided the inference engine.

Open WebUI provided the interface.

The hardware upgrades had finally turned the original $150 machine into something genuinely useful.

But I still hadn’t solved the problem that started this journey.

CyberTron could think.

CyberTron could answer questions.

But CyberTron didn’t know my stuff.

Not yet.

And fixing that would send me straight into one of the most confusing parts of modern AI:

RAG, embeddings, vector databases, document processing, and knowledge bases.

Because apparently replacing RAM, migrating Linux to an SSD, installing a new power supply, and stuffing an RTX 3060 into an ancient computer wasn’t far enough down the rabbit hole.

We were just getting started.