I LEARNED THIS
THE EXPENSIVE WAY.
WITHOUT SPENDING ANYTHING.

I studied computer science and engineering, with a minor specialisation in artificial intelligence and machine learning. The specialisation taught me the maths: what a gradient is, why attention works, how a transformer is put together.

It did not teach me what happens when a model meets a real document, a real user, and a real deadline.

Everything I know about that, I learned on my own hardware. No lab, no cluster, no budget for an API key — just a laptop and an unwillingness to treat the model as a box I wasn't allowed to open.

Constraints are a bad excuse and a very good teacher.

That was the whole brief. Not "build a chatbot" — anyone can bolt a text box onto a model. The hard half was from them: the answer had to come out of the documents in front of it, and it had to be possible to check.

The technique had a name — retrieval-augmented generation — and at the time that name was roughly all I had.

Every tutorial started the same way: get an OpenAI API key. Mine had just stopped being free.

So the shortcut closed, and the only route left was to build the thing out of parts I could actually run. Open-source embedding models instead of a hosted endpoint. My own chunking, because the default splitter kept cutting sentences in half. Rerankers, once it became obvious that "the top five nearest vectors" and "the five most useful passages" are not the same list.

I benchmarked FAISS, Milvus and Chroma against each other for latency and recall — not out of thoroughness, but because I couldn't find a straight answer anywhere about which to use, and I didn't want to pick one on vibes.

Every one of those was a decision I had to make myself and then live with. That is the part a tutorial can't give you.

First pass ran on a 406M-parameter BART summariser — small enough to fit anywhere, but the answers were thin.

Then I ran into a wall that no amount of cleverness in the retrieval layer could get me past: the bigger model I needed didn't fit in 12 GB.

Quantization stopped being a word in a paper and became the entire problem. Decoder-only or sequence-to-sequence. GPTQ, AWQ, QAT — what each one actually does to the weights, what you give up going down to four bits, and whether what comes out the other side is still worth serving to a person.

Quantization isn't what bought that 30% — a bigger, more capable model did. Four bits is just what got it to fit. That's the credit it gets.

The number I'd actually point at in that panel is the last one. One developer. Not because it was heroic — because there was nobody to ask. Every dead end had to be diagnosed rather than escalated, and that turns out to be the difference between knowing a stack and having merely used one.

I joined RedSoft Solutions in April 2024 and stopped being an intern inside the year. Since then it has been retrieval systems, agents and guardrails for clients across real estate, healthcare, manufacturing and e-commerce.

I can't name them — that is what an NDA is for — but I can tell you exactly what they taught me, because it wasn't any of the things I expected.

A demo answers the question. A system answers it at 2 a.m., on a malformed input, while an upstream API is having a bad day.

Guardrails are the product. On a clinical imaging tool, the interesting design work wasn't what the model could say — it was building the thing around what it must never say. A model that gives medical advice once is worse than no model at all.

Knowing when to stop is a feature. The agentic systems I'm proudest of are the ones that hand control back: clear the straightforward case, park anything over a threshold for a human, and make the handoff legible to whoever picks it up. I've now seen that same pattern hold for property queries and for financial transactions, which is when I started to trust it.

Nobody writes comics about backups. I run the weekly disaster-recovery backups. Zero data-loss incidents, which is the kind of achievement that is invisible right up until the day it isn't.

This is the thing I'd go back and tell myself on the laptop. I spent that first year assuming the model was the difficult part, and that everything downstream of it was plumbing I'd figure out later.

It was the reverse. The model is the part with the most written about it and the least that can go quietly wrong. What decides whether an AI feature survives contact with real users is duller and further down: schemas that don't lie, scheduled jobs that actually run, retries that don't stampede, role-based access that holds, and error handling that decides whether a bad API day becomes an incident.

Turns out the AI was never the hard part.

Two years in, the thing I'm best at is the seam between them: the place where a retrieval system meets a schema, where an agent meets a permission model, where a clever demo either becomes a product or quietly falls over.

That seam is where I want to keep working — on systems where being right matters more than being fast to demo, and where somebody is going to depend on the answer.