Every few years for the last fifteen odd years I find myself staring at the samme problem in a different costume. Server costs that don’t justify usage, a few extra milliseconds in response time, a feature that starts to stand out as a resource hog at scale - not broken, just hungry. Symptoms easy to miss if you weren’t looking real close.

“When you hear hoofbeats, think horses, not zebras.” — Dr. Theodore Woodward

Woodward’s message for his medical student’s was simple, “don’t over-diagnose”, and more often than not he was right. But listen long enough and you notice that those hoofbeats have patterns; same animal painted in different colors.

I’ve been chasing the same zebra for fifteen years.

2012 — The First Encounter

Picture the internet in 2012, the internet I was building for crawled on 512 Kbps, Firefox’s JS engine was briefly faster than Chrome’s and “cloud” would still feature in strategy discussions.

I was building browser-based psychometric games - in-game choices poured into an SVM, which defined the game progression, and the cumulative breadth of those choices would paint a picture of the player’s behavioral profile.

But games aren’t games if they stutter, and “psychometry” goes out of psychometric games if the player is thinking more about their internet connection than about their choices.

The horse: throw more servers at it, faster API. I went looking for the zebra. Why this round-trip at all? What if the whole SVM lived in the browser itself? In 2012, writing a pure JS SVM implementation wasn’t exactly a well-trodden path, but once we reached there, 1000x improvement in UX. The inference cost graph went flat, when inference runs in the user’s browser no one really pays for the compute. For a bootstrapped startup that wasn’t an optimization, that was survival.

I didn’t have a word for what I had done back then and I thought that I found a clever hack. It was not a hack, I had met the zebra for the first time.

2014 — Familiar Hoofbeats

Clusto was what Google Workspace is today but a couple of years early, living inside Gmail with deep integrations with Asana, Trello, Dropbox, Calendar and half a dozen other specialized tools.

My co-founder Utkarsh wrote and trained the ML that classified and clustered the emails, tasks, notes, and meetings into related Clusters. It worked but the problem was the math of it: every user’s inbox was thousands of emails, we had a lot of users. The server-side classification at that volume started to feel like bringing a knife to a gunfight. Every marginal user would saturate our infrastructure and then, in non-peak hours, sit idle.

Maddy, my co-founder, lost sleep trying to tame the horses - scale up, batch the jobs, queues, workers, optimize the classifier, rent more metal.

The zebra: why burn compute on the 3,000th email from 2010 when the user was staring at an email from this morning?

We moved the heavy lifting of the immediate into the browser, everything else could take its own time and saturate our infrastructure. The math finally made sense.

2016 — The Zebra Started Showing Its Stripes

HandyTrain was where it stopped being a coincidence.

HandyTrain was a mobile-first content creation and distribution platform for India’s deskless workforce, administered by India’s largest corporates. Oflline capability was a necessity. From content creation, translation, adaptive curriculum, progress analysis - AI/ML powered everything. The nature of our ML pipelines had also become a lot more sophisticated than the email-clustering days. By early 2019, the first wave of transformer-era models had started making their mark and we fell in love with BERT.

I had concocted a BiLSTM-plus-BERT model that inferred the visual hierarchy of documents. Creators loved it, now they could push a .docx,.pdf, .txt or a .pptx they already had in their arsenal and in seconds they’d get publish ready byte-sized modules for HandyTrain. But it ran every few hours, always-on servers made no economic sense. On-demand cold starts took 10–15 minutes and our creators and administrators simply lost patience.

We rewrote it in WASM (and later WebGPU). Zero cold start. Zero cost of feature deployment. Content creation that took 8–10 minutes of waiting now took 15 seconds. Adaptive learning got pushed into mobile apps — which accidentally created a category that didn’t exist before: offline adaptive learning.

It’s also where Abhishek walks in. My second-in-command at HandyTrain back then, the person I’d pull into a room when the problem was too tangled for one brain. Donald joined a little later as Product Head, and between the three of us, we’d end up at a whiteboard more often than was probably healthy. But the relentless push towards efficiency gave Donald the freedom to solve deeper problems, which meant more complex pipelines and more models. Neither of us knew it, but years of those arguments were going to make both of them my co-founders.

Not everything lived on the edge, some workflows genuinely belonged on the server. But the question had flipped. We stopped asking “where can we afford to run this?” The question became “where does this feature want to run? How much of the peak theoritical efficiency can we achieve?” The answer: closer to the user, closer to the data, to the moment of actual use, a deep integration of problem, data and usage.

2022 to 2025 — The World had Changed and the Zebra Grew Canines

The pre-ChatGPT months were a strange. Something had shifted in the realm of possibilities. GPT-3 was good but then ChatGPT dropped, and suddenly every product in the world had to have an answer to “how are you using AI?” and most of those answers were, charitably, rushed.

But what made this era different? what gave the old zebra actual teeth? It was the sheer resource hunger of the new generation of models. My 2019 BiLSTM-BERT concoction fit comfortably inside ~200MB. A modern 70B-parameter model weighs 140GB before it even got the first request, a prompt. GPUs that comfortably hosted a few whole ML pipeline five years ago now strained under a single LLM. And these models aren’t static — each conversation grows its own live memory footprint (the KV cache). And then agentic workloads arrived and the math changed again. A pipeline could no longer be equated to “a chain of models”, a model could now recursively call other models or it self, sometimes twenty times in a single user request. The zebra had grown canines.

We’d been adapting LLMs into production workflows for a while, we had built real solutions on BERT, T5, GPT-2 and their variants, and two observations dawned on me.

First

A production pipeline is almost never a single model doing everything. A small classifier for routing, a model for extraction, maybe an embedding model for retrieval, a reranker ensuring the right input goes in. All coreographed together to produce a single answer. Which means, each problem needs a different choreography.

Second

And this is the one that keeps me up at night, adding more models compounds the cost of inference, and within no time, a pipeline that should have healthy unit economics on paper ends up profoundly negative in practice. When that happens something sad and very human starts to take over.

Engineers start treating their deployed models as owned resources. Not “tools we can pick from,” but “assets we’ve already bought and have to justify.” Decisions start revolving around what you have rather than what you need. Adding a new model to a pipeline means new resources, new bills, new conversations with finance, and I’ve watched this happen a hundred times, we as developers end up firing a 70B-parameter LLM to classify and tag a thousand support tickets. Because we already own it. The right tool for the job, a tiny fine-tuned classifier or a SLM, would mean adding a model, and adding a model means pain.

Infrastructure warps the shape of the solution.

I started building the obvious answers. On-demand, event-driven, resource-orchestrated, plug-and-play AI. It worked, but kind of. Orchestrating models as events presented it’s own set of horros and nightmare. One misfire corrupted multiple pipelines. Models sat idle waiting for upstream inference. Under load, each model had its own personality, its own cold-start mood, memory quirks, scaling preferences.

Around this time, Abhishek was deep in Azure running into the same zebra from the infrastructure side. Image caching, pre-warmed node pools, aggressive driver pre-loading and every other trick the gods of cloud computing had published. All clever, but none of them dissolving the underlying problem. The abstractions cloud providers had built for web servers were now being asked to do a job they were never designed for.

We compared notes. The conversations had started sounding like therapy sessions for two people being haunted by the same ghost.

PyTorch’s Triton compiled deployments were the most efficient by a mile — until you measured cold start, and the compile step pushed the first-response latency off a cliff. Every solution was either too expensive, too slow, or destroyed accuracy in a way that would only show up in production weeks into deployment.

And then, and I don’t think either of us remembers the exact moment, but I remember the shape and form of it, Abhishek and I were on a call, mid-gripe, and one of us said it out loud:

Modern AI doesn’t need better tools. They need a fundamentally different treatment.

Not a better orchestrator. Not a smarter autoscaler. Not a faster cold start on top of the same abstractions that were never built for this. Something underneath the orchestrator. Something that treats “many models, one GPU, coordinated execution” as the first-class object — not something bolted onto infrastructure built for web servers. Something, we realized, that looked less like a framework and more like an operating system, but for inference.

The mistake we kept seeing was that we were treating models as deployable services. But the real unit of work had shifted underneath us: many models, shared resources, coordinated execution, and brutal sensitivity to latency, memory and cost.

That’s the discussion where Neurafewz was born.