Suhan Ramani
Backend and distributed systemsIIT Jodhpur
I build backend systems, and I am mostly interested in the point where they stop being correct under load.
So far that has meant retrieval that had half a second to answer, a ledger where the hard part turned out to be concurrency rather than arithmetic, and a lot of time spent on why a query stopped being fast.
Decisions · 15
Every project I have worked on came down to a handful of choices, each with an obvious alternative I did not take. These are those choices. Open one to see the reasoning.
Machine learning
Constraint. Streaming webcam frames to a server for inference puts the whole video payload on the network path.
Why this. MediaPipe can extract hand landmarks in the browser, so what actually crosses the network is a small vector of coordinates rather than an image. Normalising those features also decouples accuracy from lighting, which a server-side pipeline would have had to correct for anyway.
Machine learning
Constraint. At a sub-100ms per-frame budget, HTTP request overhead is a meaningful fraction of the budget.
Why this. Every polled request pays connection setup again, which makes latency bursty rather than predictable. A persistent WebSocket removes that cost entirely and makes the frame budget something you can actually reason about, which matters more for an interactive demo than raw throughput does.
Under 100ms per frame, end to end.
Machine learning
Constraint. The model was accurate enough and too slow to feel responsive.
Why this. Batch processing and quantisation lifted throughput 40% and cut the model to a quarter of its size, at under 2% accuracy loss. For something a person interacts with in real time, the delay is the thing they notice; the last two points of accuracy are not.
96%+ across 26 classes, 40% more throughput, 4x smaller.
Concurrency
Constraint. Several people editing one shared ledger at once, on mobile clients that may be offline mid-edit.
Why this. Mutable balances need locking, and locking assumes connectivity and a coordinator — neither of which a phone reliably has. Storing every expense as an immutable event and folding the log into a balance removes the race condition rather than defending against it. There is nothing to contend over, so there is nothing to lock.
Zero data conflicts in testing across 5+ concurrent users per session.
Concurrency
Constraint. Settling a shared ledger pairwise generates far more transfers than the group actually needs.
Why this. Who owes whom is a graph problem, not an accounting one. A minimum cash flow pass runs in O(n log n) and reduces any group of n people to at most n-1 transactions, which is the floor. The naive pairwise version is correct and asks people to make payments that cancel each other out.
Up to 70% fewer payments than settling pairwise.
Databases
Constraint. High-traffic endpoints were slow and a cache was the obvious first move.
Why this. A cache in front of an N+1 query hides the problem and adds an invalidation bug to maintain. Restructuring the indexes and eliminating the N+1 patterns made the underlying reads fast, which is a smaller system to reason about than the same reads plus a cache layer.
Around 35% lower data-retrieval latency.
Systems
Constraint. Four parallel document-ingestion streams were serialising behind synchronous I/O.
Why this. The work was almost entirely waiting — on the embedding API and on the vector store — so threads were sitting idle rather than computing. Async I/O in FastAPI let the streams interleave their waiting, which is where the bottleneck actually was.
Systems
Constraint. A document that fails to embed still leaves the pipeline running, just with worse retrieval.
Why this. Silent degradation is the worst failure mode in a retrieval system: nothing errors, results simply get less relevant, and you find out weeks later from a user. Validating at ingestion turns an invisible quality problem into a visible failure someone can fix.
Every observed embedding failure eliminated in testing.
Risk modelling
Constraint. A model trained to agree with the triage nurse learns to reproduce the nurse's misses, and in the 2022 NHAMCS data 45.8% of critical outcomes came from patients triaged level 3, 4 or 5.
Why this. Optimising for agreement optimises for the wrong target. The engine is trained on outcomes rather than on assigned levels, and it keeps watching patients while they wait — because triage is a moment and deterioration is a process. Repeat observations re-run the assessment and escalate on their own.
56.7% of critical outcomes placed at level 1-2 against 39.0% for the recorded triage level, at the same share of arrivals.
Risk modelling
Constraint. Under-triage and over-triage are not equally bad, but taking the most likely level treats them as if they are.
Why this. Missing a critical patient can kill them; escalating a well one costs a bed and some attention. So the level is chosen from the probability distribution with an explicitly asymmetric loss function rather than by argmax. The price of that choice is stated rather than hidden, because a triage assistant that over-triages everything gets worked around within a week.
50.6% of under-triaged critical patients rescued, for 18.9% of non-critical patients escalated.
Risk modelling
Constraint. A heart rate of 152 is unremarkable in a one-year-old and an emergency in a seventy-year-old.
Why this. Passing age in as one more column asks the model to learn that relationship from scarce paediatric data. Instead vital signs enter as deviations from ten age bands, estimated from the survey's own low-acuity discharged population so the reference describes well physiology at that age rather than the ED case-mix. Where a band is too thin to estimate the artifact records that it fell back to published values, and the assistant lowers its own confidence there.
Machine learning
Constraint. An attribution model is free to report that traffic contributed negatively to the air quality in a ward, which means nothing to the person deciding where to send an inspector.
Why this. Ordinary least squares will happily produce that, and it fits better for it. Non-negative least squares constrains every coefficient at or above zero, so the breakdown stays physically interpretable and can be defended when it sends someone somewhere. Fit quality is worth less here than being able to explain the answer.
Machine learning
Constraint. A forecast can look impressive in isolation and still be worse than assuming the next hour looks like this one.
Why this. Persistence is the honest baseline for anything time-series, so it is in the test suite: mean RMSE has to come in at least 10% under it or the build fails. The evaluation report leads with the caveat that the holdout is synthetic rather than with the headline improvement, because a number nobody qualified is a number that will be quoted back wrongly.
Systems
Constraint. An MCP server speaks JSON-RPC over stdio, so a single stray print statement corrupts the stream and the client disconnects.
Why this. There is no way to have both a debug log and a working server on the same channel. Every diagnostic goes to stderr, which the host shows and the parser never reads. The README is also truncated to 2,000 characters before it is returned — context is the scarce resource for an agent, and spending it on the back half of a readme leaves less for the task.
Concurrency
Constraint. A copied server object means two objects holding the same file descriptor, and the second destructor closes a socket that is already gone.
Why this. The copy constructor and copy assignment are deleted outright and move semantics provided instead, so ownership transfers rather than duplicates. The compiler then refuses the bug at the point it would be written, which is cheaper than finding it as an intermittent failure once several clients are connected.
Work
All 6 projectsCareQueue
Team lead, Team InnovX
A triage assistant built around the patients who are missed rather than the ones who are obvious: it scores risk from outcomes instead of from assigned levels, and keeps watching everyone in the waiting room for deterioration.
SignEase
Solo
A sequence-aware Transformer that reads American Sign Language from a webcam, served by a low-latency inference microservice and streamed back to the browser frame by frame.
Spendly
Solo
A group expense tracker built as an append-only ledger, so concurrent edits from several people cannot conflict, and a settlement algorithm that minimises the number of payments.
Aeris
Solo
Ward-level AQI forecasting with source attribution and inspection ranking, served by a FastAPI backend behind a React dashboard.
Writing
All writingNothing published yet. The first few will be write-ups of the decisions above.
Background
More about meTwo internships so far: a web agency where I spent most of my time on query shapes and production incidents, and an AI learning platform where I built the retrieval pipeline. On campus I coordinate the Tinkerers’ Lab, set problems for the programming society, and help run placement outreach.