LLM Gateway
A provider-neutral LLM gateway: code asks for a job, the gateway routes it through a chain of models with safe failover, a deadline per call, and the cost of every call written to a ledger.
- 1The problem
Call sites that name a vendor and model lock a product to one provider, hide what each feature really costs, and break when a provider rate-limits, stalls or runs out of credit.
- 2Our contribution
Designed and built end to end by Kumar Sahil.
- 3What we built
Job-based routing (vision, QC, research, fast…) over ordered provider:model chains; failure classification that decides per error whether to retry, fail over or stop, so malformed requests are never resent to a second vendor; whole-call deadlines; env-var rerouting without a deploy; a SQLite cost ledger per job. Standard library only, no vendor SDKs.
- 4The result
Released as open source. Read the code, tests and design notes on GitHub.
Other systems we’ve built
Reply Classifier Cascade
A cost-ordered classifier for English and Dutch cold-email replies that decides which replies become CRM deals, built so a false “hot deal” needs several layers to agree.Read the case studyLLM Eval Gauntlet
Invariant-checked evaluation suites for tool-calling agents that run in CI and fail the build when an agent invents a number, leaks its prompt or obeys an injection.Read the case studyFind the workflow worth automating first.
Tell us about one process that costs your team hours every week. We will tell you honestly whether AI or automation is the right fix, and what a first build would take.