How We Scaled Our LLM Inference Infrastructure to Serve 100+ Billion Requests a Week
By Christoph Stuber, Machine Learning Engineer, Superhuman, and Myke Troianovskyi, Tech Lead Manager, Superhuman.
A core part of Grammarly is our GEC (Grammatical Error Correction) model, which continuously runs on a user’s writing and provides suggestions to improve its clarity and correctness. Users see these suggestions as red underlines