Archive

Why Toppr started Go(ing) at scale

technical
2021 Originally published on the Toppr Engineering Blog · co-authored with Santhosh Reddy

Introduction

Over eight years, Toppr built its products using Django, a Python web framework, chosen for its convention-driven development speed. That served us well — until the pandemic hit and traffic to Toppr Answr, our homework help platform, grew faster than the architecture could absorb. This is the story of how we diagnosed what was actually wrong and why Go was the right answer.

Why Python, and why it stopped working

Python earned its place at Toppr for good reasons: it accelerated feature shipping, the ORM reduced boilerplate, it attracted talented developers, and the syntax was expressive enough that engineers could move between services without a steep ramp. For most of the company's history, those advantages outweighed anything else on the table.

Toppr Answr is an SEO-driven Q&A platform with three hard SLA requirements: low latency, high throughput, and high availability. Search engine bots generate unpredictable traffic spikes. Any performance hiccup doesn't just affect users — it affects rankings, which affects traffic, which compounds. When Answr's traffic grew significantly over six to eight months during the pandemic, those requirements stopped being theoretical.

The core problem was the Global Interpreter Lock. The GIL restricts Python to one thread holding the interpreter lock at a time, making multithreading effectively single-threaded under CPU load. In theory, GIL doesn't impact I/O-bound applications. In practice, I/O-bound apps spend more time on CPU than the theory suggests — even simple CRUD operations spend roughly 20% of execution time on CPU-bound Python code, and that's where GIL contention bites.

The response to GIL is to use multiple processes instead of threads. Multiple processes work, but they carry significant memory overhead. Each process is a full Python interpreter with its own heap. Memory constraints per machine meant we couldn't fully utilise available CPU, so we were spinning up additional machines to absorb traffic spikes — and every additional machine cascaded costs to upstream services and data sources. We were paying for capacity we couldn't actually use.

What we needed in a replacement

Before evaluating anything, we wrote down what the replacement runtime actually had to have:

Why Go

Go sits at an unusual intersection: it has enough low-level control to be fast and efficient, and enough high-level ergonomics that engineers can be productive in days rather than weeks. That combination is rarer than it sounds.

The concurrency model was the decisive factor. A goroutine is a lightweight user-level thread managed by the Go runtime — cheap to create, cheap to maintain, and multiplexed onto kernel threads by a scheduler that understands your workload. The scheduler uses per-core run queues, steals work across cores to balance load, and preempts long-running goroutines. The net effect is that Go handles thousands of concurrent operations without the memory overhead of the equivalent thread-per-request model.

The philosophy behind Go's concurrency is worth naming explicitly: don't communicate by sharing memory; share memory by communicating. Channels make the flow of data between goroutines explicit, which makes the class of locking bugs that plague shared-memory concurrency much harder to write.

The practical advantages stacked up too: cross-compilation generates architecture-specific binaries easily, the language specification is 50 pages (you can read it in an afternoon), it's a mature Google-backed open-source project, and REST service development is well-supported. Benchmarks against our Python baseline showed Go achieving at least 10x better throughput on equivalent hardware.

Migration strategy

Answr is read-heavy. The migration prioritised read APIs first — highest impact, clearest ROI, and the path that freed resources fastest while giving the team time to build fluency in the new stack. We didn't do a big-bang rewrite. We migrated service by service, kept both versions running in parallel during transitions, and used the headroom each migration created to fund the next one.

What we learned along the way

Framework. Gin proved to be the right HTTP framework for us — high performance, aligned with Go's simplicity philosophy, and well-matched to the kind of REST services we were building.

Monitoring. DataDog gave us production visibility we needed during the migration. Apache Bench and Go's built-in pprof profiler were essential for identifying and fixing memory leaks and goroutine leaks — flame graphs in particular made slow code paths obvious in a way that logs alone can't.

Dependency injection. Uber's Fx framework let us manage cross-cutting concerns as injectable dependencies rather than globals or init-time side effects. Less code rewriting, cleaner service boundaries, better developer experience as the team grew.

Developer experience. Go's imperative style combined with CSP-style concurrency via channels turned out to be immediately approachable for engineers coming from a Python background. The learning curve we'd anticipated was shorter than expected.

Results

The numbers were unambiguous:

Toppr Answr reached #2 in India and #15 globally in education sector internet traffic. Our DevOps engineer put it plainly: "We've seen a significant reduction in server costs, ease of operation increased, better resource utilisation helped shave costs across the stack."

Go is now the default technology choice for most teams at Toppr. We didn't migrate because Go was fashionable — we migrated because we had a specific, diagnosed problem with a specific root cause, and Go addressed it directly. That's the only migration worth doing.