Traffic Data · Backend
OpenTrafficDB
An open platform that collects and serves live traffic data at map scale.
OpenTrafficDB is the data foundation of an independent, self-hostable traffic and navigation stack. It ingests raw OpenStreetMap data and turns it into a routable, query-ready road network that live traffic information can be layered onto — the piece almost every mapping product needs but few can build or own.
The challenge
High-quality traffic and routing data is normally rented from a handful of commercial providers: expensive, metered per request, and closed. The naïve self-hosted alternative — loading a national or continental road graph into a relational database — collapses under its own weight. A country-scale network is tens of millions of nodes and edges; served from SQL it is slow to traverse and punishing on memory, and the import alone can exhaust a large server. The core problem was to make a full road graph portable, fast to read, and cheap to run, while keeping the volatile traffic layer separate so it can update continuously.
The approach
I designed a custom, tile-based binary graph format instead of forcing the network into a database. Nodes, edges and turn restrictions are written into compact .tile files that the routing layer memory-maps and reads directly — no query planner, no serialization overhead in the hot path. PostgreSQL is deliberately reserved for what actually changes: the live traffic tables. The importer was engineered around a strict memory budget, flushing intermediate structures to disk and memory-mapping node coordinates so it stays within a few gigabytes of heap even on very large extracts.
- Clean separation of the static graph (tiles) from the live data (Postgres), so each scales on its own terms.
- Streaming, disk-backed import that runs on modest hardware rather than a memory monster.
- A read model shaped for the consumer — the routing engine — not for generic storage.
The outcome
The result is a traffic-data backbone that can be imported and served on commodity infrastructure, with no per-request licensing and full control of the data. It became the foundation for a routing engine and navigation clients — an entire ecosystem built on top of a format designed, from the first commit, for the workload it actually has to serve.