Twelve years of distributed SQLite

In September 2014 I published a short post announcing rqlite. The idea was simple: take SQLite, put a Raft log in front of it, and replicate every write to a cluster so that the loss of a node did not mean the loss of the data. The source went on GitHub that same weekend. As far as I can tell, that was the first open-source system to combine Raft and SQLite. Nobody else was doing it, and for several years nobody else tried. Today “distributed SQLite” is a category with a dozen projects in it, a few of them venture-funded, two of them acquired. Since the people now writing about the category tend to start the story around 2021, I thought I should write down the version that starts in 2014. I was there for all of it. 2014: rqlite The need was concrete. A clustered system I worked on at jut.io needed a small amount of relational storage, and the question of which node should hold it had no good answer. Replicating SQLite with a consensus protocol solved it. We went another way at work, but I kept building rqlite because the problem was general and the solution was small. The first version used goraft, the same library InfluxDB and etcd used at the time (I subsequently joined InfluxDB as a core developer around the same time). In early 2016 I replaced it with Hashicorp’s Raft implementation which rqlite still uses. The design decision that mattered most was made then and never changed. rqlite is a CP system. Writes must go through the leader, a majority must acknowledge, and a partitioned minority refuses writes. Everything else, the HTTP API, the read-consistency levels, the snapshotting, followed from that. Before 2014, and alongside it Two things deserve mention for honesty’s sake. Expensify had been replicating SQLite inside its own infrastructure for years with a Paxos-style protocol; they released it as Bedrock in 2016. And ActorDB, an Erlang system that gave each “actor” its own SQLite database replicated by Raft, appeared around the same time as rqlite. Neither was widely adopted outside its origin, but both show the idea was in the air. What rqlite did was make it a standalone, general-purpose, open-source thing anyone could run. 2017: dqlite Canonical built dqlite to get LXD and later MicroK8s an embedded replicated store without running etcd. Mark Shuttleworth actually called me once, and we talked, hoping that rqlite could be made to work, but it never quite fit. dqlite began in Go and was rewritten in C with its own Raft library. It is the …