Si you are looking for a distributed DBMS that uses SQLite as a storage engine, let me tell you that rqlite is the one for you, since it allows to organize the work of a cluster from storages synchronized with each other.
From the rqlite features, ease of installation, implementation and maintenance is highlighted of distributed storage fault tolerant, which is somewhat similar to etcd and Consul, but it uses a relational data model instead of a key / value format.
About rqlite
The Raft consensus algorithm is used to keep all nodes in sync. Rqlite use original SQLite library and go-sqlite3 driver, in addition to which it executes a layer that processes client requests, replicates itself in other nodes and monitors the consensus reached on the choice of the main node.
Changes to the database can only be made by the node selected as the leader, but the connections with write operations can be directed to other nodes in the cluster, which will return the address of the leader to repeat the request (in the next version, they promise to add automatic forwarding of the call to the leader).
The main focus is on fault tolerance, so the DBMS scale only in read operations, and write operations are the bottleneck. It is possible to run an rqlite cluster from a single node and such a solution can be used to provide access to SQLite over HTTP without providing fault tolerance.
SQLite data in each node they are not stored in a file, but in memory. At the layer level with the implementation of the Raft protocol, a record is kept of all SQLite commands that lead to a change in the database.
This record is used for replication (replication at the query replay level to other nodes), when starting a new node, or to recover from a loss of connectivity.
To reduce the size of the record, automatic packaging is used, which starts after a specified number of changes and leads to the commit of a snapshot, against which a new record begins (the state of the database in the memory is identical to snapshot + accumulated change log).
From the rqlite features:
- Ease of cluster deployment, without the need for a separate SQLite installation.
- Ability to quickly obtain replicated SQL storage.
- Ready to use in production projects.
- Availability of HTTP (S) API, which allows updating data in batch mode and determining the leader node of the cluster. A command line interface and client libraries for various programming languages are also provided.
- The presence of a service to define other nodes that allows you to dynamically create clusters.
- Support for encryption of data exchange between nodes.
- The ability to customize the level of checking for relevance and consistency of data when reading.
- Optional ability to connect read-only nodes that do not participate in consensus determination and are used to increase the scalability of the cluster for read operations.
- Support for the native form of transactions based on combining commands in a single request (transactions based on BEGIN, COMMIT, ROLLBACK, SAVEPOINT, and RELEASE are not supported).
About rqlite 6.0
The new version introduces significant architectural changes aimed at improving cluster reliability by improving the process of directing read and write requests to the correct cluster nodes.
Rqlite nodes now can multiplex multiple logical connections between them using the TCP connections established between the nodes by the Raft protocol. If the request requires the authority of the leader node, but is sent to the secondary node, the secondary node can determine the address of the leader and transmit it to the client, without performing the Raft consensus calculation.
The change also removed the separate component for syncing metadata and removed Raft's separate handling of status and metadata.
Secondary nodes now send requests to the lead node only when necessary, when it is necessary to find out the address of the lead node. The API provides the ability to obtain information about the status of other nodes in the cluster. Sysdump command added to CLI.
Finally if you are interested in knowing more about it or consult the installation instructions and user manual, you can do it from the link below.