Skip to main content

Back Up and Restore Databend Meta Service Cluster

This guideline will introduce how to back up and restore the meta service cluster data.

Export Data From Meta Service

It supports to export from a databend-meta data dir or from a running databend-meta server.

Export from data dir

Shutdown the databend-meta service.

Then export sled DB from the dir(<your_meta_dir>) in which the databend-meta stores meta to a local file output_fn, in multi-line JSON format. E.g., every line in the output file is a JSON of an exported key-value record.

# cargo build --bin databend-metactl

./target/debug/databend-metactl --export --raft-dir "<your_meta_dir>" --db <output_fn>

# tail "<output_fn>"
# ["state_machine/0",{"Nodes":{"key":2,"value":{"name":"","endpoint":{"addr":"localhost","port":28203}}}}]
# ["state_machine/0",{"Nodes":{"key":3,"value":{"name":"","endpoint":{"addr":"localhost","port":28303}}}}]
# ["state_machine/0",{"StateMachineMeta":{"key":"LastApplied","value":{"LogId":{"term":1,"index":378}}}}]
# ["state_machine/0",{"StateMachineMeta":{"key":"Initialized","value":{"Bool":true}}}]
# ...

Note: without the --db argument, the exported data will output to the stdio instead.

Export from a running server

Similar to exporting from data dir, but with the service endpoint argument --grpc-api-address <ip:port> in place of the --raft-dir, where <ip:port> is the grpc_api_address in databend-meta config.toml, e.g.:

./target/debug/databend-metactl --export --grpc-api-address "" --db <output_fn>

# tail "<output_fn>"
# ["state_machine/0",{"Nodes":{"key":2,"value":{"name":"","endpoint":{"addr":"localhost","port":28203}}}}]
# ...

Restore a databend-meta

The following command rebuild a meta service db in <your_meta_dir> from exported metadata:

./target/debug/databend-metactl --import --raft-dir "<your_meta_dir>" --db <output_fn>

databend-meta --raft-dir "<your_meta_dir>" ...

Note: without the --db argument, the import data come from stdio instead, like:

cat "<output_fn>" | ./target/debug/databend-metactl --import --raft-dir "<your_meta_dir>"

Caveat: Data in <your_meta_dir> will be cleared.

Import data as a new databend-meta cluster

With specifies the --initial-cluster argument, the databend-metactl can import the data as a new cluster. The --initial-cluster format is: node_id=raft_advertise_host:raft_api_port,grpc_api_addr, each node config is separated by space, the meaning of raft_advertise_host,raft_api_port,grpc_api_addr is the same in raft config.


/target/debug/databend-metactl --import --raft-dir ./.databend/new_meta1 --id=1 --db meta.db --initial-cluster 1=localhost:29103, 2=localhost:29203, 3=localhost:29303,
/target/debug/databend-metactl --import --raft-dir ./.databend/new_meta2 --id=2 --db meta.db --initial-cluster 1=localhost:29103, 2=localhost:29203, 3=localhost:29303,
/target/debug/databend-metactl --import --raft-dir ./.databend/new_meta3 --id=3 --db meta.db --initial-cluster 1=localhost:29103, 2=localhost:29203, 3=localhost:29303,

The script above imports the exported data from meta.db and initializes the three cluster nodes: id 1, which raft directory is ./.databend/new_meta1, and so are id 2 and 3 with different raft directory. Note that the --initial-cluster argument in these three command line is the same.

After that, can start a new three nodes databend-meta cluster with the new config and imported data.