cbbackup
The cbbackup
tool creates a copy of data from an entire running cluster, an entire bucket, a single node, or a single bucket on a single functioning node.
Syntax
The basic syntax is:
cbbackup [options] [source] [backup-dir] -u [admin] -p [password]
Where:
-
[backup-dir]
The directory for the backup files to be stored. If an empty directory doesn’t exist, it will be created; the parent directory must exist.
The backup directory must only be used by cbbackup and cbrestore .
If you create the backup in a location which may be used by other processes, like /tmp, the structure of the archive may become corrupted.
|
The following syntax example includes a full backup and two incremental backups for a cluster:
cbbackup http://HOST:8091 /backup-42 -u Administrator -p password cbbackup http://HOST:8091 /backup-42 -m diff -u Administrator -p password cbbackup http://HOST:8091 /backup-42 -m diff -u Administrator -p password
If the backup directory exists, the following backup command will automatically be incremental. The only way to generate a full backup is to choose an unused directory. If backing up a Couchbase Server Community Edition cluster, then incremental backup is not available — all backups will be full backups. |
The following syntax example includes a full backup, two differential backups, and one accumulative backup for a single node.
cbbackup couchbase://HOST:8091 /backup-43 -m full --single-node -u Administrator -p password cbbackup couchbase://HOST:8091 /backup-43 -m diff --single-node -u Administrator -p password cbbackup couchbase://HOST:8091 /backup-43 -m diff --single-node -u Administrator -p password cbbackup couchbase://HOST:8091 /backup-43 -m accu --single-node -u Administrator -p password
A full backup task is always triggered for a new sink location no matter what backup mode is specified. |
Description
The backup process writes a copy of data onto the disk.
To create a backup using cbbackup
, the node or cluster needs to be functional.
The cbbackup
, cbrestore
, and cbtransfer
tools do not communicate with external IP addresses for server nodes outside of a cluster.
Backup, restore, or transfer operations are performed on data from a node within a Couchbase Server cluster.
They only communicate with nodes from a node list obtained within a cluster.
This also means that if Couchbase Server is installed with a default IP address, an external hostname cannot be used to access it.
This tool has several different options which you can use to:
-
Backup all buckets in an entire cluster
-
Backup one named bucket in a cluster
-
Backup all buckets on a node in a cluster
-
Backup one named buckets on a specified node
We recommended that cbbackup output be generated on a file system local to the server.
Specifically, back up to a dedicated partition.
A dedicated partition prevents the backup from filling the Couchbase Server data stores and operating system partitions.
|
Avoid routing the cbbackup output to remote share file systems (NFS).
This is because cbbackup files are based on sqlite format which have issues being written to remote file systems.
|
By default, cbbackup
backs up all global secondary indexes in a cluster.
If you are using cbbackup
to backup a single node in the cluster then global secondary index definitions will only be backed up if the node has the index service enabled.
If you backup a single index node, then all index definitions across the cluster will be stored in the backup, not just indexes on that one node.
To backup global secondary indexes, the cluster administrator username and password must be passed as parameters to cbbackup .
|
As with map-reduce views, cbbackup
only stores the definitions of these indexes (i.e which fields are being indexed), not the contents of the index itself.
These definitions are stored in the file index.json
, located in the backup directory.
This tool is located in the following directories:
Operating system | Location |
---|---|
Linux |
|
Windows |
|
Mac OS X |
|
Options
The following are the command options:
Parameters | Description |
---|---|
|
Command line help. |
|
Single named bucket from source cluster to transfer. If the backup directory only contains a single bucket, then that bucket is automatically used. |
|
Use a single server node from the source only, not all server nodes from the entire cluster. This single server node is defined by the source URL. |
|
Backup mode. The mode option takes any one of the following values:
|
|
Transfer only items that match a vbucket ID. |
|
Transfer only items with keys that match a regular expression. |
|
No actual transfer. Just validate parameters, files, connectivity and configurations. |
|
REST username for source cluster or server node. |
|
REST password for cluster or server node. |
|
Transfer data with SSL enablec. |
|
Number of concurrent workers threads performing the transfer. |
|
Verbose logging. More v’s provide more verbosity. Max: -vvv. |
|
Reduce logging verbosity to only include errors. |
|
Provide extra, uncommon configuration parameters. Comma-separated key=val(key-val)* pairs. |
The following are extra, specialized command options with the cbbackup -x
parameter.
-x options | Description |
---|---|
|
Maximum backoff time during the rebalance period. |
|
Transfer this # of bytes per batch. |
|
Transfer this # of documents per batch. |
|
Split backup file on destination cluster if it exceeds the MB. |
|
By default, disable conflict resolution. This option doesn’t work in Couchbase Server versions 4.0 and 4.1 but will be re-implemented in version 4.1.1 and in subsequent versions. |
|
For value 1, transfer only data from a backup file or cluster. |
|
For value 1, transfer only design documents from a backup file or cluster. Default: 0. Back up only design documents which include view and secondary index definitions from a cluster or bucket with the option |
|
Max number of sequential retries if the transfer fails. |
|
For value 0, display extended fields for stdout output. |
|
0 or 1, where 1 retries transfer after a NOT_MY_VBUCKET message. Default: 1. |
|
Amount of bytes for every TCP/IP batch transferred. |
|
For value 1, rehash the partition id’s of each item. This is required when transferring data between clusters with different number of partitions, such as when transferring data from an Mac OS X server to a non-Mac OS X cluster. |
|
Number batches transferred before updating progress bar in console. |
|
Number batches transferred before emitting progress information in console. |
|
By default, start seqno from beginning. |
|
Transfer documents with metadata. Default: 1. Value of 0 is only used when transferring from 1.8.x to 1.8.x. |
|
For value 1, restore data in uncompressed mode. This option is unsupported.
To create backups with compression, use |
Examples
Example 1:
An entire cluster can be backed up. This includes all of the data buckets and all design documents which includes view and secondary index definitions. To backup an entire cluster:
cbbackup http://HOST:8091 ~/backups -u Administrator -p password
Where ~/backups
is the directory where you want to store the data.
When this operation is performed, be aware that cbbackup creates the following directory structure and files in the ~/backups
directory assuming there two buckets in the cluster named my_name
and sasl
and two nodes N1
and N2
:
~/backups bucket-my_name N1 N2 design.json index.json bucket-sasl N1 N2 design.json index.json
Where bucket-my_name
and bucket-sasl
are directories containing data files, view definitions, index definitions and where N1
and N2
are two sets of data files for each node in the cluster.
Example 2:
To backup a single bucket in a cluster:
cbbackup http://HOST:8091 /backups/backup-20120501 -u Administrator -p password \ -b default
In this case, the default bucket in the cluster is specified and the data from the default bucket is backed up.
Example 3:
To backup all data from multiple buckets on a single node:
> cbbackup http://HOST:8091 /backups/ -u Administrator -p password \ --single-node
Example 4:
To backup data from a single bucket on a single node:
cbbackup http://HOST:8091 /backups -u Administrator -p password \ --single-node -b bucket_name
Example 5:
To specify the keys that are backed up using the - k
option.
For example, to backup all keys from a bucket with the prefix ‘object’:
> cbbackup http://HOST:8091 /backups/backup-20120501 -u Administrator -p password \ -b bucket_name -k '^object.*'
Example 6:
The following example creates a backup copy of all design documents which includes view definitions and secondary index definitions from foo-bucket
and store this as design.json
in the directory ~/backup/foo-bucket
.
If a bucket is not specified, design documents for all buckets in the cluster are backed up.
cbbackup http://10.5.2.117:8091 ~/backup -x design_doc_only=1 -u Administrator -p password
Response
The following example response shows only design documents for all buckets being backed up. In this case, the source node had two (2) buckets including the default bucket.
transfer design doc only. bucket msgs will be skipped. transfer design doc only. bucket msgs will be skipped. done
In the following output, two design documents were backed up.
[ { "controllers":{ "compact":"/pools/default/buckets/default/ddocs/_design%2Fddoc1/controller/compactView", "setUpdateMinChanges":"/pools/default/buckets/default/ddocs/_design%2Fddoc1/controller/setUpdateMinChanges" }, "doc":{ "json":{ "views":{ "view1":{ "map":"function(doc){emit(doc.key,doc.key_num);}" }, "view2":{ "map":"function(doc,meta){emit(meta.id,doc.key);}" } } }, "meta":{ "rev":"1-6f9bfe0a", "id":"_design/ddoc1" } } }, { "controllers":{ "compact":"/pools/default/buckets/default/ddocs/_design%2Fddoc2/controller/compactView", "setUpdateMinChanges":"/pools/default/buckets/default/ddocs/_design%2Fddoc2/controller/setUpdateMinChanges" }, "doc":{ "json":{ "views":{ "dothis":{ "map":"function (doc, meta) {\n emit(meta.id, null);\n}" } } }, "meta":{ "rev":"1-4b533871", "id":"_design/ddoc2" } } }, { "controllers":{ "compact":"/pools/default/buckets/default/ddocs/_design%2Fdev_ddoc2/controller/compactView", "setUpdateMinChanges":"/pools/default/buckets/default/ddocs/_design%2Fdev_ddoc2/controller/setUpdateMinChanges" }, "doc":{ "json":{ "views":{ "dothat":{ "map":"function (doc, meta) {\n emit(meta.id, null);\n}" } } }, "meta":{ "rev":"1-a8b6f59b", "id":"_design/dev_ddoc2" } } } ]
Example 7:
The following example requests a full backup of all the data on the specified cluster:
cbbackup -m full http://example.com:8091 /backups/backup-1 -u Administrator -p password
After an initial full backup, incremental backups can be performed. This example requests a differential incremental backup of all the data on the specified cluster:
cbbackup -m diff http://example.com:8091 /backups/backup-1 -u Administrator -p password
This example requests a cumulative incremental backup of all the data on the specified cluster:
cbbackup -m accu http://example.com:8091 /backups/backup-1 -u Administrator -p password