cbbackupwrapper and cbrestorewrapper

Two tools are included with Couchbase Server Enterprise Edition that improve throughput for the database backup process: cbbackupwrapper and cbrestorewrapper.

cbbackupwrapper and cbrestorewrapper were specifically created to address the single worker thread limitation for one backup/restore target (Bucket, Node). They should be used instead of the standard backup tools, cbbackup and cbrestore.

cbbackupwrapper and cbrestorewrapper are designed to provide a parallel processing approach to backing up a Couchbase Server cluster. At a technical level, they divide each vBucket space into a subset of vBuckets for each backup process and thus improve the speed and efficiency.

In a future release of Couchbase Server, these two tools will be folded into cbbackup and cbrestore respectively and then deprecated.

Syntax

The basic syntax is:

cbbackupwrapper CLUSTER BACKUPDIR OPTIONS
cbrestorewrapper BACKUPDIR CLUSTER OPTIONS

Options

The following are the command options:

Table 1. Basic command options for `cbbackupwrapper`
Option	Description
`-h, --help`	Show this help message and exit.
`-b BUCKET_SOURCE, --bucket-source=BUCKET_SOURCE`	Specify the bucket to backup. Defaults to all buckets.
`--single-node`	Use a single server node from the source only.
`-u USERNAME, --username=USERNAME`	REST username for source cluster or server node. Default is Administrator.
`-p PASSWORD, --password=PASSWORD`	REST password for source cluster or server node.
`-s, --ssl`	Transfer data with SSL enabled.
`-v, --verbose`	Enable verbose messaging.
`--path=PATH`	Specify the path to `cbbackup`. Defaults to current directory.
`--port=PORT`	Specify the bucket port. Defaults to 11210.
`-n NUMBER, --number=NUMBER`	Specify the number of vbuckets per process. Defaults to 100.
`-P PARALLELISM, --parallelism=PARALLELISM`	Number of vbucket backup jobs to run at a time. Defaults to 1.
`-x EXTRA, --extra=EXTRA`	Provide extra, uncommon config parameters; comma-separated key=val(,key=val)* pairs.
`-m MODE, --mode=MODE`	Backup mode: full, diff or accu [default:diff].

Table 2. Additional configuration options for `cbbackupwrapper`
Option	Description
`backoff_cap=10`	Max backoff time during rebalance period.
`batch_max_bytes=400000`	Transfer this # of bytes per batch.
`batch_max_size=1000`	Transfer this # of documents per batch.
`cbb_max_mb=100000`	Split backup file on destination cluster if it exceeds MB.
`data_only=0`	For value 1, only transfer data from a backup file or cluster.
`dcp_consumer_queue_length=1000`	A DCP client needs a queue for incoming documents/messages. A large length is more efficient, but memory proportional to `length*avg`. doc size. Below length 150, performance degrades significantly.
`design_doc_only=0`	For value 1, transfer design documents only from a backup file or cluster.
`flow_control=1`	For value 0, disable flow control to improve throughput.
`max_retry=10`	Max number of sequential retries if transfer fails.
`recv_min_bytes=4096`	Amount of bytes for every TCP/IP call transferred.
`rehash=0`	For value 1, rehash the partition IDs of each item. Rehashing is needed when transferring data between clusters with a different number of partitions, such as when transferring data from an OS X server to a non-OS X cluster.
`report=5`	Number batches transferred before updating progress bar in console.
`report_full=2000`	Number batches transferred before emitting progress information in console.
`seqno=0`	By default, start `seqno` from beginning.
`uncompress=0`	For value 1, restore data in uncompressed mode. This option is unsupported. To create backups with compression, you should use `cbbackupmgr`, which is available for Couchbase Server Enterprise Edition only. See Backup.

Table 3. Basic command options for `cbrestorewrapper`
Option	Description
`-h, --help`	Show this help message and exit.
`-B BUCKET_DESTINATION`	Allows you to specify the destination bucket for a restore (assuming that the bucket in the backup is named differently).
`-b BUCKET_SOURCE, --bucket-source=BUCKET_SOURCE`	Specify the bucket to restore. Defaults to all buckets.
`-u USERNAME, --username=USERNAME`	REST username for source cluster or server node. Default is Administrator.
`-p PASSWORD, --password=PASSWORD`	REST password for source cluster or server node.
`-s, --ssl`	Transfer data with SSL enabled.
`-v, --verbose`	Enable verbose messaging.
`--path=PATH`	Specify the path to `cbrestore`. Defaults to current directory.
`--port=PORT`	Specify the bucket port. Defaults to 11210.
`--from-date=FROM_DATE`	Restore data from the date specified as yyyy-mm-dd. By default, all data from the very beginning will be restored.
`--to-date=TO_DATE`	Restore data till the date specified as yyyy-mm-dd. By default, all data collected will be restored.
`-x EXTRA, --extra=EXTRA`	Provide extra, uncommon config parameters; comma-separated key=val(,key=val)* pairs.

Table 4. Additional configuration options for `cbrestorewrapper`
Option	Description
`backoff_cap=10`	Max backoff time during rebalance period.
`batch_max_bytes=400000`	Transfer this # of bytes per batch.
`batch_max_size=1000`	Transfer this # of documents per batch.
`cbb_max_mb=100000`	Split backup file on destination cluster if it exceeds MB.
`conflict_resolve=1`	By defaault, enable conflict resolution.
`data_only=0`	For value 1, only transfer data from a backup file or cluster.
`dcp_consumer_queue_length=1000`	A DCP client needs a queue for incoming documents/messages. A large length is more efficient, but memory proportional to `length*avg`. doc size. Below length 150, performance degrades significantly.
`design_doc_only=0`	For value 1, transfer design documents only from a backup file or cluster.
`flow_control=1`	For value 0, disable flow control to improve throughput.
`max_retry=10`	Max number of sequential retries if transfer fails.
`recv_min_bytes=4096`	Amount of bytes for every TCP/IP call transferred.
`rehash=0`	For value 1, rehash the partition IDs of each item; this is needed when transferring data between clusters with different number of partitions, such as when transferring data from an OS X server to a non-OS X cluster.
`report=5`	Number batches transferred before updating progress bar in console.
`report_full=2000`	Number batches transferred before emitting progress information in console.
`seqno=0`	By default, start `seqno` from beginning.
`uncompress=0`	For value 1, restore data in uncompressed mode. This option is unsupported. To restore from compressed backups, you should use `cbbackupmgr`, which is available for Couchbase Server Enterprise Edition only. See Backup