Storage

Couchbase Server stores certain items in compressed form on disk; and, whenever required, removes them. This allows data-sets to exceed the size permitted by existing memory-resources; since undeleted items not currently in memory can be restored to memory from disk, as needed. It also facilitates backup-and-restore procedures.

Generally, a client’s interactions with the server are not blocked during disk-access procedures. However, if a specific item is being restored from disk to memory, the item is not made available to the client until the item’s restoration is complete.

Not all items are written to disk: Ephemeral buckets and their items are maintained in memory only. See Buckets for information.

Items written to disk are always written in compressed form. Based on bucket configuration, items may be maintained in compressed form in memory also. See Compression for information.

Items can be removed from disk based on a configured point of expiration, referred to as Time-To-Live. See Expiration for information.

For illustrations of how Couchbase Server saves new and updates existing Couchbase-bucket items, thereby employing both memory and storage resources, see Memory and Storage.

Synchronized, multi-threaded readers and writers provide simultaneous, high-performance read-write operations for data on disk. Conflicts are avoided by assigning each thread (reader or writer) a specific subset of the 1024 vBuckets for each Couchbase bucket. For example, the first of five reader-threads is assigned vBuckets 0, 5, 10, 15, 20, and so on; while the second is assigned vBuckets 1, 6, 11, 16, 21, and so on. Thread-status can be viewed, by means of the cbstats command, specified with the raw workload option.

See cbstats and vBuckets for information.

Items can be deleted by a client application: either by immediate action, or by setting a Time-To-Live (TTL) value: this value is established through accessing the TTL metadata field of the item, which establishes a future point-in-time for the item’s expiration. When the point-in-time is reached, Couchbase Server deletes the item.

Following deletion by either method, a tombstone is maintained by Couchbase Server, as a record (see below).

An item’s TTL can be established either directly on the item itself, or via the bucket that contains the item. For information, see Expiration.

A tombstone is the record of a deleted item. Each tombstone includes the item’s key and metadata. Tombstones are maintained in order to provide eventual consistency both between nodes and between clusters.

The Metadata Purge Interval establishes the frequency with which Couchbase Server purges itself of tombstones: which means, removes them fully and finally. The Metadata Purge Interval setting runs as part of auto-compaction.

For more information, see Post-Expiration Purging, in Expiration.

Disk I/O — reading items from and writing them to disk — does not block client-interactions: disk I/O is thus considered a background task. The priority of disk I/O (along with that of other background tasks, such as item-paging and DCP stream-processing) is configurable per bucket. This means, for example, that one bucket’s disk I/O can be granted priority over another’s. For further information, see Create a Bucket.