Deployment Considerations

The following guide describes a set of recommended best practices for a production deployment of Couchbase Mobile.

Security

Authentication

In a Couchbase Mobile production deployment, administrators typically perform operations on the Admin REST API. If Sync Gateway is deployed on an internal network, you can bind the adminInterface of Sync Gateway to the internal network. In this case, the firewall should also be configured to allow external connections to the public interface port.

To access the Admin REST API from an entirely different network or from a remote desktop we recommend to use SSH tunneling.

Authorization

In addition to the Admin REST API, a user can be assigned to a role with additional privileges. The role and the user assigned to it can be created in the configuration file. Then, the Sync Function’s requireRole() method can be used to allow certain operations only if the user has that role.

Data Model Validation

In a NoSQL database, it is the application’s responsibility to ensure that the documents are created in accordance with the data model adopted throughout the system. As an additional check, the Sync Function’s throw() method can be used to reject documents that do not follow the pre-defined data model.

HTTPS

You can run Sync Gateway behind a reverse proxy, such as NGINX, which supports HTTPS connections and route internal traffic to Sync Gateway over HTTP. The advantage of this approach is that NGINX can proxy both HTTP and HTTPS connections to a single Sync Gateway instance.

Alternatively, Sync Gateway can be configured to only allow secure HTTPS connections, if you want to support both HTTP and HTTPS connections you will need to run two separate instances of Sync Gateway.

Database Encryption

Database Encryption is not currently supported in 2.0. This feature will be available in the next version of Couchbase Lite.

Managing Tombstones

By design, when a document is deleted in Couchbase Mobile, they are not actually deleted from the database, but simply marked as deleted (by setting the _deleted property). The reason that documents are not immediately removed is to allow all devices to see that they have been deleted—particularly in the case of devices that may not be online continuously and therefore not syncing regularly.

To actually remove the documents permanently, you need to purge them. This can be done in both Couchbase Lite and via the Sync Gateway REST API.

Beginning in Couchbase Mobile 1.3, documents can have a Time To Live (TTL) value - when the TTL expires the document will be purged from the local database. Note that this does not affect the copy of the document on any other device. This concept is covered in more detail in the Document guide.

Depending on the use case, data model and many more variables, there can be a need to proactively manage these tombstones as they are created. For example, you might decide that if a document is deleted on a Couchbase Lite client, that you want to purge the document (on that device) as soon as the delete has been successfully replicated out to Sync Gateway. Then later on Sync Gateway, set an expiration so that they are automatically purged after a set period (perhaps a week, or a month, to allow for all other devices to sync and receive the delete notifications) - more on this later. If a document is deleted on the Sync Gateway itself (say by a batch process or REST API client), you may similarly want to set a TTL, and on the Couchbase Lite devices you can monitor the Database Change Notifications and purge locally whenever a document is marked as deleted.

Log Rotation

Built-in log rotation

By default, Sync Gateway outputs the logs to standard out with the "HTTP" log key and can also output logs to a file. Prior to 1.4, the two main configuration options were log and logFilePath at the root of the configuration file.

{
    "log": ["*"],
    "logFilePath": "/var/log/sync_gateway/sglogfile.log"
}

In Couchbase Mobile 1.4, Sync Gateway can now be configured to perform log rotation in order to minimize disk space usage.

Log rotation configuration

The log rotation configuration is specified under the logging key. The following example demonstrates where the log rotation properties reside in the configuration file.

{
  "logging": {
    "default": {
      "logFilePath": "/var/log/sync_gateway/sglogfile.log",
      "logKeys": ["*"],
      "logLevel": "debug",
      "rotation": {
        "maxsize": 1,
        "maxage": 30,
        "maxbackups": 2,
        "localtime": true
      }
    }
  },
  "databases": {
    "db": {
      "server": "walrus:data",
      "bucket": "default",
      "users": {"GUEST": {"disabled": false,"admin_channels": ["*"]}}
    }
  }
}

As shown above, the logging property must contain a single named logging appender called default. Note that if the "logging" property is specified, it will override the top level log and logFilePath properties.

The descriptions and default values for each logging property can be found on the Sync Gateway configuration page.

Example Output

If Sync Gateway is running with the configuration shown above, after a total of 3.5 MB of log data, the contents of the /var/log/sync_gateway directory would have 3 files because maxsize is set to 1 MB.

/var/log/sync_gateway
├── sglogfile.log
├── sglogfile-2017-01-25T23-35-23.671.log
└── sglogfile-2017-01-25T22-25-39.662.log

Windows Configuration

On MS Windows logFilePath supports the following path formats.

"C:/var/tmp/sglogfile.log"
`C:\var\tmp\sglogfile.log`
`/var/tmp/sglogfile.log`
"/var/tmp/sglogfile.log"

Log rotation will not work if logFilePath is set to the path below as it is reserved for use by the Sync Gateway Windows service wrapper.

C:\Program Files (x86)\Couchbase\var\lib\couchbase\logs\sync_gateway_error.log

Deprecation notice

The current proposal is to remove the top level log and logFilePath properties in Sync Gateway 2.0. For users that want to migrate to the new logging config to write to a log file but do not need log rotation they should use a default logger similar to the following:

{
    "logging": {
        "default": {
            "logFilePath": "/var/log/sync_gateway/sglogfile.log",
            "logKeys": ["*"],
            "logLevel": "debug"
        }
    }
}

OS log rotation

In production environments it is common to rotate log files to prevent them from taking too much disk space, and to support log file archival.

By default Sync gateway will write log statements to stderr, normally stderr is redirected to a log file by starting Sync Gateway with a command similar to the following:

sync_gateway sync_gateway.json 2>> sg_error.log

On Linux the logrotate tool can be used to monitor log files and rotate them at fixed time intervals or when they reach a certain size. Below is an example of a logrotate configuration that will rotate the Sync Gateway log file once a day or if it reaches 10M in size.

/home/sync_gateway/logs/*.log {
    daily
    rotate 1
    size 10M
    delaycompress
    compress
    notifempty
    missingok

The log rotation is achieved by renaming the log file with an appended timestamp. The idea is that Sync Gateway should recreate the default log file and start writing to it again. The problem is Sync Gateway will follow the renamed file and keep writing to it until Sync gateway is restarted. By adding the copy truncate option to the logrotate configuration, the log file will be rotated by making a copy of the log file, and then truncating the original log file to zero bytes.

/home/sync_gateway/logs/*.log {
    daily
    rotate 1
    size 10M
    copytruncate
    delaycompress
    compress
    notifempty
    missingok
}

Using this approach there is a possibility of loosing log entries between the copy and the truncate, on a busy Sync Gateway instance or when verbose logging is configured the number of lost entries could be large.

In Sync Gateway 1.1.0 a new configuration option has been added that gives Sync Gateway control over the log file rather than relying on stderr. To use this option call Sync Gateway as follows:

sync_gateway -logFilePath=sg_error.log sync_gateway.json

The logFilePath property can also be set in the configuration file at the server level.

If the option is not used then Sync Gateway uses the existing stderr logging behavior. When the option is passed Sync Gateway will attempt to open and write to a log file at the path provided. If a Sync Gateway process is sent the SIGHUP signal it will close the open log file and then reopen it, on Linux the SIGHUP signal can be manually sent using the following command:

pkill -HUP sync_gateway

This command can be added to the logrotate configuration using the 'postrotate' option:

/home/sync_gateway/logs/*.log {
    daily
    rotate 1
    size 10M
    delaycompress
    compress
    notifempty
    missingok
    postrotate
        /usr/bin/pkill -HUP sync_gateway > /dev/null
    endscript
}

After renaming the log file logrotate will send the SIGHUP signal to the sync_gateway process, Sync Gateway will close the existing log file and open a new file at the original path, no log entries will be lost.