Upgrade

This section is an overview of the different options to upgrade a running cluster to the latest version of Sync Gateway and Couchbase Server. For a complete list of instructions, we recommend to follow the upgrade section in the travel sample tutorial. You will learn how to perform a rolling upgrade and enable the shared bucket access introduced in Sync Gateway 1.5 in order to use N1QL, Mobile and Server SDKs on the same bucket.

Sync Gateway

In each of the scenarios described below, the upgrade process will trigger views in Couchbase Server to be re-indexed. During the re-indexing, operations that are dependent on those views will not be available. The main operations relying on views to be indexed are:

A user requests data that doesn’t reside in the channel cache.
A new channel or role is granted to a user in the Sync Function.

The unavailability of those operations may result in some requests not being processed. The duration of the downtime will depend on the data set and frequency of replications with mobile clients. To avoid this downtime, it is possible to pre-build the view index before directing traffic to the upgraded node (see the view indexing section).

From To Steps

From	To	Steps
1.3	1.4	A rolling upgrade is supported: modify your load balancer’s config to stop any HTTP traffic going to the node that will be upgraded, perform the upgrade on the given node and re-balance the traffic across all nodes. Repeat this operation for each node that needs to be upgraded.
1.4	1.5 xattrs disabled	A rolling upgrade is supported: modify your load balancer’s config to stop any HTTP traffic going to the node that will be upgraded, perform the upgrade on the given node and re-balance the traffic across all nodes. Repeat this operation for each node that needs to be upgraded.
1.5 xattrs disabled	1.5 xattrs enabled	A rolling upgrade is supported: modify your load balancer’s config to stop any HTTP traffic going to the node that will be upgraded, perform the upgrade on the given node and re-balance the traffic across all nodes. Repeat this operation for each node that needs to be upgraded. The mobile metadata for existing documents is automatically migrated. The first node to be upgraded should have the `import_docs=continuous` property enabled.
1.4	1.5 xattrs enabled	This upgrade, if done directly, will result in application downtime because all the nodes must be taken offline during the upgrade. The first node to be restarted should have the `import_docs=continuous` property enabled. That being said, it is possible to avoid this downtime by running the 2 upgrade paths mentioned above (first, an upgrade from 1.4 to 1.5, and second, an upgrade from 1.5 to 1.5 with xattrs enabled).

1.3

1.4

A rolling upgrade is supported: modify your load balancer’s config to stop any HTTP traffic going to the node that will be upgraded, perform the upgrade on the given node and re-balance the traffic across all nodes. Repeat this operation for each node that needs to be upgraded.

1.4

1.5 xattrs disabled

1.5 xattrs enabled

A rolling upgrade is supported: modify your load balancer’s config to stop any HTTP traffic going to the node that will be upgraded, perform the upgrade on the given node and re-balance the traffic across all nodes. Repeat this operation for each node that needs to be upgraded.
The mobile metadata for existing documents is automatically migrated.
The first node to be upgraded should have the import_docs=continuous property enabled.

1.4

1.5 xattrs enabled

This upgrade, if done directly, will result in application downtime because all the nodes must be taken offline during the upgrade.
The first node to be restarted should have the import_docs=continuous property enabled.

That being said, it is possible to avoid this downtime by running the 2 upgrade paths mentioned above (first, an upgrade from 1.4 to 1.5, and second, an upgrade from 1.5 to 1.5 with xattrs enabled).

Enabling convergence on your existing deployment (i.e XATTRs) is not reversible. It is recommended to test the upgrade on a staging environment before upgrading the production environment.

Couchbase Server

All of the different upgrade paths mentioned above assume that Couchbase Server is running a compatible version for Sync Gateway. There are 3 commonly used upgrade paths for Couchbase Server. Depending on the one you choose, there may be additional consideration to keep in mind when using Sync Gateway:

Upgrade Strategy	Downtime	Additional Machine Requirements	Impact when using Sync Gateway
Rolling Online Upgrade	None	Low	Potential transient connection errors. The Couchbase Server re-balance operations can result in transient connection errors between Couchbase Server and Sync Gateway, which could result in Sync Gateway performance degradation. Potential for unexpected server errors during re-balance. There is an increased potential to lose in-flight ops during a fail-over.
Upgrade Using Inter-cluster Replication	Small amount during switchover	High - duplicate entire cluster	Using an XDCR (Cross Data Center Replication) approach will have incur some Sync Gateway downtime, but less downtime than other approaches where Sync Gateway is shutdown during the entire Couchbase Server upgrade. It’s important to note that the XDCR replication must be a one way replication from the existing (source) Couchbase Server cluster to the new (target) Couchbase Server cluster, and that no other writes can happen on the new (target) Couchbase Server cluster other than the writes from the XDCR replication, and no Sync Gateway instances should be configured to use the new (target) Couchbase Server cluster until the last step in the process. Start XDCR to do a one way replication from the existing (source) Couchbase Server cluster to the new (target) Couchbase Server cluster running the newer version. Wait until the target Couchbase Server has caught up to all the writes in the source Couchbase Server cluster. Shutdown Sync Gateway to prevent any new writes from coming in. Wait until the target Couchbase Server has caught up to all the writes in the source Couchbase Server cluster — this should happen very quickly, since it will only be the residual writes in transit before the Sync Gateway shutdown. Reconfigure Sync Gateway to point to the target cluster. Restart Sync Gateway. Caveats: Small amount of downtime during switchover. Since there may be writes still in transit after Sync Gateway has been shutdown, there will need to be some downtime until the target Couchbase Server cluster is completely caught up. XDCR should be monitored. Make sure to monitor the XDCR relationship as per XDCR docs.
Offline Upgrade	During entire upgrade	None	Take Sync Gateway offline Upgrade Couchbase Server using any of the options mentioned in the Upgrading Couchbase Server documentation. Bring Sync Gateway online

Upgrade Strategy

Downtime

Additional Machine Requirements

Impact when using Sync Gateway

Rolling Online Upgrade

None

Low

Potential transient connection errors. The Couchbase Server re-balance operations can result in transient connection errors between Couchbase Server and Sync Gateway, which could result in Sync Gateway performance degradation.

Potential for unexpected server errors during re-balance. There is an increased potential to lose in-flight ops during a fail-over.

Upgrade Using Inter-cluster Replication

Small amount during switchover

High - duplicate entire cluster

Using an XDCR (Cross Data Center Replication) approach will have incur some Sync Gateway downtime, but less downtime than other approaches where Sync Gateway is shutdown during the entire Couchbase Server upgrade.

It’s important to note that the XDCR replication must be a one way replication from the existing (source) Couchbase Server cluster to the new (target) Couchbase Server cluster, and that no other writes can happen on the new (target) Couchbase Server cluster other than the writes from the XDCR replication, and no Sync Gateway instances should be configured to use the new (target) Couchbase Server cluster until the last step in the process.

Start XDCR to do a one way replication from the existing (source) Couchbase Server cluster to the new (target) Couchbase Server cluster running the newer version.
Wait until the target Couchbase Server has caught up to all the writes in the source Couchbase Server cluster.
Shutdown Sync Gateway to prevent any new writes from coming in.
Wait until the target Couchbase Server has caught up to all the writes in the source Couchbase Server cluster — this should happen very quickly, since it will only be the residual writes in transit before the Sync Gateway shutdown.
Reconfigure Sync Gateway to point to the target cluster.
Restart Sync Gateway.

Caveats:

Small amount of downtime during switchover. Since there may be writes still in transit after Sync Gateway has been shutdown, there will need to be some downtime until the target Couchbase Server cluster is completely caught up.

XDCR should be monitored. Make sure to monitor the XDCR relationship as per XDCR docs.

Offline Upgrade

During entire upgrade

None

Take Sync Gateway offline
Upgrade Couchbase Server using any of the options mentioned in the Upgrading Couchbase Server documentation.
Bring Sync Gateway online

View Indexing

Sync Gateway uses Couchbase Server views to index and query documents. When Sync Gateway starts, it will publish a Design Document which contains the View definitions (map/reduce functions). For example, the Design Document for Sync Gateway is the following:

{
   "views":{
      "access":{
         "map":"function (doc, meta) { ... }"
      },
      "channels":{
         "map":"function (doc, meta) { ... }"
      },
      ...
   },
   "index_xattr_on_deleted_docs":true
}

Following the Design Document creation, it must run against all the documents in the Couchbase Server bucket to build the index which may result in downtime. During a Sync Gateway upgrade, the index may also have to be re-built if the Design Document definition has changed. To avoid this downtime, you can publish the Design Document and build the index before starting Sync Gateway by using the Couchbase Server REST API. The following curl commands refer to a Sync Gateway 1.3 → Sync Gateway 1.4 upgrade but they apply to any upgrade of Sync Gateway or Accelerator.

Start Sync Gateway 1.4 with Couchbase Server instance that isn’t your production environment. Then, copy the Design Document to a file with the following.
```
$ curl localhost:8092/<BUCKET_NAME>/_design/sync_gateway/ > ddoc.json
```

Create a Development Design Document on the cluster where Sync Gateway is going to be upgraded from 1.3:

$ curl -X PUT http://localhost:8092/<BUCKET_NAME>/_design/dev_sync_gateway/ -d @ddoc.json -H "Content-Type: application/json"

This should return:

{"ok":true,"id":"_design/dev_sync_gateway"}

Run a View Query against the Development Design Document. By default, a Development Design Document will index one vBucket per node, however we can force it to index the whole bucket using the full_set parameter:
$ curl "http://localhost:8092/sync_gateway/_design/dev_sync_gateway/_view/role_access_vbseq?full_set=true&stale=false&limit=1"
This may take some time to return, and you can track the index’s progress in the Couchbase Server UI. Note that this will consume disk space to build an almost duplicate index until the switch is made.
Upgrade Sync Gateway. When Sync Gateway 1.4 starts, it will publish the new Design Document to Couchbase Server. This will match the Development Design Document we just indexed, so will be available immediately.