Source Configuration Options

Reference of the source connector options.

Database

connection.cluster_address

Couchbase Cluster addresses to listen (use comma to specify several).

  • Type: list

  • Importance: high

connection.bucket

Couchbase bucket name.

  • Type: string

  • Importance: high

connection.username

Couchbase username for Couchbase Server 5.0 and later.

  • Type: string

  • Importance: high

  • Default: the bucket name specified by connection.bucket

connection.password

Couchbase password. For Couchbase Server versions prior to 5.0 this is the bucket password. For versions 5.0 and later, this is the password associated with connection.username.

  • Type: password

  • Importance: low

  • Default: ""

connection.timeout.ms

Connection timeout in milliseconds.

  • Type: long

  • Importance: low

connection.ssl.enabled

Use SSL to connect to Couchbase. This feature only available in Couchbase Enterprise.

  • Type: boolean

  • Importance: low

  • Default: false

  • Dependents: connection.ssl.keystore.location, connection.ssl.keystore.password

connection.ssl.keystore.location

The location of the key store file.

  • Type: string

  • Importance: low

connection.ssl.keystore.password

The password of the private key in the key store file.

  • Type: password

  • Importance: low

  • Default: ""

Connector

topic.name

Name of the Kafka topic to publish data to.

  • Type: string

  • Importance: high

use_snapshots

If true, it will only commit into Kafka when full snapshot from Couchbase received.

  • Type: boolean

  • Importance: low

  • Default: false

dcp.message.converter.class

The class name of the message converter to use.

  • Type: string

  • Importance: low

  • Default: "com.couchbase.connect.kafka.handler.source.DefaultSchemaSourceHandler"

event.filter.class

The class name of the event filter to use.

  • Type: string

  • Importance: low

  • Default: "com.couchbase.connect.kafka.filter.AllPassFilter"

couchbase.stream_from

The point in Couchbase history to start streaming from.

  • Possible values:

    • SAVED_OFFSET_OR_BEGINNING - Restart from saved state, or if none, restart from oldest available mutation in Couchbase

    • SAVED_OFFSET_OR_NOW - Restart from saved state, or if none, restart from current Couchbase state

    • BEGINNING - Restart from oldest available mutation in Couchbase (ignore any potential saved state)

    • NOW - Restart from current Couchbase state (ignore any potential saved state)

  • Since: 3.2.2

  • Type: string

  • Importance: low

  • Default: "SAVED_OFFSET_OR_BEGINNING"

couchbase.log_redaction

Optionally tag sensitive values in the log output for later redaction.

  • Possible values:

    • NONE - No redaction is performed.

    • PARTIAL - Only user data is redacted, system and metadata are not.

    • FULL - User, System and Metadata are all redacted.

  • Since: 3.2.3

  • Type: string

  • Importance: low

  • Default: "NONE"

couchbase.compression

To reduce bandwidth usage, Couchbase Server 5.5 and later can send documents to the connector in compressed form. (Messages are always published to the Kafka topic in uncompressed form, regardless of this setting.) If the requested mode is not supported by your version of Couchbase Server, compression will be disabled.

  • Possible values:

    • ENABLED - (default) Couchbase Server decides whether to use compression on a per-document basis, depending on whether the compressed form of the document is readily available. Select this mode to prioritize Couchbase Server performance and reduced bandwidth usage (recommended). Requires Couchbase Server 5.5 or later.

    • DISABLED - No compression. Select this mode to prioritize reduced CPU load for the Kafka connector.

    • FORCED - Compression is used for every document, unless compressed size is greater than uncompressed size. Select this mode to prioritize bandwidth usage reduction above all else. Requires Couchbase Server 5.5 or later.

  • Since: 3.3.0

  • Type: string

  • Importance: low

  • Default: "ENABLED"

couchbase.forceIPv4

In a network environment that supports both IPv4 and IPv6, setting this property to true will force the use of IPv4 when resolving Couchbase Server hostnames.

  • Since: 3.3.0

  • Type: boolean

  • Importance: low

  • Default: false

couchbase.persistence_polling_interval

The frequency at which the connector will poll Couchbase Server to determine which database changes are unlikely to be rolled back. A value of 0 disables polling and causes changes to be published to the Kafka topic as soon as they are received.

If polling is disabled, when rollbacks occur you are more likely to end up with events in the Kafka topic that do not match the actual database state, because they are from an "alternate timeline" in Couchbase Server’s history.

The longer the polling interval, the larger the flow control buffer required in order to maintain steady throughput, since events are buffered until persistence is confirmed.

If present, the value must be 0 or an integer followed by a time unit: (ms = milliseconds, s = seconds)

If the source is an ephemeral bucket (which never persists documents) this value must be set to 0 to disable the persistence check.

  • Since: 3.4.0

  • Type: string

  • Importance: low

  • Default: "100ms"

couchbase.flow_control_buffer

The amount of heap space to reserve for the flow control buffer. This is the amount of data Couchbase Server will push to the connector before awaiting acknowledgement that the data has been published to the Kafka topic.

If present, the value must be an integer followed by a storage size unit: (b = bytes, k = kilobytes, m = megabytes, g = gigabytes)

This value must be smaller than the connect worker’s heap size. To allocate more memory to the connect worker process, set the KAFKA_HEAP_OPTS environment variable before starting the worker. For example: export KAFKA_HEAP_OPTS=-Xmx1024M

  • Since: 3.4.0

  • Type: string

  • Importance: low

  • Default: "128m"

Parent topic: Kafka Connector

Previous topic: Quickstart

Next topic: Sink Configuration Options