MongoDB v4+

11 min

The MongoDB v4+ connector writes Litmus Edge (LE) messages into a MongoDB collection. It connects to a standalone server, a replica set, or a MongoDB Atlas deployment. The connector inserts each published JSON message into the target collection as a document, and writes documents in batches.

The connector also supports Execute Command for ad-hoc MongoDB commands.

Supported communication

Outbound (write data from LE to the database): Yes

Inbound (read data from the database to LE): No

Execute Command (ad-hoc MongoDB command): Yes

Compatibility

The connector supports the following products. Other products and versions might work.

Product	Version
MongoDB	5.0 and later. Validated on 7.0
MongoDB Atlas	-
Amazon DocumentDB (MongoDB-compatible)	-

Connector configuration

Field	Type	Description	Required	Default
Name	String	Name of this connector instance.	Yes	-
Connection scheme	Dropdown	Connection string scheme: mongodb or mongodb+srv. See Connection schemes.	Yes	mongodb
Hostname	String	DNS name or IP address of the MongoDB host. The host must be reachable from the LE system. For mongodb+srv, enter the SRV record name, such as cluster0.mongodb.net. Length is 3 to 255 characters.	Yes	-
Port	Integer	Port of the MongoDB host. Applies only when Connection scheme is mongodb. Range is 1 to 65535.	Conditional	27017
SSL Mode	Dropdown	TLS connection mode: disable, require, verify-ca, or verify-full. See SSL modes.	No	disable
CA Certificate	String (Secret)	CA certificate that the connector uses to verify the server. You can also add it to the system-wide certificate list. Applies only when SSL Mode is not disable.	No	-
Certificate	String (Secret)	PEM client certificate for mutual TLS authentication. Applies only when SSL Mode is not disable. Enter it together with Private key.	No	-
Private key	String (Secret)	Private key that matches the client certificate. Applies only when SSL Mode is not disable. Enter it together with Certificate.	No	-
Username	String	Database user. Leave it empty for an anonymous connection. See Authentication.	No	-
Password	String (Secret)	Password for the database user. If you enter a password, you must also enter Username.	No	-
Database	String	Name of the target database.	Yes	-
Auth database	String	Authentication database (authSource). Set it to the target database name only if the user was created inside the target database.	No	admin
Auth mechanism	Dropdown	Authentication mechanism: DEFAULT, SCRAM-SHA-256, SCRAM-SHA-1, PLAIN, or MONGODB-X509. See Authentication.	No	DEFAULT
Collection	String	Target collection that the connector writes into. The collection must exist in the database unless you enable Create collection.	Yes	-
Mapping	Object	Key-value pairs in the form {"db_field": "msg_key"}, where db_field is a field in the stored document and msg_key is a top-level key in the message. Prefix a value with @ to store a static value. Leave it empty to store each JSON message as published. See Document shape.	No	-
Enable JSONPath	Boolean	Read Mapping values that start with $ as JSONPath expressions, such as $.factory.line.machine.sensors.temperature. Values that don't start with $ keep their usual meaning, and an expression that matches nothing stores null.	No	Disabled
Create collection	Boolean	Create the target collection when the connector connects, if the collection doesn't exist. The database user needs permission to create a collection.	Yes	Disabled
Commit timeout	Integer	Maximum time in milliseconds that one batch insert can take before the connector treats it as failed and retries it. Range is 10 to 65535.	No	30000
Batch timeout	Integer	Maximum time in milliseconds after the previous insert attempt before the connector writes a batch that isn't full. Range is 10 to 65535.	No	500
Max transaction size	Integer	Maximum number of documents in one batch insert. The connector writes a full batch without waiting for Batch timeout. Range is 1 to 65535.	No	500
Bulk insert count	Integer	Number of messages the connector buffers before it writes a batch. A value of 0 uses Max transaction size. Range is 0 to 65535.	No	0
Write mode	Dropdown	When a message counts as delivered: sync or async. See Write modes.	No	sync
Parallel transactions	Integer	Maximum number of batch inserts that run at the same time, each on its own database connection. Values above 1 increase throughput, but the connector can then write documents out of publish order. Range is 1 to 64.	No	1
Parallel Publish Count	Integer	Number of messages the connector publishes at the same time. In sync write mode, this value limits how many documents a batch can collect. Range is 1 to 4000.	No	1000

Additional attributes

Field	Type	Description	Required	Default
Throttling limit	Integer	Maximum outbound messages per second. A value of 0 disables throttling. Minimum is 0.	No	0
Persistent storage	Boolean	Store outbound queue messages on disk. If the connection drops or the system loses power, the messages are retained and sent once the connection is restored.	No	Enabled
Persistent storage size limit	Integer	Maximum disk space in GB used for persisted data. Applies only when Persistent storage is enabled. Minimum is 1.	Conditional	1
Queue Mode	Dropdown	Order in which the connector sends buffered messages. fifo sends the earliest message first. lifo sends the latest message first. Applies only when Persistent storage is enabled.	No	lifo
Enable Logging	Boolean	Write connection and publish error messages to the connector log. Logging can increase CPU usage, so enable it only for debugging.	No	Disabled

Connection schemes

Value	Behavior
mongodb	Connects to a single host on the port in Port, in the form mongodb://host:port.
mongodb+srv	Resolves the replica set members through DNS SRV records, in the form mongodb+srv://host. MongoDB Atlas uses this form. The connector ignores Port and always uses TLS.

SSL modes

Value	Behavior
disable	No TLS. The connection is plaintext, and the connector ignores the certificate fields. With mongodb+srv, the connection is still encrypted but not verified.
require	The connector uses TLS but doesn't verify the server certificate, and doesn't use CA Certificate.
verify-ca	The connector verifies the server certificate chain against CA Certificate, or against the system trust store. It doesn't check the host name.
verify-full	The connector verifies the certificate chain, and the host name on the certificate must match Hostname. Use this mode in production.

Authentication

Method	Configuration
User name and password	Enter Username and Password. Set Auth mechanism to DEFAULT, SCRAM-SHA-256, SCRAM-SHA-1, or PLAIN. DEFAULT lets the driver choose SCRAM-SHA-256 or SCRAM-SHA-1. Use PLAIN for LDAP passthrough.
Client certificate (X.509)	Set Auth mechanism to MONGODB-X509, enter Certificate and Private key, and set SSL Mode to require, verify-ca, or verify-full. You can't save this configuration with SSL Mode set to disable.
Anonymous	Leave Username and Password empty, and set Auth mechanism to DEFAULT. The MongoDB server must allow unauthenticated access.

You can't save a configuration that has a Password but no Username.

Caution: With Auth mechanism set to PLAIN, the connector sends credentials in cleartext during authentication. In production, set SSL Mode to verify-ca or verify-full.

Document shape

Without a Mapping, the connector inserts each JSON object as published. It keeps nested objects and arrays, and stores JSON numbers as BSON doubles. An _id in the payload becomes the document ID. Otherwise, MongoDB adds one. The connector inserts each object in a JSON array as its own document.

With a Mapping, each document contains exactly the mapping keys. A missing payload key is stored as null.

In both cases, when the payload has a numeric timestamp in epoch milliseconds, the connector also writes an arrivedAt date, truncated to whole seconds.

The connector drops a payload that isn't a JSON object or array, or that decodes to an empty object, and logs mongodb message dropped with the reason.

Write modes

Write mode	Behavior
sync	The connector acknowledges a message only after it inserts the batch that holds the message. If the database is unreachable, the message stays in persistent storage, and the connector writes it after it reconnects. The connector drops a document that the server rejects, such as a duplicate _id, and writes the rest of the batch.
async	The connector acknowledges a message once it buffers it. If a batch fails to write, the connector drops it and counts the messages as dropped.

Caution: In async write mode, or with Persistent storage disabled, you can lose data during a database outage. Use async only when throughput matters more than delivery.

IT Hub

Use the IT Hub ProcessorIT Hub Processor to run an ad-hoc MongoDB command through the connector. The connector streams the documents from the result back one message per document, on the response topic supplied in the request.

Add a ConnectorAdd a Connector

Manage ConnectorsManage Connectors

IT Hub ProcessorIT Hub Processor