Snowflake Integration Guide
Review the following guide for setting up an integration between Litmus Edge and the Snowflake data warehouse.
Before You Begin
You will need a Snowflake account with the SysAdmin privilege.
Step 1: Add Device
Follow the steps to Connect a DeviceConnect a Device. The device will be used to store tags that will be eventually used to create outbound topics in the connector. Make sure to select the Enable Data Store checkbox.
Step 2: Add Tags
After connecting the device in Litmus Edge, you can Add TagsAdd Tags to the device. Create tags that you want to use to create outbound topics for the connector.
Step 3: Create Snowflake Database
You will need to create a database in Snowflake that will be used to receive the data from Litmus Edge.
To create the Snowflake database:
- Log in to Snowflake.
Navigate to Data > Databases.
Click + Database. The New Database dialog displays.
Enter a name for the database. Optionally, add a comment.
- Click Create. The new database is created.
Click the new database name, and then click + Schema. The New Schema dialog displays.
Enter a name for the schema. Optionally, add a comment.
New Schema dialog- If needed, select the checkbox for Managed access.
- Click Create. The new schema is created.
You can now set up the connector in Litmus Edge to send data to this new database and schema.
Step 4: Add the DB - Snowflake Connector
Follow the steps to Add a Connector and select the DB - Snowflake provider.
Configure the following parameters.
- Name: Enter a name to identify the connector.
- Account: Enter the Snowflake account ID. Refer to your Snowflake account info in the bottom-left corner of the Snowflake account page. See Account Identifiers to learn more.
- Region: Enter the Snowflake cloud region ID. Refer to your Snowflake account info in the bottom-left corner of the Snowflake account page. See Supported Cloud Regions to learn more.
- Username: Enter your Snowflake username.
- Password: Enter your Snowflake password.
- Database: Enter the database name you configured in Step 3.
- Schema: Enter the schema name you configured in Step 3.
- Warehouse: Enter the warehouse name that will be used for the integration. Navigate to Admin > Warehouse for your list of current warehouses. See Virtual Warehouses to learn more.
- Table: Enter a table name where the data will be displayed.
- Create table: Select this checkbox if the table name configured doesn't exist yet. A new table will be created.
- Commit timeout: Enter the transaction commit timeout in (ms).
- Max transaction size: Enter the maximum number of messages before a transaction is committed, regardless of timeout parameter.
- Bulk insert count: To buffer messages, enter a value. Bulk insert will be performed once this value is reached.
- Throttling limit: The maximum number of messages per second to be processed. The default value is zero, which means that there is no limit.
- Persistent storage: When enabled, this will cause messages to undergo a store-and-forward procedure. Messages will be stored within Litmus Edge when cloud providers are online.
- Queue Mode: Select the queue mode as lifo (last in first out) or fifo (first in first out). Selecting lifo means that the last data entry is processed first, and selecting fifo means the first data entry is processed first.
Step 5: Enable the Connector
After adding the connector, click the toggle in the connector tile to enable it.
If you see a Failed status, you can review the Connector Logs and relevant error messages.
Step 6: Create Outbound Topics for Connector
You will now need to import the tags you added in Step 2 to the connector as topics.
To create outbound topics:
- Click the connector tile. The connector Dashboard appears.
- Click the Topics tab.
- Click the Import from DeviceHub tags icon. The DeviceHub Import dialog box appears.
Step 7: Enable Topics
Ensure the topics you imported are enabled by returning to the Topics tab and clicking the Enable all topics icon.
Step 8: Verify Connection in Snowflake Database
Access the Snowflake database you created to verify that data is successfully being sent.