Skip to main content

Preparing to run the Sigma collector

Setting up pre-requisites for running the collector

Make sure that the machine from where you are running the collector meets the following hardware and software requirements.

Table 1.

Item

Requirement

Hardware (for on-premise runs only)

Note: The following specs are based upon running one collector process at a time. Please adjust the hardware if you are running multiple collectors at the same time.

RAM

8 GB

CPU

2 Ghz processor

Software (for on-premise runs only)

Docker

Click here to get Docker.

data.world specific objects (for both cloud and on-premise runs)

Dataset

You must have a ddw-catalogs dataset set up to hold your catalog files when you are done running the collector.

If you are using Catalog Toolkit , follow these instructions to prepare the datasets for collectors.

Network connection

Allowlist IPs and domains

Follow these instructions to configure your network.



Setting up Sigma

Setting up API token for authentication

You will need to set up a user account and an API token in Sigma. This information is used by the collector to authenticate to Sigma.

Important

For detailed instructions, see the Sigma Docs.

  1. Open the Sigma Admin Portal. Click on Administration and then APIs & Embed Secrets.

  2. Create a new API token

  3. Save the API token and the associated Client ID . You will use this information while setting up the collector parameters for authentication.

Setting up workbook permissions

The user account to be used for running the collector should have a minimum Can View permission on the workbook being cataloged.

Important

For detailed instructions, see the Sigma Docs.

To grant view permission:

  1. Go to the folder containing the workbook. 

  2. Click on the three dots next to the workbook you want to share. Select Share.

  3. Select team or user to grant permission.

  4. Select the Can View permission. Click Share.

Setting up connection permissions

To be able to collect lineage, the user account should have a minimum Can Use permission on the connection. The permission can be granted at database, schema, or table level.

Important

For detailed instuctions, see the Sigma docs.

To grant permission to the database:

  1. Open the connection from the left side panel.

  2. Open the Permissions tab for the database. Click Add Permission Grant.

  3. Select team or user to grant permission.

  4. Select the Can Use permission. Click Save

In this section: