Docs portal

Connection manager with DATASOURCE NAME

Using the Connection Manager
  1. Browse to the profile page of the organization for which you want to set the connection manager. You can do this from Left sidebar > Organizations or by clicking the organization name from anywhere in the application.

  2. On the Settings tab go to the Connection manager section.

  3. From here you can create new connections, edit and delete collections and manage tasks for the connection.

org_profile_settings_connection_manager_taskview.png
Create a connection

To create a new connection to your data sources, click the Add connection button.

org_profile_settings_connection_manager_newbutton.png

The dialog opened lists the database sources for which you can create an organization level connection:

org-level_connections.png

Select a source to be presented with the configuration screen for your connection.

Edit or delete existing connections

The Three dot menu to the right of a connection name gives options to Create task, Edit connection, and Delete connection.

  • Create task

  • Edit connection links to the same configuration page used to create the connection, and you need to have the same security information (login, password, etc.) to change anything about the connection.

  • Delete connection prompts you to be sure you want to proceed, but does not require any further validation.

Note

Any organization admin or other authorized person with the credentials for a connection can modify it, and any data steward or other authorized person with access to the Connection manager can delete connections.

org_profile_settings_connection_manager_manage_connections.png
Tasks

A metadata task catalogs the metadata for a data source and places the extracted information into a specified Collection. For now, the system only supports creating metadata tasks. Use the Create task option available in the Three dot menu to the right of a connection to create a new task for the configured connection.

Note

Before creating a task you need to have a collection in which to put your task output. The collection must be created before the task.

Caution

Tasks work with version 2.x of DWCC. If you are still using DWCC v1.x, see the article on DWCC v2.0 for more information.

org_profile_settings_connection_manager_manage_create_task.png

The Create task button opens a dialog where you can configure your new task. To create a task you will be prompted to choose a database and schema as appropriate. You also need to add it to a collection.

connections_new_task.png

The Three dot menu to the right of each task gives options to sync, edit, or delete the tasks.

  • Sync runs the task again, pulling in any new metadata.

  • Edit task details opens the Edit task window where you can change any of the original configuration (database, schema, or collection),

  • Delete task removes the task.

connections_manage_task.png
Create a connection

To create a new connection to your data sources, click the Add connection button.

org_profile_settings_connection_manager_newbutton.png

The dialog opened lists the database sources for which you can create an organization level connection:

org-level_connections.png

Select a source to be presented with the configuration screen for your connection.

Database configurations

Select your database from the following list to get specific configuration for it:

Edit or delete existing connections

The Three dot menu to the right of a connection name gives options to Create task, Edit connection, and Delete connection.

  • Create task

  • Edit connection links to the same configuration page used to create the connection, and you need to have the same security information (login, password, etc.) to change anything about the connection.

  • Delete connection prompts you to be sure you want to proceed, but does not require any further validation.

Note

Any organization admin or other authorized person with the credentials for a connection can modify it, and any data steward or other authorized person with access to the Connection manager can delete connections.

org_profile_settings_connection_manager_manage_connections.png
Tasks

A metadata task catalogs the metadata for a data source and places the extracted information into a specified Collection. For now, the system only supports creating metadata tasks. Use the Create task option available in the Three dot menu to the right of a connection to create a new task for the configured connection.

Note

Before creating a task you need to have a collection in which to put your task output. The collection must be created before the task.

Caution

Tasks work with version 2.x of DWCC. If you are still using DWCC v1.x, see the article on DWCC v2.0 for more information.

org_profile_settings_connection_manager_manage_create_task.png

The Create task button opens a dialog where you can configure your new task. To create a task you will be prompted to choose a database and schema as appropriate. You also need to add it to a collection.

connections_new_task.png

The Three dot menu to the right of each task gives options to sync, edit, or delete the tasks.

  • Sync runs the task again, pulling in any new metadata.

  • Edit task details opens the Edit task window where you can change any of the original configuration (database, schema, or collection),

  • Delete task removes the task.

connections_manage_task.png
Currently supported data sources

We support working with many data sources, and we continue to add new sources all the time. Below is a list of the data sources we connect to, and the options for connecting to them. The links for each go to the configuration instructions in Connection Manager or the metadata collector:

Table 4. Supported data sources

data source

Extract data from data source

Collect metadata from data source

Create a live connection to data source

Apache Spark

not yet

yes

not yet

Athena

yes

yes

yes

AWS Glue

not yet

yes

not yet

Azure Synapse

yes

yes*

yes

BigQuery

yes

yes

yes

Databricks

not yet

yes

not yet

Datakin

not yet

yes

not yet

DB2

not yet

yes

not yet

dbt

not yet

yes

not yet

Denodo

yes

yes

yes

Domo

not yet

yes

not yet

Dremio

not yet

yes

not yet

Generic JDBC

not yet

yes

not yet

Hive

not yet

yes

not yet

Hive metastore

not yet

yes

not yet

Infor Ion

yes

yes

yes

Looker

not yet

yes

not yet

Manta

not yet

yes

not yet

Marquez

not yet

yes

not yet

Monte Carlo

not yet

yes

not yet

My SQL

yes

yes

not yet

Open API

not yet

yes

not yet

Oracle

yes

yes

not yet

PostgreSQL

yes

yes

yes

PowerBI

not yet

yes

not yet

Presto

not yet

yes

not yet

Redshift

yes

yes

yes

Reltio

not yet

yes

not yet

Salesforce

not yet

yes

not yet

Snowflake

yes

yes

yes

SQL Anywhere

not yet

yes

not yet

SQL Server

yes

yes

yes

Tableau Server

not yet

yes

not yet

Vertica

not yet

yes

not yet



*Collecting metadata from Azure Synapse is currently only available through the Collection Manager.