Catalog Toolkit release notes

• Grafana collector: Updated the Grafana Panes to Grafana Dashboard relationship to use dct:isPartOf . Follow these steps to start using the new collector configurations through CTK.

• Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server collectors: Updated Stored Procedure and Function to Schema relationship to use dct:isPartOf. Ensure you update your catalog organization's Metadata Profile to incorporate these changes.

• ADF and ADLS Collectors: Removed support for deprecated versions of ADF and ADLS collectors. Please update your collectors to a version newer than Release 2.202 for these collectors if you have not already done so. Also, follow these steps to start using the latest collector configurations through CTK.

New features and changes

• Google BigQuery collector: Added Nullable and Column Size metadata fields to the BigQuery MDP module. Ensure you update your catalog organization's Metadata Profile to incorporate these changes.

• Snowflake, Microsoft SQL Server, MySQL, PostgreSQL, Redshift, DB2, Databricks, Tteradata, and Oracle collectors: Updated metadata fields for string sampling metadata for data profiling. Ensure you update your catalog organization's Metadata Profile to incorporate these changes.

• MS SQL, Netezza, Oracle, Snowflake, Teradata collectors: Added fields for extended metadata. Ensure you update your catalog organization's Metadata Profile to incorporate these changes.

• Search cards: Added support for adding custom metadata fields to search cards. Ensure you update your catalog organization's Metadata Profile to incorporate these changes.

• QlikSense Cloud collector: Introduced MDP Module for the the new QlikSense Cloud collector. Follow these steps to start using the new collector configurations through CTK.

• Informatica CDI collector: Introduced MDP Module for the Informatica collector. Follow these steps to start using the new collector configurations through CTK.

• Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server collectors: Added a Decimal Digits field for database columns.

• Added SPARQL queries to export metadata fields and relationship definitions from MDP Modules.

• Fixed SQL queries for Top 15 Most Viewed Resources, Resource Views, Edits, and Creates by Week, and User Engagements in Platform Analysis.

• Power BI and Power BI Gov collectors: Added new relationships from app report to workspace report. Ensure you update your catalog organization's Metadata Profile to incorporate these changes.

• All collectors: The Collected Via metadata field is removed, as this information is now displayed in the platform managed Source System field. Ensure you update your catalog organization's Metadata Profile to incorporate these changes.

• Oracle collector: Fixed an error in Oracle build query that was causing sync errors.

• MySQL collector: Added MDP Module for the MySQL collector. Follow these steps to start using the new collector configurations through CTK.

• Salesforce collector: Released a major update to the Salesforce MDP Module to support the enhanced Salesforce collector, which now harvests Salesforce objects. Ensure you update your catalog organization's Metadata Profile to incorporate these changes.

• Added functionality to configure the Source System field as read-only. This update is a precursor to the upcoming release of the platform-managed Source System field.

• Oracle collector: Support added for Package Specification and Package Body Definition fields.

• Azure Data Lake Storage Gen2 collector: Replace redundant permission-related relationships with a single hasPermission relationship.

• ThoughtSpot collector: Removed fields from the MDP that were not in use.

• PowerBI collector: Added support for dataset table expressions and descriptions for workspaces, apps, and datasets.

• Databricks collector: Added suport for Databricks external locations.

• Browse card: Fixed SHACL validation errors when configuring clickable section headers.

• Power BI collector: Fixed the relationship configuration between Power BI Measure to Power BI Table.

• All collectors: Fixed an error that caused source-modules.ttl file to fail to sync in certain situations.

• Added support for custom Source System select options. This update is a precursor to the upcoming release of the platform-managed Source System field.

• Amazon QuickSight collector: Corrected an issue where some metadata fields in the QuickSight MDP Module were not properly populated.

• Power BI collector: Addressed a markdown rendering issue in the Power BI connection details field.

• data.world platform metrics: Fixed an issue where the data.world Metrics dataset was not correctly linked to the data.world Platform Analysis project in some instances, causing sync errors.

• Support added for SQL Server Integration Services (SSIS) collector.

• Azure data factory collector: Support added to display parameters even when the value is empty.

• Power BI and Power BI collectors: Support added for Power BI Measures and Report Type.

• Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server collectors: The collectors now support Relationships between Database View and Database Column added for Filtered By, Aggregated By, and Ordered By relationships.

• Catalog Configuration Update:

  • Added an Other Modules select field to Catalog Configuration for configuring non-source-specific MDP Modules.

  • Moved the Column Statistics and Sensitive Data Discovery options to the new Other Modules field.

• Browse card configuration:

  • Resolved an issue where multiple items sharing the same title caused display errors and issues with the SHACL validation report.

  • Addressed a problem where spaces in file names used for the Browse Card caused the build query to timeout.

• Power BI collector:

  • The collector now supports Measures and Report Type.

  • The connection details are now visible on the Power BI Data Sources page.

• Collections: The Bulk Collection Import feature has been updated to allow the addition of descriptions and statuses to existing collections.

• Power BI Report Server collector: Errors observed while loading collector output to the intended organization are now fixed.

• Oracle collector now supports Materialized Views.

• Snowflake collector now supports additional metadata and Materialized Views.

• BigQuery collector: The BigQuery labels are now shown correctly on the tagged resources.

• All collectors: The Catalog Configuration ddw-catalog is now automatically synchronized following the deployment of CTK updates. Updates of the Source MDP module will appear in the catalog configuration without the need to manually sync the source-modules.ttl file. However, to apply updates to Metadata profile files will still require manual synchronization in the Catalog organizations.

• Tableau collector: Created and Modified timestamps added to Tableau Dashboards, Tableau Workbooks, and Tableau Views.

• All collectors: Failures observed while syncing source metadata files are now fixed. This issue was observed when there were multiple DWCC output files in a source dataset.

• Resource types: If a resource type with associated metadata fields is deleted and then recreated with the same name, the previously linked metadata fields were displayed on the newly created type.