Catalog Toolkit release notes

• Amazon Lake Formation collector: Added support for the Amazon Lake Formation collector. Follow these steps to start using the new collector configurations through CTK.

• Alteryx collector: Updated the build query to include comments in the Workflow Details Markdown table, providing additional context directly alongside workflow metadata.

• Microsoft Fabric collector: Added additional metadata properties for both Eventhouse and KQL Database types, further enriching the metadata harvested from Microsoft Fabric environments. Be sure to update your catalog organization's Metadata Profile to incorporate these changes.

• Catalog configuration improvements: The Metadata profile collection types are now instantiable.

• New default collection types: New standard collection types Domain, Glossary Collection, and Source Collection have been added to Catalog Configuration. To start using them, select Allow UI creation for these collection types and update your catalog organization's Metadata Profile to incorporate these changes.

• Azure data factory collector MDP module: Added support for parameterized linked services and their relationships to data sources. Be sure to update your catalog organization's Metadata Profile to incorporate these changes.

• Alteryx collector MDP module: Workflow markdown table updated to sort rows by version number.

• Databricks and Snowflake collectors: Markdown tables displaying Databricks Tags and Snowflake Tags are now sorted alphabetically by key name, making it easier to scan and locate specific tags within your metadata profiles.

• To prepare for an upcoming platform change to how resource types are configured to allow for UI creation, we have introduced new configuration capabilities for Collection Types, allowing you to:

  • Specify whether a Collection Type can be created via the UI (Allow UI creation)

  • Define Parent collection types to better structure your metadata hierarchy

    

• As part of this release, we have run a one-time script in each CTK stack to ensure a smooth transition:

  • All child types of Business Term (i.e., Glossary Types) and all Collection Types have been set to Allow UI creation if this setting had not already been configured.

  • This makes those types instantiable in the UI, although no behavior changes will occur immediately.

  This proactive step ensures your existing types remain usable once the upcoming enforcement takes effect.

  Important

  Be sure to update your catalog organization's Metadata Profile to incorporate these changes.

• Microsoft Fabric collector: Added support for the Microsoft Fabric collector. Follow these steps to start using the new collector configurations through CTK.

• MDP Workbook Template: A new and improved version of the template is now available for use.

New features and changes

• dbt Core and dbt Cloud collectors: Added dbt Source relationships, supporting updates to the way the dbt collector models these relationships. Be sure to update your catalog organization's Metadata Profile to incorporate these changes.

• SAP HANA collector: Added support for the SAP HANA collector. Follow these steps to start using the new collector configurations through CTK.

• Snowflake collector: Updated display of Snowflake Tags returned by the Snowflake collector to a single field with a markdown table. This provides a more compact and readable presentation. Be sure to update your catalog organization's Metadata Profile to incorporate these changes.

• Snowflake and Qlik Sense Cloud collectors: Resolved potential duplication of Owner field when MDP Modules for both collectors are activated. Be sure to update your catalog organization's Metadata Profile to incorporate these changes.

• Marquez collector: Added support for the Marquez collector. Follow these steps to start using the new collector configurations through CTK.

• Databricks collector: Improved the display of Databricks Tags by consolidating them into a single field with a markdown table, offering a more compact and readable presentation. Be sure to update your catalog organization's Metadata Profile to incorporate these changes.

• AWS Glue collector: Added support for the updated AWS Glue collector. Follow these steps to start using the new collector configurations through catalog Toolkit.

• Tableau collector:

  • Changed Creator fields to Owner for a more accurate reflection of Tableau roles. Be sure to update your catalog organization's Metadata Profile to incorporate these changes.

  • Added a SQL Query field for Custom SQL Table to the Explorer Sidebar. Be sure to update your catalog organization's Metadata Profile to incorporate these changes.

• Relationships handling: The relationship build query has been modified to better manage relationships with multiple source and/or target types. This update enables a more consistent display of filters and relationships as fields. Be sure to update your catalog organization's Metadata Profile to incorporate these changes.

• Tableau collector: Added Custom SQL Table to Workbook relationship. Be sure to update your catalog organization's Metadata Profile to incorporate these changes.

• Azure Data Factory collector: Updated Activity to Dataset relationship and added Dataset Parameter field to ADF Activity. Be sure to update your catalog organization's Metadata Profile to incorporate these changes.

• Monte Carlo collector: Removed the Reaction Type field from Incidents, as it is deprecated by Monte Carlo. Be sure to update your catalog organization's Metadata Profile to incorporate these changes.

• Microsoft SSIS collector: Fixed the Microsoft SSIS MDP Module to correct the missing relationship between SSIS Parameter and SSIS Environment Variable. Be sure to update your catalog organization's Metadata Profile to incorporate these changes.

• Snowflake collector: Refined the Snowflake Tags and Policies build query to reduce duplication and enhance efficiency

• Alteryx collector: Made minor updates to the build query, including reformatting Workflow Node names for improved clarity and consistency.

• Qlik Talend Data Integration collector: Renamed the module for the collector from Qlik Talend Data Integration to Qlik Talend DI for conciseness.

• Catalog configuration improvements: Updated the MDP generation query to correctly handle scenarios where a type's parent is deleted and no longer relevant, ensuring smooth configuration and data integrity.

• Apache Airflow collector: Added support for the Apache Airflow collector. Follow these steps to start using the new collector configurations through CTK.

• Qlik Talend Data Integration collector: Added support for the Qlik Talend Data Integration collector. Follow these steps to start using the new collector configurations through CTK.

• Power BI Service and Power BI Gov collectors: Added support for the new Entered Data Table flag and the Power BI Calculated Table resource type. Be sure to update your catalog organization's Metadata Profile to incorporate these changes.

• Databricks collector: Introduced support for the new Databricks Query resource type and the Notebook Content field. Be sure to update your catalog organization's Metadata Profile to incorporate these changes.

• DB2, Denodo, Hive, Hive Metastore, MS SQL Server, MySQL, Netezza, Oracle, PostgreSQL, Redshift, Snowflake, Teradata collectors: Enhanced the MDP modules to provide appropriate support for Database Functions and Database Stored Procedures. Be sure to update your catalog organization's Metadata Profile to incorporate these changes.

• Support for Account Fields: Added the ability to configure Account fields through CTK.

• Tableau collector: Added support for the new Tableau collector. Follow these steps to start using the new collector configurations through CTK. Please coordinate with the data.world support team to transition from the legacy Tableau collector.

• Alteryx, SQL Server, and Thoughtspot collectors:  Made minor updates to field/section mappings. Be sure to update your catalog organization's Metadata Profile to incorporate these changes.

• Power BI Service and Power BI Gov collectors: Updated the Expression fields to be multiline. Be sure to update your catalog organization's Metadata Profile to incorporate these changes.

• QlikSense Cloud collector: Refined the build SPARQL query by updating prefixes to align with standard conventions, ensuring greater consistency and reliability.

• Go here to access release notes for previous version.