Catalog collector release notes

Databricks, Alteryx, ThoughtSpot, Sigma, Qlik Sense, and Azure collectors: Fixed an issue where OAuth reauthentication could loop infinitely when authentication failed, causing collection to hang. The collectors now properly handle expired credentials and other authentication failures without retrying indefinitely, improving collector reliability and stability.

Microsoft Fabric collector: Added support for AzureSqlTable dataset type, expanding metadata coverage for Azure SQL datasets in Microsoft Fabric.

Microsoft SQL Server collector: Improved login handling to prevent failed authentication attempts on remote publisher databases. The collector now connects only to local distributor databases, eliminating unnecessary authentication failures.

Databricks collector: Fixed an issue where the collector failed to harvest jobs belonging to a different workspace, ensuring complete job metadata collection across workspaces.

Microsoft Fabric collector: The collector is now generally available.

Marquez collector: Fixed an issue that caused duplicate lineage relationships to appear for the same source table, target table, and job combination, improving lineage accuracy.

Microsoft Fabric collector: Fixed an issue where Amazon RDS for SQL Server tables were skipped during collection, ensuring complete metadata capture.

Power BI collector: Fixed a crash caused by an invalid datasource mapping file. The collector now reports a clear error message instead, improving troubleshooting.

Tableau collector: Fixed an issue where schema-less tables were being skipped during collection. The collector now catalogs these tables, ensuring complete metadata coverage.

ServiceNow collector: The collector now harvests Platform Analytics Data Snapshot sources and indicators, expanding metadata coverage for ServiceNow analytics assets.

Talend collector: Added --context-group option to specify the Talend context group for resolving context variable references, improving metadata accuracy for dynamic workflows. Context variable replacement now uses actual title values from the package.

Microsoft Fabric collector: Fixed pipeline activity deserialization failures when typeProperties contain expression objects. Fixed missing catalog records for Fabric Lakehouse columns, ensuring complete asset representation.

ServiceNow collector: Fixed cataloging of View-implemented Data Interfaces via the models relationship, improving data interface visibility.

Databricks collector: Fixed an issue where volume lineage was dropped when the source path uses the dbfs: scheme prefix, ensuring complete lineage capture.

All collectors: Fixed an issue where collectors could hang indefinitely when external APIs stall. Failed requests now skip gracefully instead of causing collection failures, improving overall collector reliability.

Qlik Replicate collector: Fixed an issue where some properties and relationships were not appearing in the catalog, ensuring complete metadata capture.

Dremio collector: Fixed metadata inconsistencies for Virtual Datasets, improving catalog accuracy and asset linking.

Tableau collector: Fixed lineage capture for Dremio data sources, ensuring complete data flow visibility.

Tableau collector: Fixed incorrect classification of tables as schemas when custom SQL uses unqualified table references, improving metadata accuracy.

Microsoft SQL Server collector: Added warning message when the distributor server cannot reach the publisher database, improving troubleshooting for replication configurations.

Sensitive Data Classification: Added support for ServiceNow Vault Discovery, enabling classification of sensitive data stored in ServiceNow Vault fields.

Microsoft Fabric collector: The collector now harvests variable libraries, expanding metadata coverage for Fabric workspaces.

ADLS collector: Corrected descriptions for Azure Blob Storage and ADLS Gen2 harvesting options, improving configuration clarity.

Dremio collector: Fixed Object not found error when harvesting column statistics, ensuring complete metadata collection.

ServiceNow collector:

Databricks collector: Added option (--governance-metadata-collection) to harvest Unity Catalog governance metadata, enabling enhanced data governance visibility for Databricks environments.

Microsoft Fabric collector: Added support for Oracle table datasets, expanding metadata coverage for Fabric workspaces.

Power BI Service and Microsoft Fabric collectors: Fixed an issue where large formulas in measures and calculated columns were not being cataloged, ensuring complete metadata capture for complex calculations.

Azure Data Factory collector: Fixed an issue where certain pipeline variable structures caused collection failures, improving collector stability.

AWS Glue collector: Support added for harvesting metadata from multiple AWS regions in a single run, improving efficiency for multi-region environments. The collector now also harvests table parameters (advanced properties), expanding metadata coverage.

ServiceNow collector: The collector now harvests Vault classification metadata, providing enhanced data governance visibility.

BigQuery collector:

Databricks collector: Fixed Service Principal authentication failures, ensuring reliable authentication.

Dremio collector: Fixed an issue where out-of-scope schemas were being collected during API calls, improving collection accuracy.

Tableau collector:

Oracle collector:

Snowflake, Redshift, Databricks, Denodo, Dremio, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server, SAP HANA collectors: Fixed errors when encountering Infinity float values in numeric columns, preventing data type errors.

Microsoft Fabric collector: Fixed an issue where Direct Lake tables displayed null schema names when the source schema was not explicitly defined, improving metadata accuracy.

Hive Metastore collector: New collector added for Hive Metastore, enabling direct metadata harvesting from the Hive Metastore database (PostgreSQL, Oracle, MySQL/MariaDB, or SQL Server) without requiring HiveServer2. The collector harvests databases, schemas, tables, columns, Hive-specific table properties, and basic view lineage. This provides a lightweight alternative for environments where HiveServer2 is unavailable or when only structural metadata is needed.

This collector replaces the existing Hive and Hive Metastore collectors.

Snowflake, Redshift, Databricks, Denodo, Dremio, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, Microsoft SQL Server, and SAP HANA collectors: Added sample data preview capability, enabling the catalog to display sample of table data. When Sensitive Data Classification is enabled, sensitive columns are automatically masked in the preview, ensuring secure data visibility.

Snowflake, Redshift, Databricks, Denodo, Dremio, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server, SAP HANA collectors: Reduced unnecessary warning messages during SQL lineage processing, improving log clarity.

BigQuery collector: The collector now harvests primary and foreign keys, expanding metadata coverage for BigQuery table relationships.

All collectors: Fixed an issue where boolean false values in YAML configuration files were not properly recognized.

Snowflake, Redshift, Databricks, Denodo, Dremio, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server, SAP HANA collectors: Resolved memory issues when processing columns with very large string values, improving collector stability.

Tableau collector: Improved handling of recursive field queries, enhancing collection reliability for complex Tableau workbooks.

Go here to access release notes for previous version.