Release notes for previous versions

Databricks collector: Now harvests models registered in Unity Catalog, expanding coverage of Databricks assets.

Confluent Cloud and Confluent Platform collectors: Added support for TLS connections to Confluent Kafka, improving security and compatibility.

Microsoft Fabric collector: Fixed issues with missing DirectLake column-level lineage and corrected some improperly formed source IRIs for DirectQuery lineage.

Monte Carlo collector: Updated enum types from Monte Carlo GraphQL definitions to fix an exception caused by unrecognized types.

Tableau collector: Added a new configuration option --tableau-convert-database-identifiers. When enabled, this converts the case of database identifiers (such as schema and table names) to the default collation of the associated database, improving consistency.

SQL Server Integration Services (SSIS) collector: Enhanced debug-level logging to support more effective troubleshooting and analysis.

Microsoft Fabric collector: The collector now harvests lineage from report pages to the columns and measures in the semantic model used by each page. Additionally, an "is hidden” flag is captured for report pages, providing clearer visibility into page-level metadata.

Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server, SAP HANA collectors: Fixed an issue to ensure harvesting continues from multiple databases even if the connection to one database fails.

Marquez collector: Added support for harvesting Marquez datasets associated with Databricks, extending lineage and metadata coverage.

SQL Server collector: Added support for SQL Server replications, improving visibility into replicated database environments.

Tableau collector: Introduced validation for user-configured site identifiers and added warnings when identifiers are invalid or inaccessible.

Microsoft Fabric collector:

Sigma collector: Now supports lineage from datasets to source tables or other datasets, improving traceability of dataset dependencies.

Databricks collector: Enhanced lineage harvesting to support SQL statements containing the struct function.

dbt core and dbt clould collectors: Added support for dbt projects targeting SQL Server databases using encryption, improving compatibility in secure environments.

Power BI collector: Added support for harvesting report images embedded in a zip file, ensuring complete metadata capture from reports.

Databricks collector:

Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server, SAP HANA collectors: Resolved parsing errors that occurred when harvesting view lineage from views whose SQL contained column comments with parentheses.

SQL Server Reporting Services (SSRS) collector: Corrected incorrect detection of when to use SOAP vs REST API, ensuring proper connectivity for older and newer SSRS versions.

OpenAPI collector: Fixed an issue with duplicate identification of API resources, ensuring unique resource cataloging.

Tableau collector: Harvests the last published date for workbooks and data sources, providing greater visibility into update history.

Alteryx collector: Catalogs additional metadata, including the caption tag in ToolContainer, the query in LockInInput, and the SQL in DbFileInput.

QlikSense collector: Added a configuration option to include or exclude applications, giving users more control over the scope of harvested metadata.

Athena collector: Fixed an issue where the collector stopped after 100 tables. It now correctly harvests more than 100 tables in a database.

Sigma collector: Enhanced the workbook filter to avoid missing workbook exceptions, improving reliability during harvesting.

MySQL collector: Fixed an error in fetching statistics for columns whose names are reserved SQL keywords.

Microsoft Fabric collector: Resolved issues in stored procedure harvesting by properly resolving names when using pipeline variables and parameters, and updated relationship types to represent dependencies more accurately.

Power BI collector: Fixed an issue to ensure that when a database name is provided in the datasources.yaml file, it is always used and not overridden by values retrieved from a database query.

Tableau collector: Fixed an issue so that Published Datasources are only cataloged when they are being used in a Project that is in scope, preventing unnecessary or irrelevant catalog entries.

Informatica CDI collector: Resolved an unexpected exception that was causing the collector to fail, improving stability and reliability.

SQL Server collector: Added support for harvesting agent jobs.

Sigma collector: Added configuration options to include or exclude workspaces, providing greater control over which resources are harvested.

Redshift collector: Now supports harvesting external tables defined via AWS Glue.

Microsoft Fabric collector:

Microsoft Fabric and Power BI collectors: Now catalog refresh schedules for resources where refresh configuration is available, helping track automated data updates.

AWS Glue collector: Now identifies partitioned columns separately from other columns.

Marquez collector: Fixed a null pointer exception that could occur when a job lacked a latest run.

Microsoft Fabric collector:

Marquez collector: Now skips unsupported dataset types, preventing errors during harvesting.

A new collector, the OpenAPI collector, is now available in public preview. It supports harvesting metadata from APIs described using OpenAPI v3.0, enabling documentation and cataloging of API assets.

Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server, SAP HANA collectors: Added a Sensitive Data Classification option to allow classification using a hosted private-ai instance.

Microsoft Fabric collector:

Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server, SAP HANA collectors: Now include column statistics support for Date, Timestamp, and Boolean data types, enhancing profiling depth across supported databases.

Postgres collector: Supports AWS IAM authentication via secret and access key parameters, offering more secure and flexible credential management.

Oracle collector: Fixed an issue in the table index feature that previously caused permission errors or max open cursor issues by updating the query logic to use DBA_ views when available.

SSIS Collector: Now harvests deeply nested control flow executables, ensuring complete control flow visibility.

Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server SAP HANA collectors: Improved sampling behavior for environments where TABLESAMPLE is unsupported by falling back to LIMIT or TOP clauses to compute statistics more reliably.

A new collector, the AWS Lake Formation collector, is now available in public preview.

Microsoft Fabric collector: Now catalogs metadata for Eventhouses, including the addition of associated KQL databases, expanding visibility into the Microsoft Fabric ecosystem.

dbt core and cloud collectors: Added support for harvesting semantic model metadata, enriching the data model layer within your catalog.

Power BI collector: Introduced support for an additional Databricks source MQuery function type in lineage resolution, improving coverage and accuracy of Power BI lineage.

SSIS collector: Enhanced debug-level logging to support better root cause analysis for missing catalog resources, aiding in troubleshooting and diagnostics.

Alteryx collector: Fixed an issue where the workflow description was not correctly captured from the user-provided meta info section.

AWS Glue collector: Resolved a null pointer exception that could occur when the Glue Data Catalog tables are empty, improving stability.

Snowflake collector: Improved data type standardization by stripping parenthesized size/length values (for example, VARCHAR(255) → VARCHAR) for cleaner and more consistent metadata.

Tableau collector: Added view-based filtering for bin and group fields to align with how calculated fields are handled.

Salesforce collector: Now harvests all reports and dashboards, not just recently viewed ones.

Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server collectors, SAP HANA collectors: Column decimal digits are now written only for appropriately typed columns.

Tableau collector: Fixed a null pointer exception in column lineage processing for Custom SQL tables.

OpenAPI collector: Resolved errors caused by malformed spec files that previously triggered null pointer exceptions.

Logging framework (all collectors): Introduced minor updates to the logging framework. As a result, users may notice slightly different log messages compared to previous versions.

Snowflake collector: Upgraded the embedded Snowflake JDBC driver to version 3.34.2, addressing potential exceptions and improving stability.

Azure data factory collector: Now catalogs parameters used in parameterized linked services, along with the relationship between each linked service and its data source, providing deeper lineage visibility.

PowerBI collector: Added support for parsing lineage from certain SQL statement types without requiring database credentials, making it easier to extract lineage in more restricted environments.

Tableau collector: Prevented exceptions that could occur when harvesting table-view relationships, particularly when table information is missing from the Tableau GraphQL API.

PowerBI collector: Fixed an issue with Denodo sources that use custom SQL, improving support for a wider range of PowerBI source types.

SQL Server Integration Services (SSIS) collector: Resolved an exception that occurred during the harvesting of column information, enhancing reliability in metadata extraction.

Tableau collector: Now determines the associated project for a Custom SQL Table based on its workbook rather than its datasource, improving accuracy in project assignments.

PowerBI collector: Added support for Oracle Autonomous Database as a source, expanding connectivity and metadata coverage within PowerBI environments.

All collectors (on-premise version): Updated the logging behavior so that each new collector run will truncate existing log files written to the local filesystem, rather than appending to them. This means logs from previous runs will be overwritten during subsequent runs.

This change does not affect uploaded log files—those remain intact.

SSRS collector: Added support for harvesting metadata using the SOAP API, enabling compatibility with older SSRS versions that do not support the REST API.

Microsoft Fabric collector: Added support for harvesting lineage from custom SQL queries used in Fabric Lakehouse and Warehouse sources

Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server collectors, SAP HANA collectors: Resolved an issue that caused exceptions when harvesting index metadata for tables lacking a defined type.

SQL Server collector: Fixed an error in how table names were qualified when calling the sp_spaceused stored procedure—ensuring accurate harvesting of table size metadata.

QlikSense collector: Prevented exceptions that occurred when the created-by or modified-by user information was missing from collected resources.

Databricks collector: Expanded lineage harvesting to include relationships between objects across different schemas, rather than only within the same schema.

Tableau collector: The collector Now harvests the last refresh date for Tableau data sources, where available—providing better visibility into data currency and freshness.

Tableau collector: Fixed an issue that was preventing the harvesting of certain lineage relationships between Column Fields and their underlying database columns.

SSIS collector: Fixed an issue that caused a stack overflow error when cycles were present in execution flows.

Tableau collector:

Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server collectors, SAP HANA collectors: Corrected the calculation of sampling percentages for column statistics, which were previously computed incorrectly in some cases.

Postgres and Redshift collectors: Added support for the ~~* symbol in view definition SQL, which serves as an alias for the LIKE keyword in SQL syntax.

SSIS Collector: Improved logging and resolved a NullPointerException caused by missing elements in execution XML files.

PowerBI collector: Improved performance by handling PowerBI API retry instructions more efficiently.

Tableau collector: Added enhanced logging to monitor collector progress and track cataloged resources.

Athena collector:

ADF collector: Fixed an issue where unparseable JSON in certain parameter values caused collector failures.

BigQuery collector: Correctly handles cases where the BigQuery API does not return table details, preventing errors.

PowerBI collector: Supports automated migration of resource IRIs (identifiers) from earlier collector versions to maintain continuity.

QlikSense collector: Implemented a rate limiter to improve performance and prevent API errors during metadata harvesting.

AWS S3 collector: Enhanced performance by applying regex filters before fetching objects within buckets, reducing unnecessary API calls.

Tableau collector: Fixed a gap in harvesting of lineage relationships between Tableau column fields and database columns.

Microsoft Fabric collector: Resolved an issue where some Data Pipelines failed to deserialize due to varying data types based on configuration.

SSIS collector: Fixed an exception caused by missing or empty elements in SSIS executable descriptor XML files.

Athena collector: Updated harvesting of databases to accommodate environments with more than 100 databases.

Microsoft Fabric collector:

Microsoft Fabric collector:

dbt Core and dbt Cloud collectors: dbt sources are now associated with database schemas (instead of tables) through a non-lineage relationship, improving metadata accuracy.

Tableau collector: Enhanced log tracing around column field lineage to improve troubleshooting and debugging clarity.

Databricks collector: Now captures table lineage to corresponding AWS S3 objects.

Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, SQL Server collectors: Added support for harvesting metadata about table indexes, providing deeper insight into database structure and performance optimization.

Tableau collector: Improved handling of missing connection types on Tableau Server to prevent ingestion errors and ensure smoother metadata extraction.

dbt collector: Transitioned from using the manifest to the catalog to retrieve database identifiers. This resolves issues with quoted identifiers (for example, Snowflake).

Athena, Amazon Database Migration Service (DMS), Amazon DynamoDB, AWS Glue, Amazon QuickSight, Amazon S3 collectors: The collectors now support multiple authentication methods, offering greater flexibility and compatibility. The following new parameters are introduced for the new authentication methods:

Amazon Database Migration Service (DMS) collector: Introduced support for S3 as both a source and target endpoint.

Tableau collector: Implemented a nonNull filter for relatedFields, addressing errors and improving stability in the Tableau collector.

Databricks collector: Fixed an issue where table lineage could not be resolved when information was incomplete.

Tableau collector: Corrected processing of column fields to resolve subtle errors in field-to-database lineage.

Azure data factory collector: Improved handling of missing table details when collecting lineage.

All collectors: Fixed logging issues where some messages were not being written to the log file as expected.

Monte Carlo collector: Fixed alignment issues in generated table information from Monte Carlo Monitor by properly escaping pipes ('|') in monitor names.

A new collector, the SAP HANA collector, is now available in public preview.

SSIS collector: Now captures the connections between packages through the control flow, improving visibility of package relationships.

Databricks collector: The collector now harvests lineage only within the current schema. A new option Harvest entire lineage (--harvest-entire-lineage) is added to enable harvesting lineage from external schemas.

Microsoft Fabric collector: Enhanced to support additional syntaxes for Semantic Model table connections to Fabric warehouse/lakehouse resources, broadening compatibility and connectivity.

QlikSense collector: Made performance improvements to increase the efficiency and speed of the collector.

Microsoft Fabric collector: Fixed issues with Lakehouses and SQL endpoints to ensure database resources are associated with the correct resource, enhancing accuracy.

Tableau collector: Fixed an issue to ensure proper handling of sites and projects with large quantities of column and data source fields.

Databricks collector: Resolved a parsing issue in view queries where aliases starting with a number caused failures.

The following two new collectors are now available in public preview.

Power BI collector: The collector now harvests column descriptions for Power BI columns.

Databricks collector: The collector now supports Oauth service principal authentication for Databricks. Two new parameters, Service principal client ID (--client-id) and Service principal client secret (--client-secret) are introduced for this.

Tableau collector: Fixed an issue with the display of lineage between Tableau Fields and Database Columns, ensuring accurate representation of data relationships.

Snowflake Collector: The collector now harvests system tags.

SSIS collector: The collector now supports the inclusion or exclusion of specific databases or servers from being harvested. For new parameters are introduced to use these features: --include-database, --exclude-database, --include-server, --exclude-server.

All collectors: Resolved an issue where collectors created new collections with a new ID, leading to duplicate collections in the catalog.

Qlik Sense collector: Fixed an issue where missing user information in Qlik Sense resulted in an exception trace in the logfile.

Azure data factory collector: Added a log message to indicate when the Dataset API response lacks sufficient information, such as schema and table details, to construct lineage.

Tableau collector: The way we represent ownership information for workbooks, views, and metrics in the Tableau catalog has been updated. The owner is now represented using the kos:hasOwner property.

Alteryx collector: Increased API call read timeout and improved error handling by capturing and logging processing exceptions.

Confluent collectors: The collector now correctly handles cases where consumer member assignments are missing a topic description.

Tableau collector: The collector now accurately harvests hidden dashboards. Previously, the feature was limited to hidden Views, which included both Sheets and Dashboards but classified them all as Sheets. With this update, the collector distinguishes between Sheets and Dashboards, assigning the correct type to each entity. This ensures a more accurate representation of hidden Views in Tableau.

A new collector, the AWS Database Migration Service (DMS) collector, is now available in public preview.

Tableau collector: Improve command guidance, documentation, and warnings about the required format of the Tableau API URL option.

Alteryx collector: The collector now harvests nested workflow nodes and catalogs their relationship with the workflow.

Azure Data Factory collector: Made improvements to Azure Data Factory lineage by enhancing the harvesting of lineage from parameterized dataset references. The collector now also harvests both downstream and upstream resources.

Oracle collector: Add a new parameter --autonomous-db-connection-string for connection string for autonomous DB.

Tableau collector: The collector now harvests relationships between SQL tables and workbooks, enhancing data connectivity and visualization.

AWS Glue collector: Enhanced the collector to harvest lineage from Glue Data Catalog tables to their underlying S3 objects and gather more metadata for tables. The enhanced collector is available with the command catalog-aws-glue, while the legacy collector remains available as catalog-aws-glue-legacy or catalog-awsglue for compatibility. Please coordinate with your Customer Success Director for a smooth transition to the new collector version soon.

Note that the AWS Glue collector is only available as an on-premise solution, not as a cloud collector.

Tableau collector:

Tableau collector:

Databricks collector: Fixed an issue where a null pointer exception occurred while harvesting tags from Databricks.

Databricks collector: Fixed an exception that occurred when the table type did not match any known types.

Fivetran collector: Updated to use new APIs for retrieving column lineage due to changes in Fivetran API.

Snowflake collector: Added support for parsing SQL that utilizes IDENTIFIER() function calls when passing exact table names.

Tableau collector (Preview): Added a null check in custom SQL Table logic to prevent errors.

SQL Server collector: Reduced excessive log and warning messages during dependency collection to streamline output.

Databricks collector: Improved error handling for connection issues with the Databricks host.

Databricks collector: Added support for harvesting resources with the browse privilege.

Tableau collector: Now supports harvesting lineage between Custom SQL tables and their upstream tables.

Power BI and Power BI Gov collectors: Handling calculated tables as a new type and cataloging table-level lineage to source tables and columns.

Monte Carlo collector: Updated the collector to remove reaction type from incidents, as it has been deprecated in Monte Carlo GraphQL responses.

Salesforce collector: Added a null check to prevent exceptions when last modified by information is missing.

The following two new collectors are now available in public preview.

Reltio collector: Resolved an issue that caused errors when cataloging containment relationships for certain attribute containers

Power BI and Power BI Gov collectors: Stopped cataloging JDBC types for columns in Power BI, as this task is best handled by the database collector. Power BI previously attempted to infer JDBC types based on its column types, which is now corrected.

Power BI and Power BI Gov collectors:

Tableau collector: Enhanced the collector to catalog unpublished views, expanding visibility into Tableau assets.

Snowflake collector: Enhanced the incremental collection process to prevent the deletion of specific database column resources, ensuring data integrity and continuity.

Monte Carlo collector: Updated to catalog Monte Carlo warehouse IDs instead of hostnames due to API changes.

Databricks collector: Fixed an issue where the --include-information-schema option produced an incorrect warning.

Denodo collector: Resolved an issue with the internal cleanup function that was unable to remove PRIMARY KEY statements from SQL.

Power BI Service and Power BI Gov collectors: Resolved an issue with handling parameters when they are referenced using @ symbol.

Databricks collector: Added support for harvesting SQL queries and their associated lineage. A new parameter Page size for harvesting queries (--query-pagination-limit) is introduced for this.

Power BI Service and Power BI Gov collectors: Added support for jdbcProperties in the datasources.yaml configuration for connecting to databases when resolving lineage.

Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server collectors: Database authentication issues are now reported as errors.

Oracle collector:

Redshift collector: The collector now correctly harvests distinct utility functions and procedures.

Databricks collector: Resolved an issue where column names were not recognized due to case sensitivity mismatches.

Oracle Collector: Now supports harvesting from Oracle Autonomous Database. A new parameter --autonomous-db is introduced for this.

Tableau collector: Now supports harvesting of personal space workbooks. A new parameter --tableau-catalog-personal-space-workbooks is introduced for this.

Power BI collector: Resolved an issue with parameter replacements in table source expressions when a parameter name is the same as the name of the table it defines.

SQL Server collector: Ensured encryption is enabled for connections when the encrypt JDBC property is configured.

Alteryx collector: Fixed an issue encountered while fetching workflow details when the user does not have permission on the workflow.

Go here to access release notes for versions released before 2025.