Catalog collector release notes
Important
Stay updated on collector releases! To keep up with the latest updates and enhancements to data.world collectors, subscribe to our RSS feed.
Release version 2.292
Important
Published versions of collectors are available as a docker image and a JAR file.
Details about the release
Item | Details |
---|---|
Release version | 2.292 |
Release date | 1 August 2025 |
Docker image ID |
|
Jar file |
|
New features and changes
SQL Server collector: Added support for harvesting agent jobs.
Sigma collector: Added configuration options to include or exclude workspaces, providing greater control over which resources are harvested.
Redshift collector: Now supports harvesting external tables defined via AWS Glue.
Microsoft Fabric collector:
Dataflow Gen2 is now treated as a separate resource type from Dataflows.
Added support for cataloging destinations and table-level lineage for sources and destinations in Dataflow Gen2 CI/CD types.
Microsoft Fabric and Power BI collectors: Now catalog refresh schedules for resources where refresh configuration is available, helping track automated data updates.
AWS Glue collector: Now identifies partitioned columns separately from other columns.
Bug Fixes
Marquez collector: Fixed a null pointer exception that could occur when a job lacked a latest run.
Release version 2.291
Important
Published versions of collectors are available as a docker image and a JAR file.
Details about the release
Item | Details |
---|---|
Release version | 2.291 |
Release date | 1 August 2025 |
Docker image ID |
|
Jar file |
|
New features and changes
Microsoft Fabric collector:
Added support for harvesting Apps and Org Apps in Microsoft Fabric.
Added support for harvesting GraphQL instances.
Bug Fixes
Marquez collector: Now skips unsupported dataset types, preventing errors during harvesting.
Release version 2.290
Important
Published versions of collectors are available as a docker image and a JAR file.
Details about the release
Item | Details |
---|---|
Release version | 2.290 |
Release date | 17 July 2025 |
Docker image ID |
|
Jar file |
|
New features and changes
A new collector, the OpenAPI collector, is now available in public preview. It supports harvesting metadata from APIs described using OpenAPI v3.0, enabling documentation and cataloging of API assets.
Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server, SAP HANA collectors: Added a Sensitive Data Classification option to allow classification using a hosted private-ai instance.
Microsoft Fabric collector:
Added support for harvesting Spark Job Definition details.
Now also captures Mirrored Database details, expanding coverage of key metadata elements in Microsoft Fabric.
Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server, SAP HANA collectors: Now include column statistics support for Date, Timestamp, and Boolean data types, enhancing profiling depth across supported databases.
Postgres collector: Supports AWS IAM authentication via secret and access key parameters, offering more secure and flexible credential management.
Bug Fixes
Oracle collector: Fixed an issue in the table index feature that previously caused permission errors or max open cursor issues by updating the query logic to use DBA_ views when available.
SSIS Collector: Now harvests deeply nested control flow executables, ensuring complete control flow visibility.
Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server SAP HANA collectors: Improved sampling behavior for environments where TABLESAMPLE is unsupported by falling back to LIMIT or TOP clauses to compute statistics more reliably.
Release version 2.289
Details about the release
Item | Details |
---|---|
Release version | 2.289 |
Release date | 4 July, 2025 |
Docker image ID |
|
Jar file |
|
New features and changes
A new collector, the AWS Lake Formation collector, is now available in public preview.
Microsoft Fabric collector: Now catalogs metadata for Eventhouses, including the addition of associated KQL databases, expanding visibility into the Microsoft Fabric ecosystem.
dbt core and cloud collectors: Added support for harvesting semantic model metadata, enriching the data model layer within your catalog.
Power BI collector: Introduced support for an additional Databricks source MQuery function type in lineage resolution, improving coverage and accuracy of Power BI lineage.
SSIS collector: Enhanced debug-level logging to support better root cause analysis for missing catalog resources, aiding in troubleshooting and diagnostics.
Bug fixes
Alteryx collector: Fixed an issue where the workflow description was not correctly captured from the user-provided meta info section.
AWS Glue collector: Resolved a null pointer exception that could occur when the Glue Data Catalog tables are empty, improving stability.
Snowflake collector: Improved data type standardization by stripping parenthesized size/length values (for example, VARCHAR(255) → VARCHAR) for cleaner and more consistent metadata.
Release version 2.288
Important
This release was for internal improvements and has no customer impacting changes.
Release version 2.287
Details about the release
Item | Details |
---|---|
Release version | 2.287 |
Release date | 17 June, 2025 |
Docker image ID |
|
Jar file |
|
New features and changes
Tableau collector: Added view-based filtering for bin and group fields to align with how calculated fields are handled.
Salesforce collector: Now harvests all reports and dashboards, not just recently viewed ones.
Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server collectors, SAP HANA collectors: Column decimal digits are now written only for appropriately typed columns.
Bug fixes
Tableau collector: Fixed a null pointer exception in column lineage processing for Custom SQL tables.
OpenAPI collector: Resolved errors caused by malformed spec files that previously triggered null pointer exceptions.
Release version 2.286
Details about the release
Item | Details |
---|---|
Release version | 2.286 |
Release date | 11 June, 2025 |
Docker image ID |
|
Jar file |
|
New features and changes
Logging framework (all collectors): Introduced minor updates to the logging framework. As a result, users may notice slightly different log messages compared to previous versions.
Snowflake collector: Upgraded the embedded Snowflake JDBC driver to version 3.34.2, addressing potential exceptions and improving stability.
Azure data factory collector: Now catalogs parameters used in parameterized linked services, along with the relationship between each linked service and its data source, providing deeper lineage visibility.
PowerBI collector: Added support for parsing lineage from certain SQL statement types without requiring database credentials, making it easier to extract lineage in more restricted environments.
Bug fixes
Tableau collector: Prevented exceptions that could occur when harvesting table-view relationships, particularly when table information is missing from the Tableau GraphQL API.
PowerBI collector: Fixed an issue with Denodo sources that use custom SQL, improving support for a wider range of PowerBI source types.
SQL Server Integration Services (SSIS) collector: Resolved an exception that occurred during the harvesting of column information, enhancing reliability in metadata extraction.
Release version 2.285
Details about the release
Item | Details |
---|---|
Release version | 2.285 |
Release date | 3 June, 2025 |
Docker image ID |
|
Jar file |
|
New features and changes
Tableau collector: Now determines the associated project for a Custom SQL Table based on its workbook rather than its datasource, improving accuracy in project assignments.
PowerBI collector: Added support for Oracle Autonomous Database as a source, expanding connectivity and metadata coverage within PowerBI environments.
Release version 2.284
Details about the release
Item | Details |
---|---|
Release version | 2.284 |
Release date | 29 May, 2025 |
Docker image ID |
|
Jar file |
|
New features and changes
All collectors (on-premise version): Updated the logging behavior so that each new collector run will truncate existing log files written to the local filesystem, rather than appending to them. This means logs from previous runs will be overwritten during subsequent runs.
This change does not affect uploaded log files—those remain intact.
SSRS collector: Added support for harvesting metadata using the SOAP API, enabling compatibility with older SSRS versions that do not support the REST API.
Microsoft Fabric collector: Added support for harvesting lineage from custom SQL queries used in Fabric Lakehouse and Warehouse sources
Bug fixes
Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server collectors, SAP HANA collectors: Resolved an issue that caused exceptions when harvesting index metadata for tables lacking a defined type.
SQL Server collector: Fixed an error in how table names were qualified when calling the sp_spaceused stored procedure—ensuring accurate harvesting of table size metadata.
QlikSense collector: Prevented exceptions that occurred when the created-by or modified-by user information was missing from collected resources.
Databricks collector: Expanded lineage harvesting to include relationships between objects across different schemas, rather than only within the same schema.
Release version 2.283
Details about the release
Item | Details |
---|---|
Release version | 2.283 |
Release date | 21 May, 2025 |
Docker image ID |
|
Jar file |
|
New features and changes
Tableau collector: The collector Now harvests the last refresh date for Tableau data sources, where available—providing better visibility into data currency and freshness.
Bug fixes
Tableau collector: Fixed an issue that was preventing the harvesting of certain lineage relationships between Column Fields and their underlying database columns.
Release version 2.282
Details about the release
Item | Details |
---|---|
Release version | 2.282 |
Release date | 20 May, 2025 |
Docker image ID |
|
Jar file |
|
Bug fixes
SSIS collector: Fixed an issue that caused a stack overflow error when cycles were present in execution flows.
Release version 2.281
Details about the release
Item | Details |
---|---|
Release version | 2.281 |
Release date | 16 May, 2025 |
Docker image ID |
|
Jar file |
|
Bug fixes
Tableau collector:
Resolved an issue where re-authentication failed after API token timeouts, improving reliability in long-running sessions.
Fixed a bug in SQL parsing for Custom SQL Tables when referenced tables were missing or not found.
Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server collectors, SAP HANA collectors: Corrected the calculation of sampling percentages for column statistics, which were previously computed incorrectly in some cases.
Postgres and Redshift collectors: Added support for the ~~* symbol in view definition SQL, which serves as an alias for the LIKE keyword in SQL syntax.
SSIS Collector: Improved logging and resolved a NullPointerException caused by missing elements in execution XML files.
Release version 2.280
Details about the release
Item | Details |
---|---|
Release version | 2.280 |
Release date | 14 May, 2025 |
Docker image ID |
|
Jar file |
|
New features and changes
PowerBI collector: Improved performance by handling PowerBI API retry instructions more efficiently.
Tableau collector: Added enhanced logging to monitor collector progress and track cataloged resources.
Athena collector:
Now harvests the S3 path for database tables, enabling lineage from Athena to S3 buckets and objects.
Also collects associated AWS tags, enriching metadata coverage.
Bug fixes
ADF collector: Fixed an issue where unparseable JSON in certain parameter values caused collector failures.
BigQuery collector: Correctly handles cases where the BigQuery API does not return table details, preventing errors.
PowerBI collector: Supports automated migration of resource IRIs (identifiers) from earlier collector versions to maintain continuity.
Release version 2.279
Details about the release
Item | Details |
---|---|
Release version | 2.279 |
Release date | 9 May 2025 |
Docker image ID |
|
Jar file |
|
New features and changes
QlikSense collector: Implemented a rate limiter to improve performance and prevent API errors during metadata harvesting.
AWS S3 collector: Enhanced performance by applying regex filters before fetching objects within buckets, reducing unnecessary API calls.
Bug fixes
Tableau collector: Fixed a gap in harvesting of lineage relationships between Tableau column fields and database columns.
Microsoft Fabric collector: Resolved an issue where some Data Pipelines failed to deserialize due to varying data types based on configuration.
SSIS collector: Fixed an exception caused by missing or empty elements in SSIS executable descriptor XML files.
Athena collector: Updated harvesting of databases to accommodate environments with more than 100 databases.
Release version 2.278
Details about the release
Item | Details |
---|---|
Release version | 2.278 |
Release date | 30 April, 2025 |
Docker image ID |
|
Jar file |
|
Bug fixes
Microsoft Fabric collector:
Fixed an issue with parameters and variables in data pipelines to ensure correct cataloging of parameter and variable names.
Resolved a problem where catalog records for pipeline activities and runs were not being created.
Corrected the Is Active property for activities, which was previously always set to false.
Release version 2.277
Details about the release
Item | Details |
---|---|
Release version | 2.277 |
Release date | 29 April, 2025 |
Docker image ID |
|
Jar file |
|
New features and changes
Microsoft Fabric collector:
Harvests notebook definitions along with endorsement details, expanding the collected metadata. A new parameter Disable harvesting notebook definition (--disable-notebook-definition) is introduced for this.
Harvests data pipeline activities and run information, providing deeper visibility into pipeline executions.
dbt Core and dbt Cloud collectors: dbt sources are now associated with database schemas (instead of tables) through a non-lineage relationship, improving metadata accuracy.
Tableau collector: Enhanced log tracing around column field lineage to improve troubleshooting and debugging clarity.
Release version 2.276
Details about the release
Item | Details |
---|---|
Release version | 2.276 |
Release date | 24 April, 2025 |
Docker image ID |
|
Jar file |
|
New features and changes
Databricks collector: Now captures table lineage to corresponding AWS S3 objects.
Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, SQL Server collectors: Added support for harvesting metadata about table indexes, providing deeper insight into database structure and performance optimization.
Bug fixes
Tableau collector: Improved handling of missing connection types on Tableau Server to prevent ingestion errors and ensure smoother metadata extraction.
Release version 2.275
Details about the release
Item | Details |
---|---|
Release version | 2.275 |
Release date | 22 April, 2025 |
Docker image ID |
|
Jar file |
|
New features and changes
dbt collector: Transitioned from using the manifest to the catalog to retrieve database identifiers. This resolves issues with quoted identifiers (for example, Snowflake).
Athena, Amazon Database Migration Service (DMS), Amazon DynamoDB, AWS Glue, Amazon QuickSight, Amazon S3 collectors: The collectors now support multiple authentication methods, offering greater flexibility and compatibility. The following new parameters are introduced for the new authentication methods:
Explicitly supplied static credentials authentication using AWS Access Key ID (--aws-access-key-id) and AWS Secret Access Key (--aws-secret-access-key).
Amazon Database Migration Service (DMS) collector: Introduced support for S3 as both a source and target endpoint.
Bug fixes
Tableau collector: Implemented a nonNull filter for relatedFields, addressing errors and improving stability in the Tableau collector.
Release version 2.274
Details about the release
Item | Details |
---|---|
Release version | 2.274 |
Release date | 17 April, 2025 |
Docker image ID |
|
Jar file |
|
Bug fixes
Databricks collector: Fixed an issue where table lineage could not be resolved when information was incomplete.
Tableau collector: Corrected processing of column fields to resolve subtle errors in field-to-database lineage.
Azure data factory collector: Improved handling of missing table details when collecting lineage.
All collectors: Fixed logging issues where some messages were not being written to the log file as expected.
Release version 2.273
Details about the release
Item | Details |
---|---|
Release version | 2.273 |
Release date | 12 April, 2025 |
Docker image ID |
|
Jar file |
|
Bug fixes
Monte Carlo collector: Fixed alignment issues in generated table information from Monte Carlo Monitor by properly escaping pipes ('|') in monitor names.
Release version 2.272
Details about the release
Item | Details |
---|---|
Release version | 2.272 |
Release date | 11 April, 2025 |
Docker image ID |
|
Jar file |
|
New features and changes
A new collector, the SAP HANA collector, is now available in public preview.
SSIS collector: Now captures the connections between packages through the control flow, improving visibility of package relationships.
Databricks collector: The collector now harvests lineage only within the current schema. A new option Harvest entire lineage (--harvest-entire-lineage) is added to enable harvesting lineage from external schemas.
Microsoft Fabric collector: Enhanced to support additional syntaxes for Semantic Model table connections to Fabric warehouse/lakehouse resources, broadening compatibility and connectivity.
Bug fixes
QlikSense collector: Made performance improvements to increase the efficiency and speed of the collector.
Microsoft Fabric collector: Fixed issues with Lakehouses and SQL endpoints to ensure database resources are associated with the correct resource, enhancing accuracy.
Release version 2.271
Details about the release
Item | Details |
---|---|
Release version | 2.271 |
Release date | 3 April, 2025 |
Docker image ID |
|
Jar file |
|
Bug fixes
Tableau collector: Fixed an issue to ensure proper handling of sites and projects with large quantities of column and data source fields.
Databricks collector: Resolved a parsing issue in view queries where aliases starting with a number caused failures.
Release notes for previous versions
Go here to access release notes for previous version.