Metadata presentation logic
While data engineers and admins work to catalog the resources and build the base of the org’s catalog (ddw-catalogs), stewards and data consumers are editing and enriching the catalog from within the platform.
Base layer
The base layer of your catalog is created by adding graph files to the Data Catalog Configuration (ddw-catalogs) dataset. The base layer is meant to capture information about what exists, load it into the platform, and define how to present this information in the UI. Human handling of this dataset should be limited. The base layer includes:
Automated cataloging of metadata (DWCC outputs)
Bulk import of a glossary
Catalog configuration files (customized metadata profile (MDP) files)
User layer
Edits created by users directly in the UI or via APIs are added to the user layer.
Catalog enrichment and management (manual edits) add context and get knowledge out of humans. Direct user edits should be applied in the user layer.
Presentation
Both the user layer and base layer are combined to present a “merged” or “flattened” view of the catalog to the user in the UI. The full flattened graph contains everything that was bulk loaded, as well as any changes made through the UI or the API. The user layer overrides the base layer where applicable.
Important
Everything required to render resource metadata must be stated in the MDP file.
If certain metadata is present in your catalog graph (no matter which layer) but it is not defined in the MDP, it won’t show up in the UI.
If you remove sections from the MDP, those presentations will be removed from the UI, even if there is underlying metadata.