You can have perfectly partitioned tables, airtight IAM, and a tidy dbt project, and still hear the same question every Monday: "which table do I actually use for revenue, and where does the number come from?" That is not a modelling problem or a permissions problem. It is a metadata problem, and in Google Cloud it is now solved inside Dataplex Universal Catalog — the service that absorbed Data Catalog and is the default metadata layer behind BigQuery search, lineage, and the business glossary.
This tutorial walks through the practical parts: searching your estate, attaching structured metadata with aspects, publishing a business glossary, reading column-level lineage, and wiring all of it into CI so it does not rot. Everything below is doable with SQL, gcloud, and a little Python.
What changed, briefly
Data Catalog as a standalone product was shut down in January 2026; its tag templates and tags live on as aspect types and aspects in Dataplex Universal Catalog. If your Terraform still references google_data_catalog_tag_template, that is the migration you have been putting off. The practical model today:
| Concept | What it is | Where you see it |
|---|---|---|
| Entry | A catalog record for a table, view, model, dataset, or non-Google asset | BigQuery Studio search, Dataplex UI |
| Aspect type | A reusable schema for metadata (fields, enums, required flags) | Defined once per org |
| Aspect | An instance of an aspect type attached to an entry or a column | Table "Details" tab |
| Glossary term | A governed business definition, linkable to columns | Glossary section |
| Lineage | Automatically captured table- and column-level edges | Lineage tab, Lineage API |
Lineage is captured automatically for BigQuery jobs, Dataform, Cloud Composer, Datastream, and Dataproc; nothing to instrument for native SQL.
1. Search before you build
Most "we need a catalog" projects start with a discovery exercise that search already answers. Dataplex search is available in BigQuery Studio and from the CLI:
# Every table in prod whose name or description mentions revenue
gcloud dataplex entries search "revenue system=bigquery projectid=acme-prod" \
--project=acme-governance
# Tables nobody has queried: pair search with INFORMATION_SCHEMA
gcloud dataplex entries search "type=TABLE projectid=acme-prod" \
--project=acme-governance --format="value(linkedResource)" > all_tables.txt
Useful search qualifiers: system=bigquery, type=TABLE|VIEW|MODEL, projectid=, location=, label:, createtime>2026-01-01, and aspect: once you have aspects in place. Combine the result set with INFORMATION_SCHEMA.JOBS over the last 90 days and you have an evidence-based deprecation list instead of an opinion-based one.
2. Describe things where the work happens
Before reaching for aspects, use what BigQuery already gives you. Descriptions are indexed by search, and they are DDL, so they live in version control:
ALTER TABLE `acme-prod.finance.fct_revenue_daily`
SET OPTIONS (
description = "Certified daily revenue by product and region. Grain: one row per (date, product_id, region). Source of truth for the exec dashboard."
);
ALTER TABLE `acme-prod.finance.fct_revenue_daily`
ALTER COLUMN net_revenue_usd
SET OPTIONS (description = "Gross revenue minus refunds and discounts, converted at the daily close FX rate.");
In dbt or Dataform, these come free from model and column descriptions — enable persist_docs in dbt so documentation lands in BigQuery rather than only in a docs site nobody opens.
3. Aspect types: structured metadata, not free text
Free-text descriptions cannot be queried for policy. Aspects can. Define an aspect type once — say, a data-contract aspect every certified table must carry:
cat > data-contract.json <<'EOF'
{
"name": "data_contract",
"type": "record",
"recordFields": [
{ "name": "owner_team", "type": "string", "constraints": { "required": true } },
{ "name": "tier", "type": "enum", "constraints": { "required": true },
"enumValues": [ {"name": "gold"}, {"name": "silver"}, {"name": "bronze"} ] },
{ "name": "freshness_sla_hours", "type": "int" },
{ "name": "contains_pii", "type": "bool" },
{ "name": "retire_after", "type": "datetime" }
]
}
EOF
gcloud dataplex aspect-types create data-contract \
--location=global --project=acme-governance \
--metadata-template-file-name=data-contract.json \
--display-name="Data Contract"
Attach it to an entry:
PROJ_NUM=$(gcloud projects describe acme-prod --format='value(projectNumber)')
ENTRY="projects/${PROJ_NUM}/locations/us/entryGroups/@bigquery/entries/bigquery.googleapis.com/projects/acme-prod/datasets/finance/tables/fct_revenue_daily"
gcloud dataplex entries update "$ENTRY" --location=us --project=acme-prod \
--aspects='{"acme-governance.global.data-contract": {"data": {
"owner_team": "finance-data",
"tier": "gold",
"freshness_sla_hours": 6,
"contains_pii": false
}}}' \
--update-mask=aspects \
--aspect-keys='acme-governance.global.data-contract'
Now the metadata is queryable: aspect:data-contract.tier=gold in search returns your certified set, and an aspect can be attached to a single column (...data-contract@net_revenue_usd) when the fact is column-specific.
4. A business glossary that people actually link to
A glossary is only useful when terms are attached to columns, so a user hovering net_revenue_usd in BigQuery Studio sees the finance team's definition rather than guessing.
gcloud dataplex glossaries create acme-business-glossary \
--location=global --project=acme-governance --display-name="Acme Business Glossary"
gcloud dataplex glossary-categories create revenue \
--glossary=acme-business-glossary --location=global --project=acme-governance \
--display-name="Revenue"
gcloud dataplex glossary-terms create net-revenue \
--glossary=acme-business-glossary --location=global --project=acme-governance \
--display-name="Net Revenue" \
--description="Gross revenue less refunds, chargebacks, and discounts, excluding tax and shipping."
Terms are then linked to columns from the BigQuery Studio schema panel or the Dataplex API. Two rules keep a glossary alive: every term has a named human owner, and no term is created without at least one linked column. A glossary of 400 unlinked terms is a wiki with extra steps.
5. Lineage: answer "what breaks if I change this?"
Lineage is captured automatically for BigQuery jobs and is queryable through the Data Lineage API. The common operational question is impact analysis before a schema change:
from google.cloud import datacatalog_lineage_v1 as lineage
client = lineage.LineageClient()
parent = "projects/acme-prod/locations/us"
target = lineage.EntityReference(
fully_qualified_name="bigquery:acme-prod.staging.orders_raw"
)
# Everything downstream of orders_raw, one hop at a time
links = client.search_links(request={"parent": parent, "source": target})
for link in links:
print(link.target.fully_qualified_name)
# Which job produced that edge?
for proc in client.batch_search_link_processes(
request={"parent": parent, "links": [link.name]}
):
print(" via", proc.process)
Walk the graph recursively and you get a blast-radius report you can post on the change ticket. Column-level lineage is available for BigQuery SQL, which is what makes "is net_revenue_usd derived from the column we are about to drop?" answerable in seconds instead of a grep through a dbt repo.
A warning worth internalising: lineage is captured from jobs, so anything that moves data outside of a job — a manual CSV upload, an export to a spreadsheet, a reverse-ETL tool using the Storage Read API — produces a gap. Report those gaps rather than pretending the graph is complete; you can also push custom lineage events through the API for external systems.
6. Make it CI, not a cleanup sprint
Catalogs decay the moment they become someone's quarterly chore. Three automations keep ours honest:
- Fail the pipeline on missing contracts. In your deploy job, list entries in the production datasets, check each has a
data-contractaspect with atier, and fail the build on new tables without one. Ten lines of Python against the Dataplex client. - Alert on orphaned gold tables. Scheduled query: join the search results for
tier=goldagainstINFORMATION_SCHEMA.JOBSfor the last 30 days; anything certified but unqueried, or queried but uncertified, goes to a weekly digest. - Reconcile PII flags against policy tags. If
contains_pii = truebut no column in the table carries a policy tag, that is a governance finding — see our walkthrough of row-level security, column masking, and policy tags for the enforcement side, and Dataplex data quality scans for the correctness side.
7. A realistic rollout order
- Turn on descriptions in dbt/Dataform and backfill the top 50 tables by query volume. One afternoon, most of the value.
- Define two or three aspect types maximum: data contract, PII classification, and perhaps cost ownership. Resist the forty-field template.
- Use lineage read-only for a month before you build anything on it, so you learn where your gaps are.
- Start the glossary with the twenty metrics that appear in board reporting, each linked to a column.
- Only then automate enforcement, once the data is good enough that failing builds will not just be noise.
Where this lands
Metadata work earns its keep in the arguments it prevents: which table is canonical, who owns the fix, what breaks if this column goes. Dataplex Universal Catalog makes that cheap enough that there is no longer an excuse — the lineage is already being recorded whether you look at it or not.
If you want this stood up properly — aspect types modelled against your actual org, glossary seeded, lineage-based impact checks in CI — that is a short, well-bounded engagement for our team. See our BigQuery Data Governance and Compliance and Data Architecture Consulting services, or just get in touch with a description of your estate.