Governance Workflow¶

Knowledge capture turns catalog metadata into something that evolves with use. Instead of metadata being a static artifact maintained by a central team, it improves continuously as domain experts share what they know during their normal work. This is active metadata management: the catalog gets better every time someone uses it.

This page covers the admin-side workflow for reviewing captured insights and writing approved changes back to DataHub.

The Review Process¶

All captured insights start with status pending. Admins review them through the apply_knowledge tool or the Admin REST API.

Bulk Review¶

Get an overview of all pending insights:

{"action": "bulk_review"}

Response:

{
  "total_pending": 7,
  "by_entity": [
    {
      "entity_urn": "urn:li:dataset:(urn:li:dataPlatform:trino,hive.sales.orders,PROD)",
      "count": 3,
      "categories": ["correction", "business_context"],
      "latest_at": "2025-01-15T14:30:00Z"
    },
    {
      "entity_urn": "urn:li:dataset:(urn:li:dataPlatform:trino,hive.finance.revenue,PROD)",
      "count": 4,
      "categories": ["data_quality", "usage_guidance", "enhancement"],
      "latest_at": "2025-01-15T16:45:00Z"
    }
  ],
  "by_category": {
    "correction": 2,
    "business_context": 1,
    "data_quality": 2,
    "usage_guidance": 1,
    "enhancement": 1
  },
  "by_confidence": {
    "high": 3,
    "medium": 4
  },
  "oldest_pending_at": "2025-01-15T14:30:00Z",
  "oldest_pending_age_days": 94,
  "pending_over_30d": 2,
  "note": "Counts are pending insights only (the global review queue). by_entity counts an insight once per entity URN it carries and omits insights that have no entity URN, so it does not sum to total_pending. Pass itemize:true to enumerate every pending insight (with id, captured_by and sink_class)."
}

The counts label their own scope: total_pending is the size of the global pending queue, while by_entity counts a multi-entity insight once per URN and omits entity-agnostic insights, so it does not sum to total_pending.

Review-Queue Staleness¶

The knowledge flywheel only turns if humans work the queue, so bulk_review also reports how stale the pending queue is (present on both the counts and itemize paths, omitted when the queue is empty):

oldest_pending_at / oldest_pending_age_days: the age of the oldest pending insight, so a long-unreviewed queue is visible at a glance.
pending_over_30d: the count of pending insights aged 30 or more days (the accumulating review debt). The portal's per-row age badge flags a row stale at the same 30-day mark, so the count and the badge agree.

An agent can use these to nudge a reviewer (for example, "6 insights are pending review, the oldest is 94 days old"). The same rollup appears in platform_info under features.knowledge_apply.review_queue for any caller who can reach apply_knowledge. In the admin portal, the review queue view badges each pending insight by age and can be sorted oldest-first (order=oldest on the insights API) to work the stalest debt first.

Enumerate the Review Queue¶

bulk_review returns counts only. To list the pending insights themselves (the relevance-ranked search tool cannot enumerate the queue completely), pass itemize: true. The list is paginated by offset/limit (default 20, max 100) and carries next_offset while more remain:

{"action": "bulk_review", "itemize": true, "limit": 50}

Each returned insight is the full record, including its insight_text body so a review pass can read every insight without a per-insight fetch, plus id, captured_by (the author), session_id, persona, sink_class, category, confidence, status, entity_urns, related_columns, and created_at. The suggested_actions array is replaced by suggested_actions_count to keep the listing bounded (fetch mcp:insight:<id> for the full actions). An entity-agnostic insight (one with no entity_urns, invisible to per-entity review) is still enumerable here. Page through by passing the previous response's next_offset as the next offset.

Each part of the response is bounded so it stays under the tool output limit even for a large queue: the insights window is byte-budgeted (when the budget is reached the page returns fewer than limit insights and sets page_size_capped: true), and by_entity is capped to the busiest entities (by_entity_truncated: true flags the cut; each insight still carries its full entity_urns). Treat page_size_capped as a normal "more remain" signal and continue paging with next_offset.

Review by Entity¶

Drill into a specific entity to see its pending insights alongside current DataHub metadata:

{
  "action": "review",
  "entity_urn": "urn:li:dataset:(urn:li:dataPlatform:trino,hive.sales.orders,PROD)"
}

Response includes:

Current DataHub metadata (description, tags, glossary terms, owners)
All insights for this entity with their categories, text, confidence, and suggested actions

This side-by-side view helps admins assess whether an insight adds value compared to what's already in the catalog.

Approve or Reject¶

Transition insight statuses with optional review notes:

{
  "action": "approve",
  "insight_ids": ["a1b2c3d4e5f6", "f6e5d4c3b2a1"],
  "review_notes": "Verified with data engineering team"
}

{
  "action": "reject",
  "insight_ids": ["deadbeef1234"],
  "review_notes": "Already documented in the column description"
}

Response:

{
  "action": "approve",
  "updated": 2,
  "total": 2
}

If any IDs are invalid or the status transition is not allowed, those are reported in an errors array without blocking the valid ones.

Synthesizing Changes¶

Once insights are approved, the synthesize action gathers them and builds structured change proposals:

{
  "action": "synthesize",
  "entity_urn": "urn:li:dataset:(urn:li:dataPlatform:trino,hive.sales.orders,PROD)"
}

Response:

{
  "entity_urn": "urn:li:dataset:(urn:li:dataPlatform:trino,hive.sales.orders,PROD)",
  "current_metadata": {
    "description": "Order records",
    "tags": ["financial"],
    "glossary_terms": [],
    "owners": ["Data Platform Team"]
  },
  "approved_insights": [
    {
      "id": "a1b2c3d4e5f6",
      "category": "correction",
      "insight_text": "The amount column represents gross margin before returns, not revenue.",
      "suggested_actions": [
        {"action_type": "update_description", "target": "entity", "detail": "Order records with gross margin amounts (before returns)"}
      ]
    }
  ],
  "proposed_changes": [
    {
      "change_type": "update_description",
      "target": "entity",
      "current_value": "Order records",
      "suggested_value": "Order records with gross margin amounts (before returns)",
      "source_insight_ids": ["a1b2c3d4e5f6"]
    }
  ]
}

The synthesis output shows current values alongside proposed changes, so admins can see exactly what will change. The source_insight_ids field traces each change back to the insight that proposed it.

Applying Changes¶

The apply action writes changes to DataHub and records a changeset. This is the data catalog write-back step.

{
  "action": "apply",
  "entity_urn": "urn:li:dataset:(urn:li:dataPlatform:trino,hive.sales.orders,PROD)",
  "changes": [
    {
      "change_type": "update_description",
      "target": "entity",
      "detail": "Order records with gross margin amounts (before returns)"
    },
    {
      "change_type": "add_tag",
      "target": "entity",
      "detail": "gross-margin"
    }
  ],
  "insight_ids": ["a1b2c3d4e5f6"],
  "confirm": true
}

When require_confirmation is enabled in configuration and confirm is not true, the tool returns a confirmation prompt instead of applying:

{
  "confirmation_required": true,
  "entity_urn": "urn:li:dataset:(urn:li:dataPlatform:trino,hive.sales.orders,PROD)",
  "changes_count": 2,
  "message": "Set confirm: true to apply these changes."
}

Supported Change Types¶

Change Type	Description	Target Field	Detail Field
`update_description`	Update entity or column description	`column:<fieldPath>` for columns, empty for dataset-level	New description text
`add_tag`	Add a tag to the entity	(ignored)	Tag name or URN (e.g., `pii` or `urn:li:tag:pii`)
`remove_tag`	Remove a tag from the entity	(ignored)	Tag name or URN to remove
`add_glossary_term`	Associate a glossary term	(ignored)	Glossary term name or URN
`flag_quality_issue`	Add fixed `QualityIssue` tag and, on incident-supporting entity types with a detail, raise a deduped DataHub incident ("Data quality issue") carrying the detail	(ignored)	Issue description (raised as the incident description; empty, or an unsupported entity type, means tag only)
`add_documentation`	Add a documentation link	URL of the documentation	Link description
`add_curated_query`	Create a reusable Query entity linked to the dataset	(ignored)	Query name. Also requires `query_sql` (SQL statement) and optionally `query_description`

Tag names and glossary term names are automatically normalized to full DataHub URNs (e.g., pii becomes urn:li:tag:pii).

Apply Response¶

{
  "changeset_id": "cs_x1y2z3a4b5c6",
  "entity_urn": "urn:li:dataset:(urn:li:dataPlatform:trino,hive.sales.orders,PROD)",
  "changes_applied": 2,
  "insights_marked_applied": 1,
  "revertible": true,
  "resulting_state": {
    "description": "Order records with gross margin amounts (before returns)",
    "tags": ["urn:li:tag:gross-margin"],
    "glossary_terms": [],
    "owners": []
  },
  "message": "Changes applied to DataHub. Roll back with action=rollback changeset_id=cs_x1y2z3a4b5c6. changes_applied counts requested changes; verify against resulting_state below."
}

Source insights move to applied status with a reference to the changeset.

The revertible boolean states whether the changeset can be rolled back automatically before you try. Rollback is all-or-nothing (it reverts nothing if any single change lacks a recoverable before-image), so revertibility is binary: true only when every change is revertible, false when any change is not (there is no partial rollback). It is computed with the same gate rollback enforces, so the response never advertises a rollback the changeset cannot perform. When revertible is false, an unrevertible_change_types array names the blocking change types and the message states why (no before-image) instead of instructing a rollback. The common case is a column-level update_description: the before-image captures only the entity-level description, so a column description change is recorded for audit but cannot be auto-reverted; restore prior state with a new apply.

revertible reports structural revertibility (whether the recorded before-image supports an inverse), not runtime availability. A structurally revertible changeset can still be refused at rollback time because it was already rolled back or a newer changeset has since modified the same aspect. The list_changesets view folds the already-rolled-back state into its revertible flag; the newer-changeset conflict is surfaced only when the rollback is attempted.

changes_applied counts the changes that were dispatched without error; a duplicate add (for example a tag that was already present) is a no-op upstream and still counts. The resulting_state field is a fresh read-back of the entity's description, tags, glossary terms, and owners after the apply, so callers can confirm what actually persisted without a follow-up call. Writes are not transactional: if a change in the middle of the list fails, earlier changes have already persisted and are reported in the error message rather than silently rolled back.

resulting_state reads tags and glossary_terms from the authoritative DataHub aspects. REST-exposed types (datasets, dashboards, charts, data flows, data jobs, containers, data products) are read from the REST aspects, and three GraphQL-only types (domain, glossaryTerm, glossaryNode) are read through the entity query (which surfaces those aspects as of mcp-datahub v1.10.2). description and owners come from the entity query for datasets and dashboards and from dedicated getters for data products and glossary terms.

Some fields cannot be read for some entity types and come back empty even when set, so for those types treat an empty value as "not read" rather than "not set":

tags and glossary_terms are empty for document entities (no read path exists for their associations).
description and owners are empty for entity types with no dedicated read and no dataset/dashboard entity-query fragment (domain, glossaryNode, container, chart, dataFlow, dataJob).

These same gaps apply to the changeset before-image, so a rollback cannot restore a field it could not read: do not rely on rollback to restore tags/glossary_terms on a document, or description/owners on those getter-less types.

Changeset Tracking¶

Every apply action creates a changeset record in the knowledge_changesets table:

Field	Description
`id`	Unique changeset identifier
`target_urn`	The DataHub entity that was modified
`change_type`	Summary of change types applied (or `multiple`)
`previous_value`	Entity metadata before changes (description, tags, glossary terms, owners)
`new_value`	Changes that were applied
`source_insight_ids`	Insights that produced this changeset
`applied_by`	User who applied the changes
`rolled_back`	Whether this changeset has been reverted
`rolled_back_by`	Who reverted the changes
`rolled_back_at`	When the changes were reverted

The previous_value field captures the entity's metadata (description, tags, glossary terms, owners) at the time of application. This before-image is what bounds rollback: a change can be reverted only when its prior state is recoverable from this snapshot.

Discovering Changesets¶

Use the list_changesets action to find an entity's changesets without already holding their ids (for example, before a rollback):

{ "action": "list_changesets", "entity_urn": "urn:li:dataset:(urn:li:dataPlatform:trino,hive.sales.orders,PROD)" }

Each entry returns changeset_id, created_at, applied_by, change_type, source_insight_ids, the current rolled_back status, and the same revertible boolean (plus unrevertible_change_types when false) the apply response carries, so a caller can filter out structurally unrevertible or already-rolled-back changesets before issuing a rollback.

Rollback¶

A changeset can be rolled back through either the apply_knowledge MCP tool or the Admin REST API. Both paths use the same revert engine.

{ "action": "rollback", "changeset_id": "cs_x1y2z3a4b5c6", "confirm": true }

curl -X POST \
  https://mcp.example.com/api/v1/admin/knowledge/changesets/cs_x1y2z3a4b5c6/rollback \
  -H "Authorization: Bearer $ADMIN_TOKEN"

Rollback reverts each change the apply made, back to its before-image:

A tag, glossary term, or documentation link the apply added is removed, unless it was already present in the before-image (in which case the add was a no-op and the pre-existing value is kept). This is what preserves an entity's canonical glossary term when an apply added another term alongside it.
A tag the apply removed is restored.
A description the apply changed is reset to the prior description.
An incident the apply created (the flag_quality_issue detail incident) is resolved. Its URN is recorded in the changeset's new_value (created_urns) so the rollback can find it.

The rollback also transitions the changeset's source insights from applied to rolled_back and records the rollback as its own auditable tool call referencing the original changeset_id.

When rollback is refused (rather than silently applied):

The changeset has already been rolled back.
A newer, not-yet-rolled-back changeset has since modified the same aspect on the same entity. Reverting would clobber that newer change, so the rollback is blocked and names the conflicting changeset; roll the newer one back first, or restore the desired state with a fresh apply.
The changeset contains change types whose prior state was not captured in the before-image and therefore cannot be reverted automatically: column-level descriptions, structured properties, incidents, curated queries, context documents, and prompts. For these, restore the desired state with a new apply.
The changeset targets an entity type whose affected field could not be read into the before-image, so reverting would write an empty value over a real one. This covers update_description on entity types with no readable description (domain, glossaryNode, container, chart, dataFlow, dataJob) and tag or glossary-term changes on document entities. The rollback is refused rather than risking the overwrite; restore the desired state with a new apply.

When the changeset contains a mix of revertible and unrevertible change types, the rollback is refused as a whole so it never leaves the entity in a partially reverted state.

Complete Workflow Example¶

An analyst discovers that the amount column in the orders table represents gross margin, not revenue. Here's how that knowledge flows through the system:

sequenceDiagram
    participant Analyst
    participant AI as AI Assistant
    participant Platform as mcp-data-platform
    participant DB as PostgreSQL
    participant Admin
    participant DH as DataHub

    Note over Analyst,AI: Discovery phase
    Analyst->>AI: What does the amount column mean?
    AI->>Platform: trino_describe_table(orders)
    Platform-->>AI: amount: DECIMAL (no description)
    Analyst->>AI: That's gross margin before returns,<br/>not revenue like the name suggests

    Note over AI,Platform: Capture
    AI->>Platform: memory_capture(<br/>type: schema_entity,<br/>source: user,<br/>entity_urns: [urn:li:dataset:...orders...],<br/>content: "amount column is gross margin<br/>before returns, not revenue",<br/>confidence: high,<br/>suggested_actions: [{<br/>  action_type: update_description,<br/>  target: amount,<br/>  detail: "Gross margin before returns"<br/>}])
    Platform->>DB: INSERT (status: pending)
    Platform-->>AI: Insight captured: a1b2c3

    Note over Admin,DH: Review phase (later)
    Admin->>Platform: apply_knowledge(action: bulk_review)
    Platform-->>Admin: 1 pending insight for orders

    Admin->>Platform: apply_knowledge(action: review,<br/>entity_urn: ...orders...)
    Platform->>DH: Get current metadata
    Platform-->>Admin: Current: no description<br/>Insight: "amount is gross margin"

    Admin->>Platform: apply_knowledge(action: approve,<br/>insight_ids: [a1b2c3])
    Platform->>DB: UPDATE status = approved

    Note over Admin,DH: Apply phase
    Admin->>Platform: apply_knowledge(action: synthesize,<br/>entity_urn: ...orders...)
    Platform-->>Admin: Proposed: update amount description

    Admin->>Platform: apply_knowledge(action: apply,<br/>changes: [{update_description,<br/>amount, "Gross margin before returns"}],<br/>insight_ids: [a1b2c3],<br/>confirm: true)
    Platform->>DH: Update column description
    Platform->>DB: INSERT changeset (previous_value saved)
    Platform->>DB: UPDATE insight status = applied
    Platform-->>Admin: Changeset cs_x1y2 recorded

The next time anyone queries this table, the enriched response includes the corrected description.

Next Steps¶

Admin API -- REST endpoints for managing insights and changesets
Knowledge Overview -- feature overview, categories, configuration
Audit Logging -- all knowledge tool calls are audit logged