Skip to content
mcp-data-platform composable mcp data platform
v1.x part of txn2 ↗

Multi-Provider Configuration

mcp-data-platform supports connecting to multiple instances of each service type. This allows you to:

  • Query different Trino clusters (production, staging, data warehouse)
  • Search across multiple DataHub instances
  • Access different S3 accounts or regions

Configuring Multiple Instances

Each toolkit section accepts multiple named instances:

toolkits:
  trino:
    enabled: true
    instances:
      production:
        host: trino-prod.example.com
        port: 443
        user: analyst
        ssl: true
        catalog: hive
        connection_name: Production

      staging:
        host: trino-staging.example.com
        port: 443
        user: analyst
        ssl: true
        catalog: hive
        connection_name: Staging

      warehouse:
        host: trino-dw.example.com
        port: 443
        user: analyst
        ssl: true
        catalog: iceberg
        connection_name: Data Warehouse
    default: production

  datahub:
    enabled: true
    instances:
      primary:
        url: https://datahub.example.com
        token: ${DATAHUB_TOKEN}
        connection_name: Primary Catalog

      legacy:
        url: https://datahub-legacy.example.com
        token: ${DATAHUB_LEGACY_TOKEN}
        connection_name: Legacy Catalog
    default: primary

  s3:
    enabled: true
    instances:
      data_lake:
        region: us-east-1
        access_key_id: ${AWS_ACCESS_KEY_ID}
        secret_access_key: ${AWS_SECRET_ACCESS_KEY}
        connection_name: Data Lake

      archive:
        region: us-west-2
        access_key_id: ${ARCHIVE_AWS_ACCESS_KEY_ID}
        secret_access_key: ${ARCHIVE_AWS_SECRET_ACCESS_KEY}
        connection_name: Archive
    default: data_lake

Using Connections in Tools

Every tool accepts a connection parameter to specify which instance to use:

Query the orders table in the staging environment

Tool call: trino_query with: - query: SELECT * FROM orders LIMIT 10 - connection: staging

Listing Available Connections

Use the *_list_connections tools to see configured instances:

  • trino_list_connections - Lists all Trino connections
  • datahub_list_connections - Lists all DataHub connections
  • s3_list_connections - Lists all S3 connections

Example response:

{
  "connections": [
    {
      "name": "production",
      "display_name": "Production",
      "host": "trino-prod.example.com",
      "catalog": "hive"
    },
    {
      "name": "staging",
      "display_name": "Staging",
      "host": "trino-staging.example.com",
      "catalog": "hive"
    }
  ]
}

Default Connection

The first instance configured becomes the default. You can rely on this behavior or explicitly specify which instance to use for semantic providers:

semantic:
  provider: datahub
  instance: primary    # Use the "primary" DataHub instance

query:
  provider: trino
  instance: production # Use the "production" Trino instance

storage:
  provider: s3
  instance: data_lake  # Use the "data_lake" S3 instance

Cross-Enrichment with Multiple Providers

When you have multiple instances, the semantic enrichment uses the configured provider instances:

semantic:
  provider: datahub
  instance: primary       # Semantic context comes from this instance

query:
  provider: trino
  instance: production    # Query availability checks use this instance

storage:
  provider: s3
  instance: data_lake     # Storage availability uses this instance

enrichment:
  trino_semantic_enrichment: true   # All Trino results get DataHub context
  datahub_query_enrichment: true    # DataHub results show Trino availability
  column_context_filtering: true    # Only enrich columns referenced in SQL (default: true)

With this configuration:

  • Querying any Trino instance enriches results with metadata from the primary DataHub
  • Searching any DataHub instance shows query availability from production Trino

Environment-Specific Configuration

Use environment variables to manage different environments:

toolkits:
  trino:
    enabled: true
    instances:
      main:
        host: ${TRINO_HOST}
        port: ${TRINO_PORT}
        user: ${TRINO_USER}
        password: ${TRINO_PASSWORD}
        ssl: true
    default: main

Development: 

export TRINO_HOST=localhost
export TRINO_PORT=8080
export TRINO_USER=admin

Production: 

export TRINO_HOST=trino-prod.example.com
export TRINO_PORT=443
export TRINO_USER=service-account
export TRINO_PASSWORD="..."

Connection-Specific Personas

You can restrict personas to specific connections using tool patterns:

personas:
  analyst:
    display_name: Data Analyst
    roles: ["analyst"]
    tools:
      allow: ["*"]
      deny: []
    connections:
      allow: ["*"]

  staging_user:
    display_name: Staging User
    roles: ["staging"]
    tools:
      allow:
        - "platform_info"
        - "search"              # Required: the search-first gate refuses
        - "fetch"               #   trino_query until search has been called
        - "trino_query"         # Can query
        - "trino_browse"        # Can explore catalogs/schemas/tables
        - "trino_describe_*"    # Can describe tables
        - "list_connections"    # Can list connections
      deny:
        - "trino_explain"       # Cannot see execution plans
    connections:
      allow: ["staging"]        # Only the staging Trino instance

Tool patterns are one of two axes. Personas also restrict which toolkit connections a caller may reach, through connections.allow / connections.deny — connections are deny-by-default, so a persona that should use data must list the connections it may use. See Connection Access Control.

Practical Examples

Data Mesh Setup

toolkits:
  datahub:
    enabled: true
    instances:
      sales:
        url: https://datahub-sales.example.com
        token: ${SALES_DATAHUB_TOKEN}
        connection_name: Sales Domain

      marketing:
        url: https://datahub-marketing.example.com
        token: ${MARKETING_DATAHUB_TOKEN}
        connection_name: Marketing Domain

      finance:
        url: https://datahub-finance.example.com
        token: ${FINANCE_DATAHUB_TOKEN}
        connection_name: Finance Domain
    default: sales

Hybrid Cloud

toolkits:
  s3:
    enabled: true
    instances:
      aws:
        region: us-east-1
        connection_name: AWS S3

      gcs:
        endpoint: https://storage.googleapis.com
        region: auto
        access_key_id: ${GCS_ACCESS_KEY}
        secret_access_key: ${GCS_SECRET_KEY}
        connection_name: Google Cloud Storage

      minio:
        endpoint: http://minio.internal:9000
        use_path_style: true
        disable_ssl: true
        access_key_id: ${MINIO_ACCESS_KEY}
        secret_access_key: ${MINIO_SECRET_KEY}
        connection_name: On-Premise MinIO
    default: aws

Next Steps