Troubleshooting

This guide helps you diagnose and resolve common issues with mcp-data-platform. If you don't find your issue here, check the GitHub Issues or open a new one.

---

Quick Diagnosis

Start here to quickly identify your issue category.

Symptom	Likely Cause	Jump To
Server exits immediately	Configuration error	Server Won't Start
401 Unauthorized	Invalid credentials	Authentication Issues
403 Forbidden	Persona/tool mismatch	Persona Issues
No enrichment data	Enrichment misconfigured	Enrichment Issues
Slow responses	Performance bottleneck	Performance Issues
Connection refused	Service unreachable	Connection Issues

---

Server Won't Start

Symptom: Server exits immediately

Check configuration syntax:

# Validate YAML syntax
python3 -c "import yaml; yaml.safe_load(open('platform.yaml'))"

# Or with yq
yq eval platform.yaml

Check for missing environment variables:

# The server logs which variables are missing
mcp-data-platform --config platform.yaml 2>&1 | grep -i "missing\|undefined\|required"

Common configuration errors:

WRONG: missing quotes around a URL with special characters (may be parsed incorrectly):

toolkits:
  graphql:
    enabled: true
    instances:
      metadata:
        endpoint_url: https://metadata.example.com:8080/graphql?tenant=a&env=b

CORRECT: quote URLs:

toolkits:
  graphql:
    enabled: true
    instances:
      metadata:
        endpoint_url: "https://metadata.example.com:8080/graphql?tenant=a&env=b"

A configuration that still names DataHub starts, logs the DataHub settings it found once as deprecated_key, and imports that DataHub into the platform catalog after the first catalog scan; see Moving from DataHub.

WRONG: environment variable without braces (won't expand):

auth:
  api_keys:
    keys:
      - key: $API_KEY

CORRECT: use ${} syntax:

auth:
  api_keys:
    keys:
      - key: ${API_KEY}

Symptom: Server hangs on startup

Likely cause: Cannot connect to a required service (Trino, PostgreSQL).

Debug steps:

# Check Trino connectivity
curl -v https://trino.example.com:443/v1/info

# Check PostgreSQL connectivity
psql $DATABASE_URL -c "SELECT 1"

Symptom: Port already in use

# Find what's using the port
lsof -i :8080

# Kill the process if needed
kill -9 <PID>

# Or use a different port
mcp-data-platform --transport http --address :8081

---

Connection Issues

Cannot connect to Trino

Symptom: trino_query returns connection errors.

Debug output example:

Error: failed to connect to Trino: dial tcp: lookup trino.example.com: no such host

Step 1: Verify network connectivity

# DNS resolution
nslookup trino.example.com

# TCP connectivity
nc -zv trino.example.com 443

# HTTP connectivity
curl -v https://trino.example.com:443/v1/info

Step 2: Check SSL configuration

toolkits:
  trino:
    enabled: true
    instances:
      primary:
        host: trino.example.com
        port: 443
        ssl: true
        ssl_verify: true  # Try false for self-signed certs

        # For custom CA certificates
        ssl_ca_file: /path/to/ca.crt
    default: primary

Step 3: Verify credentials

# Test with Trino CLI
trino --server https://trino.example.com:443 \
  --user $TRINO_USER \
  --password \
  --execute "SELECT 1"

Cannot connect to S3

Symptom: s3_list returns credential errors.

Debug output example:

Error: operation error S3: ListBuckets, https response error StatusCode: 403,
RequestID: ABC123, api error AccessDenied: Access Denied

Step 1: Verify credentials

# Test with AWS CLI
AWS_ACCESS_KEY_ID=$KEY AWS_SECRET_ACCESS_KEY=$SECRET \
  aws s3 ls --region us-east-1

Step 2: Check endpoint configuration for MinIO/custom S3

toolkits:
  s3:
    enabled: true
    instances:
      minio:
        endpoint: "http://minio.local:9000"  # Include protocol
        use_path_style: true                  # Required for MinIO
        disable_ssl: true                     # For non-TLS endpoints
        region: "us-east-1"                   # Still required
    default: minio

Step 3: Verify bucket permissions

# Check bucket policy
aws s3api get-bucket-policy --bucket your-bucket

# Check IAM permissions
aws sts get-caller-identity

---

Authentication Issues

OIDC token rejected (401)

Debug output example:

Error: OIDC validation failed: token signature verification failed

Step 1: Decode and inspect the token

# Decode JWT payload
echo $TOKEN | cut -d. -f2 | base64 -d 2>/dev/null | jq

# Check key claims
echo $TOKEN | cut -d. -f2 | base64 -d 2>/dev/null | jq '{iss, aud, exp, sub}'

Step 2: Verify issuer URL matches exactly

auth:
  oidc:
    # Must match token's "iss" claim EXACTLY (including trailing slash if present)
    issuer: "https://auth.example.com/realms/myrealm"

    # Common mistake: missing /realms/name for Keycloak
    # issuer: "https://auth.example.com"  # WRONG

Step 3: Check token expiration and server clock

The platform validates exp/nbf with a fixed 30-second clock-skew tolerance; this is not currently configurable via YAML. If the server clock drifts by more than 30 seconds a valid token can be rejected, so keep the server clock synchronized (NTP) and confirm the token has not expired:

# Check server time
date -u

# Check token expiration
echo $TOKEN | cut -d. -f2 | base64 -d 2>/dev/null | jq '.exp | todate'

Step 4: Verify audience claim

auth:
  oidc:
    # Must match one of the token's "aud" claims
    audience: "mcp-data-platform"

API key rejected (401)

Debug output example:

Error: API key validation failed: key not found

Step 1: Check for invisible characters

# Print key with visible whitespace
echo "Key: [$API_KEY_ADMIN]" | cat -A

# Remove any trailing whitespace
export API_KEY_ADMIN=$(echo "$API_KEY_ADMIN" | tr -d '[:space:]')

Step 2: Verify configuration matches

auth:
  api_keys:
    enabled: true  # Must be enabled
    keys:
      - key: ${API_KEY_ADMIN}  # Variable name must match exactly
        name: "admin"
        roles: ["admin"]

Step 3: Test the key directly

# For HTTP transport
curl -H "Authorization: Bearer $API_KEY_ADMIN" \
  http://localhost:8080/health

OAuth flow fails

Debug output example:

Error: OAuth callback failed: state mismatch

Step 1: Check redirect URI configuration

oauth:
  clients:
    - id: "claude-desktop"
      redirect_uris:
        - "http://localhost"      # Claude Desktop uses these
        - "http://127.0.0.1"
        # Must match exactly what the client sends

Step 2: Verify upstream IdP configuration

oauth:
  upstream:
    issuer: "https://keycloak.example.com/realms/your-realm"
    client_id: "mcp-data-platform"
    client_secret: ${KEYCLOAK_CLIENT_SECRET}

    # This must be registered as a valid redirect URI in Keycloak
    redirect_uri: "https://mcp.example.com/oauth/callback"

Step 3: Check browser console for CORS errors

If using a web-based client, check for CORS issues in the browser console.

---

Persona Issues

User gets wrong persona

Debug steps:

Step 1: Check what roles are in the token

# Decode token and show roles
echo $TOKEN | cut -d. -f2 | base64 -d 2>/dev/null | jq '.realm_access.roles'

Step 2: Verify role_claim_path matches token structure

auth:
  oidc:
    # For Keycloak with realm roles
    role_claim_path: "realm_access.roles"

    # For Keycloak with client roles
    # role_claim_path: "resource_access.mcp-data-platform.roles"

    # For Auth0
    # role_claim_path: "https://example.com/roles"

Step 3: Check role prefix filtering

auth:
  oidc:
    role_prefix: "dp_"  # Only roles starting with "dp_" are used

Token roles: ["dp_analyst", "user", "admin"] Filtered roles: ["analyst"] (prefix stripped)

Step 4: Verify persona role matching

personas:
  analyst:
    roles: ["analyst"]  # Must match filtered role exactly

Tool denied unexpectedly (403)

Debug output example:

Error: tool access denied: trino_query not allowed for persona 'analyst'

Step 1: Check allow/deny patterns

personas:
  analyst:
    tools:
      # Deny patterns are checked FIRST
      deny: ["trino_query"]  # This blocks trino_query
      allow: ["trino_*"]     # This would allow it, but deny wins

Step 2: Verify wildcard pattern matching

Pattern	Matches	Doesn't Match
trino_*	trino_query, trino_execute, trino_browse	search
s3_*	s3_list, s3_object	fetch
*	Everything	Nothing

Step 3: List available tools

There is no CLI flag to list registered tools. Connect an MCP client (e.g. MCP Inspector) to the running server and send a tools/list request to see the tools your persona currently has access to.

---

Enrichment Issues

No semantic context in Trino results

Debug steps:

Step 1: Verify enrichment is enabled

enrichment:
  trino_semantic_enrichment: true  # Must be true

Step 2: Check the semantic layer is on

With a database, the platform catalog is the semantic provider. A semantic.provider: noop turns it off.

Step 3: Check the catalog holds the table

Check that the connection has been scanned (Admin > Catalog > Scans) and that the catalog settings do not exclude the table, then read the dataset:

fetch mcp:dataset:(trino,<connection>,<catalog>.<schema>.<table>)

A not-found means the catalog has no dataset for the table. A dataset with no curation enriches with little.

Enrichment errors in logs

Debug output example:

WARN  enrichment failed for table orders: dataset not found

This is expected for tables the catalog does not hold, such as one created since the last scan or one the catalog settings exclude. The platform returns the original result without enrichment, so these warnings are harmless. They are emitted at WARN level; raise the process log level (LOG_LEVEL) if they are noisy in your environment.

Column enrichment shows all columns instead of referenced ones

Column context filtering limits enrichment to columns mentioned in the SQL query. If you see all columns:

enrichment:
  column_context_filtering: true  # Default: true — set to false to include all columns

Note: trino_describe_table always returns all columns. Filtering only applies to trino_query results. PII/sensitive columns are always included regardless of this setting.

---

Performance Issues

Slow queries

Step 1: Check query timeout configuration

toolkits:
  trino:
    enabled: true
    instances:
      primary:
        timeout: 300s  # Increase if needed
    default: primary

Step 2: Verify row limits

toolkits:
  trino:
    enabled: true
    instances:
      primary:
        default_limit: 1000   # Default rows returned
        max_limit: 10000      # Maximum allowed
    default: primary

Step 3: Profile the query in Trino

EXPLAIN ANALYZE SELECT * FROM your_table LIMIT 1000;

Step 4: Check network latency

# Time a simple query
time curl -H "Authorization: Bearer $TOKEN" \
  "https://trino.example.com:443/v1/statement" \
  -d "SELECT 1"

Slow enrichment

Step 1: Check the database

Enrichment reads the catalog from the platform's PostgreSQL. Slow enrichment is usually a slow database; check db_client_operation_duration_seconds and the slow-statement warnings.

Step 2: Temporarily disable enrichment to isolate

enrichment:
  trino_semantic_enrichment: false  # Disable to test raw performance

High memory usage

Likely causes:

1. Large query results being held in memory
2. Too many cached entries
3. Connection pool too large

Solutions:

toolkits:
  trino:
    enabled: true
    instances:
      primary:
        max_limit: 10000  # Reduce maximum result size
    default: primary

semantic:
  cache:
    ttl: 1m             # Shorter TTL, bounds how long stale entries persist

database:
  max_open_conns: 10    # Reduce connection pool

---

Debugging Guide

Enable verbose logging

# Set log level
export LOG_LEVEL=debug

# Run with verbose output
mcp-data-platform --config platform.yaml 2>&1 | tee debug.log

Log format

2024-01-15T10:30:45.123Z INFO  server started address=:8080 transport=http
2024-01-15T10:30:46.456Z DEBUG auth middleware: validating token
2024-01-15T10:30:46.457Z DEBUG persona middleware: resolved persona=analyst
2024-01-15T10:30:46.458Z INFO  tool call tool=trino_query [email protected] persona=analyst
2024-01-15T10:30:46.789Z DEBUG enrichment: fetching semantic context table=orders
2024-01-15T10:30:47.012Z INFO  tool call complete tool=trino_query duration=554ms

Request tracing

Each request has a unique ID for correlation:

X-Request-ID: a1b2c3d4-e5f6-7890-abcd-ef1234567890

Find all logs for a request:

grep "a1b2c3d4-e5f6-7890-abcd-ef1234567890" debug.log

Audit log queries

If audit logging is enabled, query the database:

-- Recent tool calls
SELECT * FROM audit_logs
ORDER BY timestamp DESC
LIMIT 100;

-- Failed requests
SELECT * FROM audit_logs
WHERE success = false
ORDER BY timestamp DESC;

-- Requests by user
SELECT tool_name, COUNT(*) as count
FROM audit_logs
WHERE user_id = '[email protected]'
GROUP BY tool_name;

---

Common Error Codes

Code	Meaning	Solution
AUTH_ERROR	Authentication failed	Check credentials, token expiration
AUTHZ_ERROR	Authorization failed	Check persona tool rules
TOOLKIT_ERROR	Toolkit operation failed	Check service connectivity
PROVIDER_ERROR	Provider operation failed	Check the Trino and database configuration
CONFIG_ERROR	Configuration invalid	Validate YAML, check env vars
TIMEOUT_ERROR	Operation timed out	Increase timeout, check service
RATE_LIMIT_ERROR	Too many requests	Wait and retry, increase limits

---

Getting Help

1. Search existing issues: GitHub Issues

2. Report a bug: Include:

3. Platform version (mcp-data-platform --version)
4. Configuration (redact secrets)
5. Full error message
6. Steps to reproduce

7. Relevant logs

8. Community support: GitHub Discussions

9. Security issues: Email security@txn2.com (do not open public issues for security vulnerabilities)