Skip to content
Test Manual
MCP E2E Test

MCP E2E Test

This chapter describes the ShardingSphere-MCP Functionality, Conformance, and LLM end-to-end test suites.

Scope

The MCP E2E workflow contains three independent test suites:

  • MCP Functionality E2E validates distribution startup and configuration, HTTP and STDIO runtimes backed by real MySQL, PostgreSQL, and Proxy processes, cross-process discovery and execution of tools, resources, prompts, and completions, and the Encrypt, Mask, Broadcast, Readwrite-Splitting, Shadow, and Sharding workflows.
  • MCP Conformance E2E uses the official runner to validate MCP protocol scenarios applicable to the published server capabilities.
  • MCP LLM E2E validates that a real model can use MCP over HTTP for read-only queries, metadata discovery, side-effect previews, and invalid-resource recovery.

The StreamableHttpMCPServerIT in mcp/bootstrap covers HTTP protocol, session, and security boundaries that require neither Docker nor an external service.

Feature template acceptance

When an MCP feature is used as a workflow template, E2E tests should cover protocol discoverability, model usability, and negative contracts. For the Encrypt workflow, template-level acceptance should include at least:

  • Completion returns feature-available algorithms or candidate values.
  • Plan output contains only DistSQL artifacts supported by the current feature.
  • Plan output does not contain unsupported physical DDL, index, migration, backfill, or cleanup artifacts.
  • resources_to_read points to feature-owned algorithm, rule, or configuration resources, not physical metadata resources outside the feature.
  • Descriptor output schema does not expose output fields unsupported by the current feature.
  • Plan, workflow resource, preview, apply, validate, recovery, and trace-visible outputs do not leak sensitive properties.
  • Custom algorithms or algorithms with unknown capabilities are marked as unconfirmed instead of being treated as known-capability algorithms.
  • Drop scenarios validate rule removal semantics and do not use physical cleanup as a success condition.
  • Encrypt, Mask, and Sharding ALTER expansion, physical DDL, migration, and backfill remain excluded commercial-edition capabilities.
  • Apply must be preceded by preview and must validate user-approved steps.

Test reuse should stay in local helpers under test/e2e/mcp; do not add a test jar or a cross-module test-support module for template acceptance.

Local preparation

Build and install the MCP E2E dependencies and distribution:

./mvnw -pl test/e2e/mcp,distribution/mcp -am install -DskipTests -DskipITs -Dspotless.skip=true -B -ntp

Build the local distribution image:

docker build --platform "$(docker version --format '{{.Server.Os}}/{{.Server.Arch}}')" -f distribution/mcp/Dockerfile -t apache/shardingsphere-mcp-e2e:local distribution/mcp/target

LLM Runtime

Install Docker Model Runner following the official Docker instructions. For Docker Desktop, enable host TCP access:

docker desktop enable model-runner --tcp 12434

For Docker Engine, install and start the CPU runner:

MODEL_RUNNER_CONTROLLER_VERSION=v1.2.8@sha256:5bdc2bac71c1b70453f7dec5b527031bc6d8d99b8d1b2a10a8bcca073ae257a3 \
MODEL_RUNNER_CONTROLLER_VARIANT=cpu docker model install-runner --gpu none

Pull, configure, and start the model:

docker model pull ai/qwen3.5:9b-q4_K_XL
docker model configure --context-size 8192 ai/qwen3.5:9b-q4_K_XL -- \
  --jinja --reasoning-budget 0 --reasoning-format deepseek --chat-template-kwargs '{"enable_thinking":false}' \
  --parallel 1 -b 256 -ub 128 --cache-ram 0 --no-cache-prompt
docker model run --detach ai/qwen3.5:9b-q4_K_XL

Run MCP Functionality E2E

MCP E2E runtime configuration is centralized in test/e2e/mcp/src/test/resources/env/e2e-env.properties. For local runs, edit that file or override the same keys with -D system properties.

./mvnw -pl test/e2e/mcp verify -Pe2e.mcp.functionality -De2e.run.type=DOCKER

Run MCP HTTP IT

This test starts the real HTTP server without connecting to Docker, a database, or a model:

./mvnw -pl mcp/bootstrap verify

Run MCP LLM E2E

./mvnw -pl test/e2e/mcp verify -Pe2e.mcp.llm -De2e.run.type=DOCKER

LLMHttpE2EIT covers five autonomous HTTP scenarios: read-only query, metadata discovery, Mask rule planning, side-effect preview, and invalid-resource recovery. Each scenario uses the live tools/list response and preserves the model response, structured MCP response, interaction trace, and assertion report. Missing Docker, model, database, or MCP infrastructure fails the selected llm-e2e lane instead of converting the failure into a skipped case.

MCP Conformance E2E

The CI conformance lane pins modelcontextprotocol/conformance to commit 21a9a2febd7100d7c17ac1021ee7f2ed9f66a1e0, passes protocol version 2025-11-25, and executes only the applicable generic server scenarios declared in the workflow. Upstream calls tied to fixed test_* tools or resources, unadvertised optional capabilities, and scenarios outside the fixed HTTP transport surface do not apply to this project. Product capabilities remain covered by deterministic E2E tests, and no production test hooks are added for upstream fixtures. The packaged server runs with its loopback HTTP configuration so the DNS rebinding scenario validates the loopback Origin policy rather than the separate Docker remote-binding policy.

External Debug

For local debugging only, connect to an already running OpenAI-compatible endpoint:

./mvnw -pl test/e2e/mcp verify -Pe2e.mcp.llm -De2e.run.type=DOCKER -Dit.test=LLMHttpE2EIT -Dmcp.llm.runtime-mode=external-debug -Dmcp.llm.base-url=http://127.0.0.1:12434/engines/v1

External debug endpoints are only for local debugging; formal LLMHttpE2EIT scenarios require Docker and a prepared Docker Model Runner.

Artifacts

MCP LLM E2E artifacts are written under:

test/e2e/mcp/target/llm-e2e/

Each scenario records the question, actual answer, raw model response, MCP interaction trace, live tool definitions, and assertion report. Artifact writing redacts secret-shaped values, and the test fails if an unredacted secret pattern or the known model API key is present.

GitHub Actions entry points:

  • .github/workflows/e2e-mcp.yml

This workflow is the shared entry point for all three MCP E2E suites. If a very large PR misses a path-filter match, use workflow_dispatch to add manual evidence.