0
0
Fork 0
Codesys-MCP-SP21-plus/tests
Karstein Kvistad 0fc23ef8cd docs(TEST_OVERVIEW): real bench numbers + mark broken tools as fixed
- Headline result table at the top: persistent is 15-24x faster than
  headless across all CODESYS-roundtripping tools. Numbers are from a
  fresh bench-results.json run on MCPTest2.
- Updated mode primer with measured first-call vs warm-call costs:
  persistent first launch ~14.6s, subsequent calls ~1.5s; headless
  first cold call ~58s, warm calls ~22s.
- Inventory table: replaced "typical" estimates with measured numbers
  for the 9 tools the bench covers.
- Deep-dive section: each broken tool now marked "fixed in 2607063"
  with concise summary of the fix that landed (instead of "proposed
  fix"). Removed the long pre-fix code blocks since they're in the
  commit history now.
- Status legend: create_folder, compile_project, get_compile_messages,
  connect_to_device, open_project all flipped from broken to fixed
  (with caveats: some need PLC or runtime verification).
- Footnote about set_pou_code bench-harness failure (multi-line code
  through triple-quoted-string interpolation -- not a tool bug).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 16:58:05 +02:00
..
integration Initial release: MCP server for CODESYS with persistent UI instance 2026-02-08 18:59:16 +10:00
unit Initial release: MCP server for CODESYS with persistent UI instance 2026-02-08 18:59:16 +10:00
bench-results.json fix: 4 broken-tool fixes (compile json long, connect_to_device LoginMode, 2026-04-26 16:53:25 +02:00
bench.mjs docs+test: TEST_OVERVIEW + bench harness for headless vs persistent 2026-04-26 16:41:52 +02:00
mock_watcher.py Initial release: MCP server for CODESYS with persistent UI instance 2026-02-08 18:59:16 +10:00
TEST_OVERVIEW.md docs(TEST_OVERVIEW): real bench numbers + mark broken tools as fixed 2026-04-26 16:58:05 +02:00