Eval scores per skill

Each bar is one agent model with or without the plugin, sorted by mean score.

With pluginWithout plugin

solve-inverse-problem (pylops plugin)

Mean score over 6 cases, 0–1 scale.

0.00 0.25 0.50 0.75 1.00 Opus 5.5 with: 1.000 1.00 Opus 5.5 With Haiku 5.5 with: 0.960 0.96 Haiku 5.5 With Opus 5.5 without: 0.952 0.95 Opus 5.5 Without Haiku 5.5 without: 0.923 0.92 Haiku 5.5 Without Sonnet 5.5 with: 0.920 0.92 Sonnet 5.5 With Sonnet 5.5 without: 0.857 0.86 Sonnet 5.5 Without Haiku 4.5 with: 0.688 0.69 Haiku 4.5 With Haiku 4.5 without: 0.518 0.52 Haiku 4.5 Without

debug-inversion (pylops plugin)

Mean score over 9 cases, 0–1 scale.

0.00 0.25 0.50 0.75 1.00 Sonnet 5.5 without: 1.000 1.00 Sonnet 5.5 Without Sonnet 5.5 with: 0.986 0.99 Sonnet 5.5 With Opus 5.5 with: 0.981 0.98 Opus 5.5 With Opus 5.5 without: 0.981 0.98 Opus 5.5 Without Haiku 5.5 with: 0.962 0.96 Haiku 5.5 With Haiku 5.5 without: 0.930 0.93 Haiku 5.5 Without Haiku 4.5 with: 0.754 0.75 Haiku 4.5 With Haiku 4.5 without: 0.689 0.69 Haiku 4.5 Without

tune-regularization (pylops plugin)

Mean score over 7 cases, 0–1 scale.

0.00 0.25 0.50 0.75 1.00 Sonnet 5.5 with: 0.987 0.99 Sonnet 5.5 With Opus 5.5 without: 0.983 0.98 Opus 5.5 Without Opus 5.5 with: 0.977 0.98 Opus 5.5 With Haiku 5.5 with: 0.949 0.95 Haiku 5.5 With Sonnet 5.5 without: 0.949 0.95 Sonnet 5.5 Without Haiku 5.5 without: 0.883 0.88 Haiku 5.5 Without

solve-nonsmooth-problem (pyproximal plugin)

Mean score over 6 cases, 0–1 scale.

0.00 0.25 0.50 0.75 1.00 Opus 5.5 with: 0.980 0.98 Opus 5.5 With Sonnet 5.5 with: 0.967 0.97 Sonnet 5.5 With Haiku 5.5 with: 0.920 0.92 Haiku 5.5 With Opus 5.5 without: 0.903 0.90 Opus 5.5 Without Sonnet 5.5 without: 0.875 0.88 Sonnet 5.5 Without Haiku 4.5 with: 0.833 0.83 Haiku 4.5 With Haiku 5.5 without: 0.658 0.66 Haiku 5.5 Without Haiku 4.5 without: 0.603 0.60 Haiku 4.5 Without