SkillForge Agent Skill

SkillForge v6

A skill creator that proves its skills work.

SkillForge routes any skill-related request (use, improve, create, compose), creates new skills through an evidence-driven pipeline, and maintains the health of your whole skill ecosystem. Its core principle: skill quality is a property of behavior, not documents - a skill is done when a fresh agent demonstrably does better with it than without it.

What v6 changed (and why)

v6 is a ground-up rework following a deep external audit (see SKILLFORGE_AUDIT.md on the repo, not shipped with the skill). The headline shifts:

v5	v6
4-agent "unanimous synthesis panel" reads the skill	Skills are executed: baseline (RED) runs before writing, with-skill (GREEN) runs after, behavioral delta is the gate
Self-scored "timelessness >= 7" approval	Falsifiable checks moved to lint (validate_skill.py); one adversarial reviewer refutes what lint can't catch
5,049-word SKILL.md with <details> "progressive disclosure"	1,158-word SKILL.md; depth lives in references/ loaded on demand
Descriptions = "what this skill does"	Descriptions = trigger conditions only (workflow summaries make agents skip the body)
launchd background advisor that analyzed / and queued into a file nothing read	Advisor delivered through Claude Code hooks (SessionStart + UserPromptSubmit), caps enforced, opt-in
Personal-directory scanning on by default, hardcoded GitHub handles	Personal Context strictly opt-in with recorded consent; no shipped defaults
Hand-rolled YAML parser that failed SkillForge's own SKILL.md	One shared typed parser (scripts/frontmatter.py), 100+ unit tests, and a regression test that SkillForge validates itself
Index missed the Claude Code plugin cache entirely	Cross-runtime discovery: personal, Codex, Claude Code plugin cache; deduped; auto-refresh

The pipeline

Phase 0  TRIAGE      index + word-boundary matching -> USE | IMPROVE | CREATE | COMPOSE | CLARIFY
Phase 0b RED GATE    fresh subagent attempts the task WITHOUT the skill; no failure = no skill
Phase 1  ANALYSIS    load-bearing lenses (Inversion, Pareto, Root Cause), failure-form matching
Phase 2  SPEC        tiered (minimal default / full for infrastructure), decisions + WHY
Phase 3  GENERATE    fresh-context subagent receives ONLY the spec + baseline failures
Phase 4  GREEN GATE  with-skill runs must clear the recorded baseline failures; trigger tests
Phase 5  REVIEW      lint (validate_skill.py) + ONE adversarial reviewer charged to refute
Phase 6  SHIP        with evals/ - a per-skill regression suite runnable forever after

What no other skill creator has

• Dedup before create (Phase 0): an index of every skill across runtimes answers "should this exist?" first.
• Skills ship with their tests: evals/ (trigger queries + behavioral scenarios) + run_skill_evals.py = regression testing for skills.
• Ecosystem doctor: skillforge_doctor.py finds trigger collisions between skills, duplicates, stale file references, budget violations, and pinned models across your entire roster.
• Cross-runtime compile: author once, compile_skill.py --target claude|codex|agentskills.
• Friction mining (opt-in): mine_skill_friction.py --consent finds skill gaps in your own local session history.
• Proactive advisor (opt-in): evidence-backed skill suggestions delivered via hooks, never auto-invoked.

Install (Claude Code)

git clone https://github.com/tripleyak/SkillForge.git /tmp/skillforge
cp -r /tmp/skillforge ~/.claude/skills/skillforge
cd ~/.claude/skills/skillforge && rm -rf README.md LICENSE CONTEXT.md docs .git .gitignore .skillignore index.html assets/images scripts/tests SKILLFORGE_AUDIT.md
cp /tmp/skillforge/commands/skillforge.md ~/.claude/commands/skillforge.md   # optional /skillforge command

Optional advisor + hooks (interactive, everything opt-in):

python3 ~/.claude/skills/skillforge/scripts/install_skillforge.py

Requirements: Claude Code (or Codex CLI), Python 3.8+ (stdlib only; PyYAML used if present).

Toolbox

Command	Purpose
python3 scripts/discover_skills.py	Build/refresh the skill index
python3 scripts/triage_skill_request.py "<request>" --json	Route a request
python3 scripts/validate_skill.py <dir>	Full validation + lint
python3 scripts/run_skill_evals.py <dir> [--live]	Run a skill's regression evals
python3 scripts/skillforge_doctor.py	Ecosystem health report
python3 scripts/init_skill.py <name> --path <dir>	Scaffold (includes evals/)
python3 scripts/compile_skill.py <dir> --target <t>	Cross-runtime compile
python3 scripts/package_skill.py <dir> ./dist	Package as .skill
python3 scripts/mine_skill_friction.py --consent	Mine local transcripts for skill gaps

CI: copy assets/templates/github-workflow-skill-ci.yml into .github/workflows/ of any skill repo.

Releases

Version history and detailed release notes live on the releases page. v6.0.0 is a ground-up rework; the audit that drove it ships in-repo as SKILLFORGE_AUDIT.md.

License

MIT - see LICENSE