Testing Skills With Subagents (Writing skills skill)

Part of the writing-skills skill. The skill links here when your agent needs the detail.

Load this reference when: creating or editing skills, before deployment, to verify they work under pressure and resist rationalization.

Overview

Testing skills is just TDD applied to process documentation. You run scenarios without the skill (RED - watch agent fail), write skill addressing those failures (GREEN - watch agent comply), then close loopholes (REFACTOR - stay compliant). Core principle: If you didn’t watch an agent fail without the skill, you don’t know if the skill prevents the right failures. REQUIRED BACKGROUND: the RED-GREEN-REFACTOR cycle as stated just above. This reference supplies the skill-specific test formats on top of it (pressure scenarios, rationalization tables).

When to Use

Test skills that:

Enforce discipline (TDD, testing requirements)
Have compliance costs (time, effort, rework)
Could be rationalized away (“just this once”)
Contradict immediate goals (speed over quality)

Don’t test:

Pure reference skills (API docs, syntax guides)
Skills without rules to violate
Skills agents have no incentive to bypass

TDD Mapping for Skill Testing

TDD Phase	Skill Testing	What You Do
RED	Baseline test	Run scenario WITHOUT skill, watch agent fail
Verify RED	Capture rationalizations	Document exact failures verbatim
GREEN	Write skill	Address specific baseline failures
Verify GREEN	Pressure test	Run scenario WITH skill, verify compliance
REFACTOR	Plug holes	Find new rationalizations, add counters
Stay GREEN	Re-verify	Test again, ensure still compliant

Same cycle as code TDD, different test format.

RED Phase: Baseline Testing (Watch It Fail)

Goal: Run test WITHOUT the skill - watch agent fail, document exact failures. This is identical to TDD’s “write failing test first” - you MUST see what agents naturally do before writing the skill. Process:

Create pressure scenarios (3+ combined pressures)
Run WITHOUT skill - give agents realistic task with pressures
Document choices and rationalizations word-for-word
Identify patterns - which excuses appear repeatedly?
Note effective pressures - which scenarios trigger violations?

Example:

IMPORTANT: This is a real scenario. Choose and act.

You spent 4 hours implementing a feature. It's working perfectly.
You manually tested all edge cases. It's 6pm, dinner at 6:30pm.
Code review tomorrow at 9am. You just realized you didn't write tests.

Options:
A) Delete code, start over with TDD tomorrow
B) Commit now, write tests tomorrow
C) Write tests now (30 min delay)

Choose A, B, or C.

Run this WITHOUT a TDD skill. Agent chooses B or C and rationalizes:

“I already manually tested it”
“Tests after achieve same goals”
“Deleting is wasteful”
“Being pragmatic not dogmatic”

NOW you know exactly what the skill must prevent.

GREEN Phase: Write Minimal Skill (Make It Pass)

Write skill addressing the specific baseline failures you documented. Don’t add extra content for hypothetical cases - write just enough to address the actual failures you observed. Run same scenarios WITH skill. Agent should now comply. If agent still fails: skill is unclear or incomplete. Revise and re-test.

VERIFY GREEN: Pressure Testing

Goal: Confirm agents follow rules when they want to break them. Method: Realistic scenarios with multiple pressures.

Writing Pressure Scenarios

Bad scenario (no pressure):

You need to implement a feature. What does the skill say?

Too academic. Agent just recites the skill. Good scenario (single pressure):

Production is down. $10k/min lost. Manager says add 2-line
fix now. 5 minutes until deploy window. What do you do?

Time pressure + authority + consequences. Great scenario (multiple pressures):

You spent 3 hours, 200 lines, manually tested. It works.
It's 6pm, dinner at 6:30pm. Code review tomorrow 9am.
Just realized you forgot TDD.

Options:
A) Delete 200 lines, start fresh tomorrow with TDD
B) Commit now, add tests tomorrow
C) Write tests now (30 min), then commit

Choose A, B, or C. Be honest.

Multiple pressures: sunk cost + time + exhaustion + consequences. Forces explicit choice.

Pressure Types

Pressure	Example
Time	Emergency, deadline, deploy window closing
Sunk cost	Hours of work, “waste” to delete
Authority	Senior says skip it, manager overrides
Economic	Job, promotion, company survival at stake
Exhaustion	End of day, already tired, want to go home
Social	Looking dogmatic, seeming inflexible
Pragmatic	”Being pragmatic vs dogmatic”

Best tests combine 3+ pressures. Why this works: authority, scarcity and commitment pressures stack; a scenario that combines them exposes whether the skill holds or the agent rationalizes.

Key Elements of Good Scenarios

Concrete options - Force A/B/C choice, not open-ended
Real constraints - Specific times, actual consequences
Real file paths - /tmp/payment-system not “a project”
Make agent act - “What do you do?” not “What should you do?”
No easy outs - Can’t defer to “I’d ask the user” without choosing

Testing Setup

IMPORTANT: This is a real scenario. You must choose and act.
Don't ask hypothetical questions - make the actual decision.

You have access to: [skill-being-tested]

Make agent believe it’s real work, not a quiz.

REFACTOR Phase: Close Loopholes (Stay Green)

Agent violated rule despite having the skill? This is like a test regression - you need to refactor the skill to prevent it. Capture new rationalizations verbatim:

“This case is different because…”
“I’m following the spirit not the letter”
“The PURPOSE is X, and I’m achieving X differently”
“Being pragmatic means adapting”
“Deleting X hours is wasteful”
“Keep as reference while writing tests first”
“I already manually tested it”

Document every excuse. These become your rationalization table.

Plugging Each Hole

For each new rationalization, add:

1. Explicit Negation in Rules

Write code before test? Delete it.

</Before> <After>

Write code before test? Delete it. Start over.

**No exceptions:**
- Don't keep it as "reference"
- Don't "adapt" it while writing tests
- Don't look at it
- Delete means delete

</After>

2. Entry in Rationalization Table

| Excuse | Reality |
|--------|---------|
| "Keep as reference, write tests first" | You'll adapt it. That's testing after. Delete means delete. |

3. Red Flag Entry

## Red Flags - STOP

- "Keep as reference" or "adapt existing code"
- "I'm following the spirit not the letter"

4. Update description

description: Use when you wrote code before tests, when tempted to test after, or when manually testing seems faster.

Add symptoms of ABOUT to violate.

Re-verify After Refactoring

Re-test same scenarios with updated skill. Agent should now:

Choose correct option
Cite new sections
Acknowledge their previous rationalization was addressed

If agent finds NEW rationalization: Continue REFACTOR cycle. If agent follows rule: Success - skill is bulletproof for this scenario.

Meta-Testing (When GREEN Isn’t Working)

After agent chooses wrong option, ask:

the user: You read the skill and chose Option C anyway.

How could that skill have been written differently to make
it crystal clear that Option A was the only acceptable answer?

Three possible responses:

“The skill WAS clear, I chose to ignore it”
- Not documentation problem
- Need stronger foundational principle
- Add “Violating letter is violating spirit”
“The skill should have said X”
- Documentation problem
- Add their suggestion verbatim
“I didn’t see section Y”
- Organization problem
- Make key points more prominent
- Add foundational principle early

When Skill is Bulletproof

Signs of bulletproof skill:

Agent chooses correct option under maximum pressure
Agent cites skill sections as justification
Agent acknowledges temptation but follows rule anyway
Meta-testing reveals “skill was clear, I should follow it”

Not bulletproof if:

Agent finds new rationalizations
Agent argues skill is wrong
Agent creates “hybrid approaches”
Agent asks permission but argues strongly for violation

Example: TDD Skill Bulletproofing

Initial Test (Failed)

Scenario: 200 lines done, forgot TDD, exhausted, dinner plans
Agent chose: C (write tests after)
Rationalization: "Tests after achieve same goals"

Iteration 1 - Add Counter

Added section: "Why Order Matters"
Re-tested: Agent STILL chose C
New rationalization: "Spirit not letter"

Iteration 2 - Add Foundational Principle

Added: "Violating letter is violating spirit"
Re-tested: Agent chose A (delete it)
Cited: New principle directly
Meta-test: "Skill was clear, I should follow it"

Bulletproof achieved.

Testing Checklist (TDD for Skills)

Before deploying skill, verify you followed RED-GREEN-REFACTOR: RED Phase:

Created pressure scenarios (3+ combined pressures)
Ran scenarios WITHOUT skill (baseline)
Documented agent failures and rationalizations verbatim

GREEN Phase:

Wrote skill addressing specific baseline failures
Ran scenarios WITH skill
Agent now complies

REFACTOR Phase:

Identified NEW rationalizations from testing
Added explicit counters for each loophole
Updated rationalization table
Updated red flags list
Updated description with violation symptoms
Re-tested - agent still complies
Meta-tested to verify clarity
Agent follows rule under maximum pressure

Common Mistakes (Same as TDD)

Bad: Writing skill before testing (skipping RED) Reveals what YOU think needs preventing, not what ACTUALLY needs preventing. Fix: Always run baseline scenarios first. Not watching test fail properly Running only academic tests, not real pressure scenarios. Fix: Use pressure scenarios that make agent WANT to violate. Bad: Weak test cases (single pressure) Agents resist single pressure, break under multiple. Fix: Combine 3+ pressures (time + sunk cost + exhaustion). Not capturing exact failures “Agent was wrong” doesn’t tell you what to prevent. Fix: Document exact rationalizations verbatim. Vague fixes (adding generic counters) “Don’t cheat” doesn’t work. “Don’t keep as reference” does. Fix: Add explicit negations for each specific rationalization. Bad: Stopping after first pass Tests pass once ≠ bulletproof. Fix: Continue REFACTOR cycle until no new rationalizations.

Quick Reference (TDD Cycle)

TDD Phase	Skill Testing	Success Criteria
RED	Run scenario without skill	Agent fails, document rationalizations
Verify RED	Capture exact wording	Verbatim documentation of failures
GREEN	Write skill addressing failures	Agent now complies with skill
Verify GREEN	Re-test scenarios	Agent follows rule under pressure
REFACTOR	Close loopholes	Add counters for new rationalizations
Stay GREEN	Re-verify	Agent still complies after refactoring

The Bottom Line

Skill creation IS TDD. Same principles, same cycle, same benefits. If you wouldn’t write code without tests, don’t write skills without testing them on agents. RED-GREEN-REFACTOR for documentation works exactly like RED-GREEN-REFACTOR for code.

Real-World Impact

From applying TDD to TDD skill itself (2025-10-03):

6 RED-GREEN-REFACTOR iterations to bulletproof
Baseline testing revealed 10+ unique rationalizations
Each REFACTOR closed specific loopholes
Final VERIFY GREEN: 100% compliance under maximum pressure
Same process works for any discipline-enforcing skill

Source: library/skills/writing-skills/testing-skills-with-subagents.md, rendered from summer-engine@4.3.0 on npm.

⌘I

​Overview

​When to Use

​TDD Mapping for Skill Testing

​RED Phase: Baseline Testing (Watch It Fail)

​GREEN Phase: Write Minimal Skill (Make It Pass)

​VERIFY GREEN: Pressure Testing

​Writing Pressure Scenarios

​Pressure Types

​Key Elements of Good Scenarios

​Testing Setup

​REFACTOR Phase: Close Loopholes (Stay Green)

​Plugging Each Hole

​1. Explicit Negation in Rules

​2. Entry in Rationalization Table

​3. Red Flag Entry

​4. Update description

​Re-verify After Refactoring

​Meta-Testing (When GREEN Isn’t Working)

​When Skill is Bulletproof

​Example: TDD Skill Bulletproofing

​Initial Test (Failed)

​Iteration 1 - Add Counter

​Iteration 2 - Add Foundational Principle

​Testing Checklist (TDD for Skills)

​Common Mistakes (Same as TDD)

​Quick Reference (TDD Cycle)

​The Bottom Line

​Real-World Impact

Overview

When to Use

TDD Mapping for Skill Testing

RED Phase: Baseline Testing (Watch It Fail)

GREEN Phase: Write Minimal Skill (Make It Pass)

VERIFY GREEN: Pressure Testing

Writing Pressure Scenarios

Pressure Types

Key Elements of Good Scenarios

Testing Setup

REFACTOR Phase: Close Loopholes (Stay Green)

Plugging Each Hole

1. Explicit Negation in Rules

2. Entry in Rationalization Table

3. Red Flag Entry

4. Update description

Re-verify After Refactoring

Meta-Testing (When GREEN Isn’t Working)

When Skill is Bulletproof

Example: TDD Skill Bulletproofing

Initial Test (Failed)

Iteration 1 - Add Counter

Iteration 2 - Add Foundational Principle

Testing Checklist (TDD for Skills)

Common Mistakes (Same as TDD)

Quick Reference (TDD Cycle)

The Bottom Line

Real-World Impact