Skills Library

---
name: validate-claim-with-humans
description: Use when the user wants to test whether a marketing claim, headline, value-prop, or factual statement is believable to real respondents. Triggers on "is this believable", "would people believe", "test this claim", "validate this headline", "fact-check with humans".
---

When the user asks to validate a claim with humans:

1. **Confirm the claim text verbatim.** Quote it back to the user to avoid drift.
2. **Confirm the target audience.** Default to "general" if the user does not specify. Suggest a tighter segment if the claim is industry-specific (e.g. "SaaS founders" for a B2B SaaS claim).
3. **Preview cost first.** Call `ask_humans` with `mode: "claim"`, `text: <claim>`, `audience: <segment>`, `n: 25` (default), `dry_run: true`. Show the user the estimated cost.
4. **Wait for explicit approval** before running without dry_run. Never spend Panel budget without a "yes" or equivalent confirmation.
5. **Launch.** Call `ask_humans` again with `dry_run: false`. Capture the returned `study_id` and `dashboard_url`. Surface the dashboard URL to the user so they can monitor.
6. **Poll `get_results`** every 5 minutes until status is `complete` or `partial`. Stop after 1 hour and tell the user the study is still fielding — do not block indefinitely.
7. **Summarize on completion:**
   - Believability score (single number).
   - 3 most common reasons respondents found it believable.
   - 3 most common reasons respondents did not.
   - 2 verbatim quotes representing the strongest signal on each side.
8. **Recommend a next step.** If believability ≥ 0.7, suggest moving to messaging tests. If < 0.5, suggest rephrasing and re-validating. Between, suggest a follow-up `mode: "message"` study to find a clearer phrasing.

---
name: run-preference-study
description: Use when the user wants to compare 2-5 options (taglines, headlines, designs, value propositions, feature names) and find out which one real respondents prefer. Triggers on "which tagline wins", "A/B test these", "compare these options", "preference test".
---

When the user wants a head-to-head preference test:

1. **List every option as a bullet** and ask the user to confirm. Preference studies allow 2-5 options — refuse politely if outside that range, suggesting they pick a shortlist.
2. **Identify the audience.** Default to "general". For visual/design comparisons, suggest matching the audience to the target user persona.
3. **Decide N.** Default to 30 respondents per study (gives 90% power to detect a 20-point preference gap). Suggest 50+ if the user wants tighter confidence intervals.
4. **Preview cost** with `ask_humans` `mode: "preference"`, `options: [<list>]`, `audience: <segment>`, `n: <N>`, `dry_run: true`.
5. **Wait for explicit cost approval.**
6. **Launch and capture `study_id` + `dashboard_url`.** Surface the dashboard URL.
7. **Poll `get_results`** every 5 minutes until `complete` or `partial`.
8. **Report results:**
   - Winner: option + win-share (e.g. "Option B, 62% preference").
   - Margin: difference between winner and runner-up.
   - Statistical significance flag (`is_significant`, if returned).
   - Top reason each option won (one sentence each, drawn from open-ended responses).
9. **If the winner's margin is < 10 points:** call this out as inconclusive and suggest re-running with N≥75 or refining the options.

---
name: recruit-with-screeners
description: Use when the user wants to recruit a specific subset of Panel respondents — by role, behavior, demographics — before asking the main research question. Triggers on "only ask X", "recruit only Y", "screen for Z", "filter respondents".
---

When the user wants screened recruitment:

1. **Clarify the screening criterion.** What attribute defines a qualified respondent? (Role, software they use, frequency of behavior, etc.)
2. **Draft 1-3 screener questions.** Each must have:
   - A `question` (close-ended, multiple choice).
   - `choices`: 3-5 options including at least one disqualifying option to avoid leading the witness.
   - `pass_logic.include`: the subset of choices that qualify a respondent.
3. **Show the screener to the user for approval before launch.** Bad screeners waste budget — respondents who fail are paid for screening then dismissed.
4. **Warn about cost inflation.** Tight screeners can 3-5x the per-respondent cost. Estimate: if you expect 1-in-K respondents to pass, budget is roughly K× the unscreened price.
5. **Launch via `ask_humans`** with the `screeners` argument populated, plus the main study args. Always start with `dry_run: true` to see the inflated estimate.
6. **On approval, run for real**, then proceed as in `run-preference-study` or `validate-claim-with-humans` for the polling and reporting phase.
7. **In the summary, report the screen-pass rate** — this tells the user how realistic their targeting was and informs future screener design.

---
name: analyze-fielded-study
description: Use when the user references an already-fielded study (by study_id, name, or "the one I just ran") and wants insights pulled from its results. Triggers on "analyze my study", "what came back from X", "pull insights from study Y", "summarize results".
---

When the user wants analysis of an existing study:

1. **Identify the study.** If the user gives a `study_id`, use it. If they give a name, call `list_studies` and disambiguate. If they say "the one I just ran", use the most recently created study where they are the owner.
2. **Check status.** Call `get_results`. If status is not `complete` or `partial`, tell the user it's still fielding and offer to poll.
3. **Pull results.**
4. **Summarize structurally:**
   - Headline finding (one sentence the user could put in a Slack message).
   - The winning option / believability score / message clarity score (whichever applies to the study's mode).
   - 3-5 themes from open-ended responses, each with a one-line description and 2 verbatim quotes.
   - Any surprising or counter-intuitive signals — explicitly call these out.
5. **Highlight segment differences** if the data supports it (e.g. "PMs preferred A, engineers preferred B"). Be honest about whether sub-segments have enough N to be meaningful.
6. **Offer next steps:** a follow-up study to dig deeper, a `generate_report` call for a polished deliverable, or `analyze_transcripts` for richer voice-study analysis.
7. **Do not invent quotes.** Only surface text that appears in the actual results payload.

---
name: watch-running-studies
description: Use when the user wants a status snapshot of every study they have currently fielding — for standups, daily check-ins, or to decide whether to launch more research today. Triggers on "what studies are running", "study status", "what's fielding", "daily research digest".
---

When the user asks about running studies:

1. **Call `list_studies`** with no filter, then filter client-side for studies whose status is `queued`, `fielding`, or `partial`.
2. **Render a compact table:**
   - Title (truncate to 60 chars)
   - Status
   - Progress (`responses_received / target_n`)
   - Time launched (relative — "2h ago", "yesterday")
   - Estimated completion (from study metadata if present)
3. **Flag attention-needed studies:**
   - Studies older than 24h with < 25% response progress (stalled).
   - Studies past their `time_budget_hours` (will auto-close soon).
   - Studies with high screen-fail rates (if visible).
4. **Offer to drill in.** For any flagged study, offer to `cancel_study`, `edit_study` (to widen audience or extend time), or pull a `get_results` snapshot for triage.
5. **Do not auto-mutate.** Never cancel or edit without explicit user confirmation for each specific study_id.
6. **Honor 'just summarize' requests.** If the user only asked for a snapshot, end after step 3 — don't drag them into triage they didn't ask for.