Lint and eval harness for agent skills. One CLI with two halves: a keyless structural lint any repo can run in CI, and a promptfoo-driven eval loop that grades what a skill actually makes an agent do.
Built for uinaf skill repos. Nothing in it is uinaf-specific. It ships no opinion about what a skill should say, only about where skills sit and how a scenario is scored.
pnpm add -D @uinaf/skillcheckNode 24 or newer. The package ships compiled ESM, runs no install scripts, and
has no regular dependencies, so a runner using --ignore-scripts is fine and
the lint-only install stays at a handful of packages. The promptfoo eval engine
and provider SDKs are optional peers, installed only on the operator machine
that runs evals (adoption). Consumers still pinned to the
pre-npm git tags are covered there too.
skillcheck lint # structural lint, no credentials
skillcheck run skills/wat/evals/basic # one scenario, graded end to end
skillcheck sweep && skillcheck summarize # every scenario, then a scorecardlint needs nothing. run and sweep need model auth, which is why sweeps
stay operator-run and consumer repos never hold credentials. Every command,
flag, and auth variable is in usage.
Frozen, not configurable. Every command reads one root: --root <dir>, or the
current directory.
<root>/skills/<skill>/SKILL.md linted
<root>/skills/<skill>/evals/<scenario>/task.md the problem + input files
<root>/skills/<skill>/evals/<scenario>/criteria.json the weighted checklist
<root>/.skillcheck/ results, scratch, scorecards
cli/*/skills/<skill>/ is scanned too, for repos that keep a skill next to the
CLI it documents.
| Doc | When |
|---|---|
| Usage | Every subcommand, flag, and auth path |
| Scenarios | Writing an eval scenario |
| Authoring | Writing and auditing the skill itself |
| Adoption | Wiring the lint into another repo |
| Releasing | The npm pipeline |
| Contributing | Local setup and the verify gate |
| Security | Reporting a vulnerability |
MIT · undefined is not a function LLC