← Back to Open Source Due Diligence Team
See how it helps
Grade technical research evidence
Labels capability claims according to documentation, inspected source, executed experiments or explicit inference.
evidence-grading-research-verification
Team members using this skill
When is it useful?
Use when a technology recommendation depends on claims that may only exist in a README.
What does it help you do?
- Traces decisive mechanisms to a specific source version.
- Designs small reproducible experiments for assumptions that affect the decision.
- Examines benchmark realism and records what remains unverified.
Try asking this
Assess this library's streaming claims and report which are documented, code-confirmed or demonstrated in our runtime. Test cancellation after the first streamed chunk. Cite the implementation path and distinguish observed cleanup from behavior inferred only from the documentation.
Detailed documentation
Read the skill’s original instructions, examples and checks in their authored language.
Show less
Evidence Grading & Research Verification
Credible technical research never blurs marketing claims, code reading, live experiments, and personal inference. Every conclusion must carry its exact evidence grade.
The Four Evidence Tiers
Label every major capability or finding explicitly:
- Claimed (Documentation / README / Marketing): Stated in docs, blog posts, or comments, but not yet verified in source code or execution.
- Source-Verified (Code-Traced): Traced through the actual implementation at a specific version/commit (
repo@commit:path#L10-L45), confirming the mechanism and edge-case handling. - Empirically Verified (Executed / Tested): Proven by running the project's test suite or a minimal, reproducible experiment in our target runtime environment.
- Inferred (Researcher Judgment): A reasoned architectural conclusion or extrapolation — stated with its assumptions and what would falsify it.
Workflow
- Verify decisive claims with the smallest possible experiment (Willison & Beck):
- When a recommendation hinges on a critical claim (e.g., "supports streaming tool calls without buffering", "handles 50k nodes under 50 ms", "works in Node 22 ESM without native builds"), write and run a 20–50 line isolated verification script if the environment allows.
- Preserve the exact command, input fixture, and observed output in the research report so any teammate can reproduce it in seconds.
- Audit third-party benchmarks critically:
- Never repeat a README benchmark chart at face value. Check the benchmark source code for:
- Dataset size and realism, warm-up iterations, and whether competitors were configured fairly.
- Hardware, concurrency model, and whether error checking or durability was disabled to inflate numbers.
- Never repeat a README benchmark chart at face value. Check the benchmark source code for:
- Track open unknowns and verification triggers:
- If an environment constraint prevents live testing (e.g., requires an external enterprise account or specific hardware), mark the item as Source-Verified / Untested at Runtime and define the exact verification step for phase 1 of implementation.
- Stop when decision sufficiency is reached:
- Once the evidence clearly separates the winning option from the alternatives and bounds the integration risks, conclude the research and deliver the recommendation.
Verification Checklist
- Every key recommendation cites primary source files and version/commit identifiers.
- Project claims, code-traced facts, executed tests, and inferences are clearly distinguished.
- No repository URLs, file paths, APIs, or benchmark results are fabricated or guessed.
- Remaining unknowns have a concrete, low-cost verification plan.
Included files 1
SKILL.md