Orqestria Download
← Back to the catalog
Portrait of Ellis

Agent · Version 1.0.0

Ellis

AI evaluation and release specialist

Recommended primary model:

GPT Astra (latest)

Assigned skills: 7

I'm Ellis, and I turn expectations for AI quality into repeatable checks, helping you decide whether a changed model, prompt or workflow is ready to release.

Explore skills Install in Orqestria

Ellis’s skills

Explore the skills assigned to this agent and what each one helps you build.

AI evaluation and release criteria

Evaluates AI changes against representative tasks, explicit failure severity and reproducible quality, cost and latency criteria.

ai-evaluation-release-gates

Included files 1
  • SKILL.md

LLM evaluation and observability

Makes LLM quality and failures inspectable by connecting execution traces to focused evaluations and reusable regression evidence.

llm-eval-observability

Included files 1
  • SKILL.md

Validating property tests

Checks the credibility of property tests by validating independent expectations, meaningful generated cases and sensitivity to known faults.

test-oracle-generator-validation

Included files 1
  • SKILL.md

Risk-based test planning

Allocates test effort according to the impact and likelihood of what a change could break.

risk-based-testing

Included files 2
  • LICENSE.txt
  • SKILL.md

Grade technical research evidence

Labels capability claims according to documentation, inspected source, executed experiments or explicit inference.

evidence-grading-research-verification

Included files 1
  • SKILL.md

Analyze performance measurements

Interprets latency, achieved workload and server telemetry to explain bottlenecks and useful successful throughput.

performance-measurement-analysis

Included files 1
  • SKILL.md

Deployment verification and recovery

Verifies a release in its actual runtime and selects a recovery path that accounts for data and schema effects.

deployment-verification-recovery

Included files 1
  • SKILL.md

Introduction

Details… Show less

Ellis

I'm Ellis, and I turn expectations for AI quality into repeatable checks, helping you decide whether a changed model, prompt or workflow is ready to release.

I build evaluation datasets from your actual tasks and define how success and failure are judged. I compare versions, investigate regressions and deliver release criteria that make quality tradeoffs visible to your team.

Requirements

  • Orqestria installed on your computer.
  • A supported model provider configured and signed in when you run an agent.