Diagnostics / Agent harness check

Is your AI agent ready for production?

12 checks across the 6 parts of an agent harness. You'll get a readiness level, a score for each part and the three gaps to close first.

01 / 6Tools & permissions

Can the agent call only the tools on an explicit allowlist?

Does each tool get only the access it needs, with read-only access where possible?

02 / 6Memory & state

Have you decided what persists between runs and what is thrown away?

Are memory and data kept separate per customer or tenant?

03 / 6Quality gate

Does output pass automatic checks (format, business rules, banned content) before it leaves?

Do you run an evaluation set of real cases before each prompt, model or tool change?

04 / 6Human review

Is it written down which actions a person must approve before they happen?

When the agent is unsure, does it hand the case to a person with the context attached?

05 / 6Budget & limits

Is there a hard cost cap per run, in tokens or spend?

Do runs have a timeout, a retry limit and a step limit?

06 / 6Observability

Is every model and tool call logged with its inputs, outputs and cost?

Can you trace any output back to the run, inputs and versions that produced it?

What the check covers

Six parts that sit around the model.

The same harness we run our own agents in. A model can pass a demo without any of it; real work needs all six.

01

Tools & permissions

The agent calls only allowlisted tools, and each tool gets the narrowest access it needs.

02

Memory & state

A clear rule for what persists between runs, kept separate for each customer or tenant.

03

Quality gate

Automatic checks and an evaluation set stand between the agent's output and the outside world.

04

Human review

A person approves the actions that matter, and unsure cases reach a person with context.

05

Budget & limits

A cost cap per run, plus timeouts, retry limits and step limits so a run always ends.

06

Observability

Every model and tool call is logged, and any output can be traced back to its inputs.

FAQ

Questions about agent readiness

What is an agent harness?

The code and rules around a model that make it safe to run: which tools it may call, what it remembers, the checks its output must pass, where a person approves, the budget per run and the logs. The model does the reasoning; the harness decides what it is allowed to do.

How is the score calculated?

Each of the 12 checks scores 2 points for yes, 1 for partly and 0 for no, scaled to 100. Production-ready needs 80 or more with no part below half. Pilot-ready needs 50 or more with no part at zero. Both need at least a partial yes on which actions a person approves. Anything else is a prototype.

We scored Prototype. Should we stop the pilot?

Not necessarily. Prototype is the right stage for testing on sample data. Close the top gaps before real customers, money or records are involved, starting with approvals and the quality gate.

Is anything I enter sent to you?

No. The check runs in your browser. Your answers are not stored or sent, and no model is called.

Book an AI scoping call

Bring one workflow. We'll tell you if an agent should run it.

Book an AI scoping call →

or write to contact@byteandrise.com · we work under NDA