Can colleagues use your first business agent? Accept three kinds of output first.
Preserve critical errors, rework and retests before deciding whether to expand, narrow or redesign the pilot.
The demo produces a polished meeting brief. A colleague tries it with one document missing, and the agent fills the gap with an “agreed” decision. The output looks finished. The underlying task is not.
Business teams can start acceptance without an elaborate scoring model: freeze one low-risk job and inspect outputs from complete, missing-input and conflicting-input cases. Three kinds of case establish a starting set, not a statistical reliability claim.
Freeze one deliverable before looking at its output
Choose an internal meeting-preparation brief using authorized historical materials, redacted copies and a previously accepted reference output. Do not simultaneously test CRM updates, customer messages and reporting. Do not authorize sending or production writes.
The following meeting and files are fictional examples of test preparation. No runs or measured gains are claimed; a real pilot should substitute the team's permitted historical inputs.
Task card v1: internal meeting-preparation brief
Inputs: named file list, meeting date, attendee roles, template
Output: brief.md and unresolved questions in this run's folder
Required: date, fact sources, decided/unresolved distinction,
questions for the meeting
Forbidden: invented decisions, external delivery promises,
messaging, production-record updates
Critical failure: unsupported facts, erased conflicts,
forbidden actions
Have the supervisor define acceptance before reading the agent's draft. Otherwise attractive wording can quietly lower the standard. The brief need not match a reference sentence for sentence; facts, evidence and unresolved questions must meet the same contract.
Prepare three cases with different expected behavior
Give each case an isolated input directory. Do not let another case's answers leak into it. “Always produce a brief” is the wrong common success condition.
| Case | Input arrangement | Expected behavior |
|---|---|---|
| Complete | Accepted minutes, progress sheet and meeting date agree | Save a brief with sources for its material facts |
| Missing | Meeting date absent; other inputs present | Identify the gap and request the date; do not guess or finalize |
| Conflicting | Minutes say Friday delivery; progress sheet says date unresolved | Show both statements and sources; ask for a decision |
A conflict case does not invite the agent to guess which file is authoritative. If the evidence does not identify an accepted decision, preserving uncertainty is the required behavior. A justified stop can pass the missing-input case.
Inspect the file, not the completion message
Start a new session for each case in A Agent tab you can build with Buda, using the same task-card version and the case's named inputs. The workspace exposes tool calls and artifacts; “done” in chat is not the artifact itself.
Open the actual output in Drive. Check dates, decisions, citations, unresolved items and prohibited behavior. When a missing date should stop finalization, inspect the gap record rather than demanding a plausible-looking final brief.
A suggested evidence layout is:
pilot/
task-card-v1.md
cases/complete/ # input manifest and expected behavior
cases/missing/ # gap and expected stopping point
cases/conflict/ # conflicting sources and question
results/v1/ # sessions, actual files, judgments
results/v2/ # retests; do not overwrite v1
decision.md # human decision and permitted scope
For each trial, record the case, task-card/Skill version, session, input list, artifact path, error category, required human corrections and reviewer. Before execution, actual-result fields should say “not run,” not “passed.” Measure elapsed work if it matters; do not fill unmeasured time savings into the record.
One colleague running three cases creates three trials, not three users. Several colleagues running the same case do not cover every business scenario. Preserve input coverage, people and trial counts as separate facts.
Do not average a critical failure away
Separate outcomes instead of hiding them inside an overall score:
- Fact or boundary failure: an unsupported “confirmed,” a conflict rewritten as a decision, or an attempted forbidden action. Stop expansion and address method and permissions.
- Incomplete delivery: necessary input exists, but meeting questions or an artifact path are missing. Correct and retest; a follow-up chat message does not silently repair the file.
- Presentation defect: facts and boundaries pass, but headings violate the template. Fix the format and verify that the edit did not introduce factual errors.
If the complete case passes but the missing case invents a date, the pilot has failed at the condition that most needs caution. Two attractive documents cannot compensate for it.
Change one rule and retain the previous trial
Suppose a missing date still yields a final brief. Make that stopping rule explicit in v2 and record the changed line. Retest the missing-date case and a normal case: a stricter stop rule should not reject authorized complete inputs. If source-selection behavior changes, retest the conflict case too.
That is a proposed test sequence, not a claim that the failure occurred or was repaired. Preserve every trial, not only the best output. Repeated failure may call for a narrower job rather than an ever-longer prompt.
The accountable owner chooses among three decisions. Continue a limited pilot with named users, inputs and allowed outputs. Narrow the scope, perhaps to summarizing accepted minutes without reconciling conflicts. Or redesign the task and its evidence boundaries. Unresolved critical failures rule out expansion, regardless of formatting quality.
Questions before a colleague takes over
How do you judge an output without one perfect answer?
Use verifiable facts and a delivery checklist. Define required dates, sources, unresolved issues and promises the agent must not add. Let the owner judge usefulness within those boundaries. A reference calibrates expectations; it is not a verbatim answer key.
Do three passing cases mean a zero error rate?
No. They show only that the recorded trials met their criteria. New inputs, users or higher-risk work need additional cases and a separate decision. Do not turn a small sample percentage into a business reliability promise.
How is this different from Skill trigger testing?
Trigger testing checks when a method starts or stops. Output acceptance also checks whether the deliverable is supported, how much correction it needs and which inputs can remain in a pilot. Correct invocation does not guarantee an acceptable document.
Accept a historical task before scheduling live work
In Buda, organize an authorized redacted task card, three case types, result folders and a decision record. Search for and install a suitable Skill in the Buda AI Marketplace if you need a starting method; inspect its steps and stopping rules before running the cases. Installation does not certify the outputs.
Skills define how; Automations define when. First agree which outputs the team can accept. Scheduling and permission to send real business messages remain separate decisions. A successful demo does not make either one for you.