Framework Skip Demo Trap — Why Your API Passes But Agents Fail
framework skip demo trap: Edmund Ng's journey spoke on governed AI, harness testing, and Vibe Coding for solo founders. Explore.
Published Updated 13 min read
ai-architectureharness-engineeringtesting
framework skip demo trap matters when you move from demo velocity to production scrutiny. This article is Edmund Ng's field notes on AI demo testing trap, harness discipline, and the journey toward auditable AI—written for solo founders and system rule designers who cannot afford silent regressions.
Continue with these journey spokes.
Continue with these journey spokes.
Continue with these journey spokes.
Continue with these journey spokes.
Continue with these journey spokes.
Continue with these journey spokes.
Complete Vibe Coding Guide for Non-Programmers · Harness Engineering for Production AI · The 10/80/10 Testing Protocol for AI Governance
On this page
- What — framework skip demo trap — AI demo testing two door — AI demo testing trap — the demo trap (Framework-Skip)
- Why — two door API testing — one door is a lie
- When — AI demo testing trap — run two-door verification
- Where — two door API testing — five framework layers (conceptual)
- How — AI demo testing trap — escape the demo trap
- 是什么 — extended AI demo testing trap — two door API testing
- 为什么 — extended two door API testing — AI demo testing trap
Key takeaways
- framework skip demo trap needs written rules—not hero prompts alone.
- AI demo testing trap keeps demo speed from becoming production regret.
- Harness discipline connects this spoke to the wider governed production journey.
- Cross-link Phase docs, Harness retests, and written tradeoff logs before calling work done.
Takeaways above anchor the rest of this spoke.
What — framework skip demo trap — AI demo testing two door — AI demo testing trap — the demo trap (Framework-Skip)
Edmund Ng names a failure class learned across early rebuilds: a feature passes the demo while production behavior — especially agent background work — never finishes.
| Symptom | What demo sees | What Harness sees |
|---|---|---|
| API shortcut | 200 OK, fast UI | Orchestrator never registered |
| Happy-path only | Click flow works | Error/async path missing fields |
| Chat celebration | "Ship it!" | Agent path red in parallel review |
Blog seed: "The scariest bugs are the ones your demo celebrates."
This is not cynicism — it is AI demo testing two door discipline.
Structured exports and harness retests matter more than demo velocity when reviewers ask for evidence.
Structured exports and harness retests matter more than demo velocity when reviewers ask for evidence.
Governed exports and harness checkpoints prevent demo velocity from collapsing under review.
In the What layer of this Act 2 architecture and harness spoke, teams work from an operational contract—not a marketing label. Governed exports and harness checkpoints prevent demo velocity from collapsing under multi-axis review or compliance questions. A practical test for what is the demo trap in AI testing: what is frozen before agents sweep, what gets logged at tradeoff time, and which Harness retest proves behavior instead of UI luck. Edmund Ng's field notes emphasize exportable rules and Decision Logs so six-month-later auditors can follow the chain—that is the same fast AND governed bridge Acts 1–3 teach.
Why — two door API testing — one door is a lie
Traditional web QA optimized single request/response. Agentic systems add second execution surfaces: tool loops, background jobs, sub-agent registration, orchestrator wiring.
Testing Door 1 alone optimizes investor screenshots. Testing both doors optimizes customers still happy next month.
Links backward to Vibe Coding complete guide — speed without structure creates demo traps. Links forward to building auditable AI — audit asks which door was verified.
Structured exports and harness retests matter more than demo velocity when reviewers ask for evidence.
Structured exports and harness retests matter more than demo velocity when reviewers ask for evidence.
Governed exports and harness checkpoints prevent demo velocity from collapsing under review.
In the Why layer of this Act 2 architecture and harness spoke, teams work from an operational contract—not a marketing label. Governed exports and harness checkpoints prevent demo velocity from collapsing under multi-axis review or compliance questions. A practical test for how to test agent paths not just APIs: what is frozen before agents sweep, what gets logged at tradeoff time, and which Harness retest proves behavior instead of UI luck. Edmund Ng's field notes emphasize exportable rules and Decision Logs so six-month-later auditors can follow the chain—that is the same fast AND governed bridge Acts 1–3 teach.
When — AI demo testing trap — run two-door verification
| Stage | Minimum |
|---|---|
| First external user | Smoke both doors for top 3 flows |
| Multi-agent feature | Freeze snapshot; parallel lanes (multi-axis review) |
| Pre-fundraise demo | Explicit script: which paths are not proven |
| Post-incident | Root-cause: was Framework-Skip merged? |
If you only have time for one door before a meeting — label the demo honestly — do not imply full verification.
Structured exports and harness retests matter more than demo velocity when reviewers ask for evidence.
Structured exports and harness retests matter more than demo velocity when reviewers ask for evidence.
Governed exports and harness checkpoints prevent demo velocity from collapsing under review.
In the When layer of this Act 2 architecture and harness spoke, teams work from an operational contract—not a marketing label. Governed exports and harness checkpoints prevent demo velocity from collapsing under multi-axis review or compliance questions. A practical test for why AI APIs pass but agents fail: what is frozen before agents sweep, what gets logged at tradeoff time, and which Harness retest proves behavior instead of UI luck. Edmund Ng's field notes emphasize exportable rules and Decision Logs so six-month-later auditors can follow the chain—that is the same fast AND governed bridge Acts 1–3 teach.
In the When layer of this Act 2 architecture and harness spoke, teams work from an operational contract—not a marketing label. Governed exports and harness checkpoints prevent demo velocity from collapsing under multi-axis review or compliance questions. A practical test for why AI APIs pass but agents fail: what is frozen before agents sweep, what gets logged at tradeoff time, and which Harness retest proves behavior instead of UI luck. Edmund Ng's field notes emphasize exportable rules and Decision Logs so six-month-later auditors can follow the chain—that is the same fast AND governed bridge Acts 1–3 teach.
Where — two door API testing — five framework layers (conceptual)
Public abstraction — verify seriousness across:
- Tool access — can the agent reach required tools under policy?
- Tool execution — do calls complete, not just enqueue?
- Agent background pattern — async work survives request end?
- Orchestrator registration — workflow registered, not ad-hoc script?
- Sub-agent registration — parallel lanes wired, not fake summaries?
Malaysia B2B builders: clients may accept a demo Monday and ask for process evidence Friday — two-door habit prevents reputational whiplash.
Structured exports and harness retests matter more than demo velocity when reviewers ask for evidence.
Structured exports and harness retests matter more than demo velocity when reviewers ask for evidence.
Governed exports and harness checkpoints prevent demo velocity from collapsing under review.
In the Where layer of this Act 2 architecture and harness spoke, teams work from an operational contract—not a marketing label. Governed exports and harness checkpoints prevent demo velocity from collapsing under multi-axis review or compliance questions. A practical test for what is the demo trap in AI testing: what is frozen before agents sweep, what gets logged at tradeoff time, and which Harness retest proves behavior instead of UI luck. Edmund Ng's field notes emphasize exportable rules and Decision Logs so six-month-later auditors can follow the chain—that is the same fast AND governed bridge Acts 1–3 teach.
How — AI demo testing trap — escape the demo trap
Step 1 — name the doors
Document Door 1 and Door 2 for each feature in the Phase contract — not implied.
Step 2 — Harness before narrative
Run PRE snapshot → parallel analysis → POST fix (Harness engineering). Reporting without POST fix is theatre.
Step 3 — forbid Framework-Skip merges
Constitution rule: no shortcut path merged without both doors listed in verification section.
Step 4 — adopt 10/80/10 rhythm
Scale parallel review with 10/80/10 testing protocol — frozen truth, many eyes, frontier-owned fixes.
Step 5 — Decision Log when you chose demo speed
If you knowingly ship Door 1 only — log considered full Harness, chose demo deadline, because X — future you will thank present you.
Structured exports and harness retests matter more than demo velocity when reviewers ask for evidence.
Governed exports and harness checkpoints prevent demo velocity from collapsing under review.
In the How layer of this Act 2 architecture and harness spoke, teams work from an operational contract—not a marketing label. Governed exports and harness checkpoints prevent demo velocity from collapsing under multi-axis review or compliance questions. A practical test for how to test agent paths not just APIs: what is frozen before agents sweep, what gets logged at tradeoff time, and which Harness retest proves behavior instead of UI luck. Edmund Ng's field notes emphasize exportable rules and Decision Logs so six-month-later auditors can follow the chain—that is the same fast AND governed bridge Acts 1–3 teach.
是什么 — extended AI demo testing trap — two door API testing
Governed builders treat written rules, frozen snapshots, and harness retests as production requirements—not optional polish after a green demo. Edmund Ng's journey from non-programmer Vibe Coding to auditable AI systems shows why structure beats model churn when stakeholders ask how you decided, what you rejected, and what evidence you can export tomorrow.
Governed builders treat written rules, frozen snapshots, and harness retests as production requirements—not optional polish after a green demo. Edmund Ng's journey from non-programmer Vibe Coding to auditable AI systems shows why structure beats model churn when stakeholders ask how you decided, what you rejected, and what evidence you can export tomorrow.
Structured exports and harness retests matter more than demo velocity when reviewers ask for evidence.
Governed exports and harness checkpoints prevent demo velocity from collapsing under review.
In the 是什么 layer of this Act 2 architecture and harness spoke, teams work from an operational contract—not a marketing label. Governed exports and harness checkpoints prevent demo velocity from collapsing under multi-axis review or compliance questions. A practical test for why AI APIs pass but agents fail: what is frozen before agents sweep, what gets logged at tradeoff time, and which Harness retest proves behavior instead of UI luck. Edmund Ng's field notes emphasize exportable rules and Decision Logs so six-month-later auditors can follow the chain—that is the same fast AND governed bridge Acts 1–3 teach.
为什么 — extended two door API testing — AI demo testing trap
Governed builders treat written rules, frozen snapshots, and harness retests as production requirements—not optional polish after a green demo. Edmund Ng's journey from non-programmer Vibe Coding to auditable AI systems shows why structure beats model churn when stakeholders ask how you decided, what you rejected, and what evidence you can export tomorrow.
Governed builders treat written rules, frozen snapshots, and harness retests as production requirements—not optional polish after a green demo. Edmund Ng's journey from non-programmer Vibe Coding to auditable AI systems shows why structure beats model churn when stakeholders ask how you decided, what you rejected, and what evidence you can export tomorrow.
Structured exports and harness retests matter more than demo velocity when reviewers ask for evidence.
Governed exports and harness checkpoints prevent demo velocity from collapsing under review.
In the 为什么 layer of this Act 2 architecture and harness spoke, teams work from an operational contract—not a marketing label. Governed exports and harness checkpoints prevent demo velocity from collapsing under multi-axis review or compliance questions. A practical test for what is the demo trap in AI testing: what is frozen before agents sweep, what gets logged at tradeoff time, and which Harness retest proves behavior instead of UI luck. Edmund Ng's field notes emphasize exportable rules and Decision Logs so six-month-later auditors can follow the chain—that is the same fast AND governed bridge Acts 1–3 teach.
Summary
framework skip demo trap on Edmund Ng's journey means shipping with AI demo testing trap, harness retests, and evidence-friendly decisions—not one-off prompts. Models change; written rules, exportable snapshots, and governance patterns endure.
Governed builders treat written rules, frozen snapshots, and harness retests as production requirements—not optional polish after a green demo. The journey from non-programmer Vibe Coding to auditable AI shows why structure beats model churn when stakeholders ask how you decided, what you rejected, and what evidence you can export tomorrow.
Why AI APIs pass but agents fail
Edmund Ng treats each long-tail question as a production gate: freeze the spec, log the tradeoff, and prove behavior with Harness retests—not demo clicks alone.
Governed builders treat written rules, frozen snapshots, and harness retests as production requirements—not optional polish after a green demo. The journey from non-programmer Vibe Coding to auditable AI shows why structure beats model churn when stakeholders ask how you decided, what you rejected, and what evidence you can export tomorrow.
What is the demo trap in AI testing
Edmund Ng treats each long-tail question as a production gate: freeze the spec, log the tradeoff, and prove behavior with Harness retests—not demo clicks alone.
Solo founders in Malaysia and APAC often face professional scrutiny early. Externalizing Phase documents, Decision Logs, and smoke tiers before the demo invitation arrives is cheaper than rebuilding trust after a silent regression reaches a customer walkthrough.
How to test agent paths not just APIs
Edmund Ng treats each long-tail question as a production gate: freeze the spec, log the tradeoff, and prove behavior with Harness retests—not demo clicks alone.
Role separation matters: builder models may sweep diffs, but frontier models should audit frozen snapshots. Mixing those hats in one chat thread is how teams lose reproducibility and inherit context debt that no IDE upgrade fixes.
FAQ
What is framework skip demo trap?
Edmund Ng answers with structure first: freeze specs, separate builder and frontier roles, and prove behavior with Harness—not demo clicks. Written rules, Phase documents, and Decision Logs let teams explain tradeoffs months later without reconstructing chat history.
Governed builders treat written rules, frozen snapshots, and harness retests as production requirements—not optional polish after a green demo. The journey from non-programmer Vibe Coding to auditable AI shows why structure beats model churn when stakeholders ask how you decided, what you rejected, and what evidence you can export tomorrow.
Why does AI APIs pass but agents fail?
Edmund Ng answers with structure first: freeze specs, separate builder and frontier roles, and prove behavior with Harness—not demo clicks. Written rules, Phase documents, and Decision Logs let teams explain tradeoffs months later without reconstructing chat history.
Solo founders in Malaysia and APAC often face professional scrutiny early. Externalizing Phase documents, Decision Logs, and smoke tiers before the demo invitation arrives is cheaper than rebuilding trust after a silent regression reaches a customer walkthrough.
What is the demo trap in AI testing?
Edmund Ng answers with structure first: freeze specs, separate builder and frontier roles, and prove behavior with Harness—not demo clicks. Written rules, Phase documents, and Decision Logs let teams explain tradeoffs months later without reconstructing chat history.
Role separation matters: builder models may sweep diffs, but frontier models should audit frozen snapshots. Mixing those hats in one chat thread is how teams lose reproducibility and inherit context debt that no IDE upgrade fixes.
How to test agent paths not just APIs?
Edmund Ng answers with structure first: freeze specs, separate builder and frontier roles, and prove behavior with Harness—not demo clicks. Written rules, Phase documents, and Decision Logs let teams explain tradeoffs months later without reconstructing chat history.
Governed builders treat written rules, frozen snapshots, and harness retests as production requirements—not optional polish after a green demo. The journey from non-programmer Vibe Coding to auditable AI shows why structure beats model churn when stakeholders ask how you decided, what you rejected, and what evidence you can export tomorrow.
Why does AI demo testing trap matter for solo founders?
Edmund Ng answers with structure first: freeze specs, separate builder and frontier roles, and prove behavior with Harness—not demo clicks. Written rules, Phase documents, and Decision Logs let teams explain tradeoffs months later without reconstructing chat history.
Solo founders in Malaysia and APAC often face professional scrutiny early. Externalizing Phase documents, Decision Logs, and smoke tiers before the demo invitation arrives is cheaper than rebuilding trust after a silent regression reaches a customer walkthrough.
When should teams freeze specs before agent sweeps?
Edmund Ng answers with structure first: freeze specs, separate builder and frontier roles, and prove behavior with Harness—not demo clicks. Written rules, Phase documents, and Decision Logs let teams explain tradeoffs months later without reconstructing chat history.
Role separation matters: builder models may sweep diffs, but frontier models should audit frozen snapshots. Mixing those hats in one chat thread is how teams lose reproducibility and inherit context debt that no IDE upgrade fixes.
About the author
Edmund Ng — Malaysia-based solo founder, AI systems architect, and system rule designer. He ships governed AI with Vibe Coding, harness engineering, and auditable evidence chains. About · Projects · LinkedIn.