prove → fix → verify → coach

7 min

is what it took to review, prove, fix, and verify five planted bugs in a small Python module — a task that would take an experienced developer roughly 3 hours 45 minutes by hand.

~32× faster on this run, across five bug classes: a remote-code-execution risk, an off-by-one total, an unvalidated quantity, float-based money math, and duplicated logic.

Why this is different

Most AI review tools stop at suggesting a fix. ProofFix treats a suggestion as a claim that has to be checked.

Run live in IBM Bob

Each stage below ran as its own IBM Bob mode, with access limited to what that job needed — read-only for review, edit-and-run for fixes, markdown-only for the report.

What it found

Five bugs were deliberately planted in a small shopping-cart module. Click a finding to see the proof, the fix, and the lesson.

F-01 Discount codes evaluated with eval() critical 28×

Risk

A crafted discount-code string could execute arbitrary code inside the server process.

Proof

An injected payload triggered evaluation before the fix — confirmed with a failing test.

Fix

Replaced eval() with a plain dictionary lookup.

Lesson

Never pass user-supplied strings into eval or exec; use a lookup table instead.

▸ Run replay

F-02 Off-by-one error drops the last item from every total high 40×

Risk

Every multi-item cart under-charged by the price of its last item; single-item carts totalled zero.

Proof

A two-item cart returned the wrong subtotal before the fix.

Fix

Replaced manual index arithmetic with a plain sum over all items.

Lesson

Prefer iterating a collection directly over hand-written index ranges.

▸ Run replay

F-03 Zero and negative quantities silently accepted high 40×

Risk

A quantity of zero or less could insert ghost items or push stock counts negative.

Proof

Adding an item with quantity 0 completed without error before the fix.

Fix

Added a validation guard that raises on non-positive quantities.

Lesson

Validate domain rules at the point of entry, not downstream.

▸ Run replay

F-04 Money stored and summed as float medium 30×

Risk

Floating-point rounding drift accumulates across line items, producing penny-level errors.

Proof

Ten items at $0.10 summed to a value slightly off $1.00 before the fix.

Fix

Switched prices and rates to Decimal throughout.

Lesson

Never use float for currency; use a fixed-point type from the start.

▸ Run replay

F-05 Duplicated branches in the total calculation medium 30×

Risk

Two identical branches meant a future change to the discount formula could be made in one place and missed in the other.

Proof

A type-hint check confirmed the method hadn't been reconciled with the rest of the codebase.

Fix

Collapsed the duplicated branches into a single, direct calculation.

Lesson

When two branches are identical, extract the shared logic instead of guarding around it.

▸ Run replay

The same run also surfaced three additional real issues beyond these five — a related empty-cart edge case, an unguarded negative-quantity update, and a float/Decimal mismatch in the discount table.

Result

Built with three IBM Bob modes

Each mode has exactly the access its job needs — nothing more.

Mode	Role	Access
Reviewer	Finds and ranks issues by risk	Read only
Debugger	Writes the failing test, applies the fix, runs the suite	Read, edit, run commands
Coach	Writes the report and scores the run	Edit limited to Markdown files