Bug-fix history

Churn tells you a file changes a lot, which is often just where the work is. Bug-fix history tells you a file breaks a lot, which is a different and more actionable thing. Repowise records which files and which functions have actually been fixed, how recently, and how often, then hands you a small set of honest numbers: a per-file fix count over a trailing 180-day window, a per-symbol breakdown, a decayed "bug magnet" flag, and the age of the most recent fix.

No LLM calls. This is git history, a -U0 diff pass, and arithmetic, run inside the existing git phase of repowise init and repowise update.

On this repo the top bug magnets (pipeline/persist.py, pipeline/incremental.py, dead_code/analyzer.py, update_cmd/command.py) match the churn hotspot list, arrived at independently from bug fixes rather than from commit counts.

What counts as a bug fix

A commit whose subject matches the fix keywords is a candidate, not a fact. Plenty of fix: commits fix a README typo, tighten a test, or bump a version, and counting those inflates every downstream number. So each candidate's -U0 diff is classified into one shape, and only one shape counts.

The comment-only rule is line-level, not path-level: it strips trailing comments and compares the code part of each removed and added line, so a commit that rewords a docstring in a .py file does not count as a bug fix.

The classifier was validated against 240 hand-labelled fix commits across four repos (repowise, flask, django, zod) and agrees with the labels 98.3% of the time (repowise, flask and django all 60/60; zod 56/60). The four residual disagreements are all the same shape, "a file with a code extension changed but no product code did", and were left as honest disagreements rather than closed with repo-specific ignore lists.

The effect varies enormously by repo. On flask, 58.7% of keyword-matched fix commits are noise, and filtering removes half the raw attributions. On repowise it is 7.0%. That is the point: a number that means the same thing in both repos is worth more than a bigger number in one. GitMetadata keeps both prior_defect_count (filtered) and prior_defect_raw_count (unfiltered), so the delta stays inspectable.

Filtering deliberately did not change the health score. Re-running the defect-weight calibration on the 21-repo corpus showed it moves pooled OOF AUC by +0.0002, inside noise, so the prior_defect marker stays at weight 1.0.

Per-symbol attribution

Each counted fix produces one row per file it touched in a fix_events table, carrying the line ranges the fix replaced, numbered on the fix's own parent commit. Persisting the events, rather than only a count, is what makes per-symbol attribution possible later without re-walking git.

Those replaced ranges are matched against the symbol spans in each file, producing a per-file map of which function or class absorbed each fix.

"In-envelope" means the event's lines land somewhere between the file's first and last indexed symbol. Gaps between symbols still count as misses, so this stays a test of the join rather than a restatement of it. The misses were measured, not argued: django and flask are line drift (symbol spans are at HEAD, fix ranges are at each fix's parent, and 99.8% of django's events are 3+ years old), and zod is symbol density in test files the parser indexes with one or two named symbols.

The bug magnet rollup

Each file's counted fixes collapse into one decayed mass:

mass = Σ 0.5 ^ (age_days / 90)
bug_magnet = mass >= 3.0

The 90-day half-life came from a sweep of 60 / 90 / 180 days against the 21-repo calibration corpus. Decay was the only lever that moved the prior_defect coefficient at all (0.15 undecayed to 0.23 decayed); 90d and 180d tied inside noise, and 90d is the tighter of the two.

Read the threshold with the decay in mind. Only a same-day fix is worth a full 1.0, so three real fixes spread over a couple of weeks land near 2.9 and do not flag. In practice the flag needs four recent fixes, or three very recent ones. That is deliberately the conservative end: the flag exists to interrupt someone mid-edit, so it should be rare enough to be worth reading.

The rollup recomputes the whole repo on every index and update rather than patching only the files an update touched, because decay ages a rollup with nothing in the file changing. It costs about 0.25s on this repo (1,172 events, 739 files): two indexed queries and arithmetic.

The recency contract

One rule keeps the feature honest, and it is enforced in the type definitions rather than left to each surface:

bug_magnet is the decayed fix mass past its trigger, so it is a recency claim. Any copy that shows it must show last_fix_at too.

Two consequences:

1. A magnet claim never appears without its age. If last_fix_at is null, the flag is dropped rather than shown unanchored.
2. The surface goes silent outside the window. The pre-edit hook does not fire at all when the last fix is older than 180 days, no matter how large the historical count is. "This file was fixed nine times, in 2019" is not a warning, it is trivia.

Where it surfaces

Pre-edit hook

When an agent is about to edit a file with a real recent run of fixes, one line goes into its context:

[repowise] pipeline/persist.py has been bug-fixed 5x in the last 6 months,
last 2 weeks ago (bug magnet); mostly in run_pipeline.

Three gates must all hold: at least 3 counted fixes, a last fix inside 180 days, and one claim per file per session (an atomic ledger, so two racing hook processes cannot double-fire). A file that also has a governing architectural decision emits two lines, never more. Both the Claude Code and Codex hook paths are wired up.

get_risk

Files with counted fixes gain a defect_profile block:

get_risk(targets=["packages/core/src/repowise/core/pipeline/persist.py"])
# defect_profile: {fix_count, window, last_fix_days_ago,
#                  bug_magnet, top_symbols}

bug_magnet appears only when true, top_symbols is the top three with the path prefix stripped, and the whole block is omitted on repos with no fix data. Measured at 46 tokens mean, 80 worst case, against a 150-token ceiling across 803 files.

Shipping defect_profile also retired the old keyword risk_type="bug-prone" derivation, which used to scan commit subjects for /fix|bug|patch/, counted doc and test commits, had no recency, and disagreed with the count every other surface showed. It now reads bug_magnet or prior_defect_count >= 3. That does silently reclassify some files: a high-traffic file with 3 fixes in 200 commits now reads bug-prone where it read churn-heavy.

get_change_risk

A diff or commit range gains a prior_fixes block: per changed file, how many past fixes it carries and how many of the change's lines fall inside a past fix's replaced ranges. The per-file counts are exact (counted by COUNT(DISTINCT fix_sha), so one commit fixing three files reports one fix, not three). The line overlap is labelled approximate in the payload, because past ranges are numbered on their own parent commit. Files rank by overlap then count, and the list carries a truncated flag rather than silently dropping the tail.

Dashboard and editor

• Health drawer: a collapsed Bug history section with per-symbol counts, the last-fix age on the toggle, and an approximation footnote.
• File signals panel: the bug-fix tile gains a Bug magnet badge and folds the last-fix age into its caption.
• Symbol detail: a Bug fixes stat tile beside Modifications. How often something changes next to how often it breaks is the contrast that earns the cell.
• Symbols list: a Bug-fixed facet and a per-row fix chip. The filter is symbol-level, unlike the file-level filters beside it, because a bug-fixed file says very little about the one function you are reading.
• VS Code: one line on the file hover, off the same signals block.

What was measured and deliberately not shipped

The most useful thing this layer can tell you is what it refuses to tell you.

SZZ inducing-commit attribution: built, measured, deleted

The obvious next step from "this line was fixed" is "and here is the commit that introduced the bug". That is the SZZ algorithm, and it was fully built: a blame pass walking each fix's replaced lines back through history, refactor-aware, with overlap-based ranking of the candidates.

Then it was measured against a hand-labelled set. Fifty-three judged rows, five reviewers. File-level SZZ tops out at 74.5% top-candidate precision on that corpus. The gate was 80%.

74.5% is not a bad algorithm; it is a good SZZ implementation. It is also not good enough to put a commit SHA and a person's name in a UI, because one time in four the UI would accuse the wrong commit and the reader has no way to tell which time. So the blame pass was deleted, szz.py is gone, and no repowise surface names an inducing commit. The planned inducing-commit tail on get_change_risk and the inducing-commit link on commit rows were both cut.

The findings are kept because they are reusable by anyone attempting this again:

• Refactor-aware blame is worth building. Walking through behaviour-preserving moves to the parent lifted strict precision from 70.6% to 74.5%. All 14 of the initial false calls were refactors inheriting moved lines.
• Overlap ranking beats earliest-commit ranking by 12 points on judged rows. Earliest-commit's apparent 80% was an artefact of its wrong answers going unjudged: 7 of its 16 unknowns were this repo's initial commit, and all 7 came back "not plausible" once judged.
• The two residual failure modes are properties of SZZ, not bugs. An initial import has no earlier commit to name, and package splits that move lines wholesale while re-sorting imports defeat the carried-through test on exactly the lines that moved.

Removing the pass also gave back 9.1s of index time on this repo and 5.9s on zod. Restoring the tracer is a git revert of one commit, not a re-derivation, so if someone finds a ranking that clears 80% on the frozen label set, the data layer is ready for it.

"AI vs human introduced bugs": cut

A repo-level card comparing defects introduced by AI-authored commits against human-authored ones, per KLOC, was planned and gated on a concentration check before build. The check failed. The top 5 inducing commits held 30.3% of all traced mass against a 20% ceiling, with the initial import alone at 13.9%. SZZ error does not average out: when blame concentrates on a handful of enormous commits, any aggregate built on inducing commits inherits that concentration, and it becomes mostly a statement about who authored the repo's five biggest commits. A stat like "AI introduces N% more bugs" is exactly the kind of number that gets screenshotted, which raises the bar rather than lowering it. So it was killed before it was built.

What survived

Counts and recency survived. Accusations did not. Everything shipped here answers "this code has been broken before, recently, and roughly here", and nothing answers "and it was your fault". The first question is answerable from git with measurable accuracy. The second is not, yet.

Limitations

• Symbol counts are approximate. Current-tree spans against historical line ranges, stated on every surface that shows them.
• Fix detection is keyword-matched. A bug fixed in a commit whose subject does not say so is invisible. The shape filter improves precision, not recall.
• The 180-day window is fixed. A file fixed heavily two years ago and quiet since reads as clean, which is intentional but is a choice, not a truth.
• Workspace lookup in the hook does not filter by repository_id, matching the co-change query beside it. In a workspace whose members lack their own .repowise this degrades to silence rather than a wrong answer.
• The hook's usefulness is unproven. It shipped without the planned week of live-session dogfooding, so whether the notice changes behaviour or becomes ignorable noise is still open. If it is noise, the plan is to cut or reword the hook surface and keep every data layer beneath it.

See also

• Code health, the prior_defect marker and the "does the score find the bugs?" self-check that reads the same counts.
• get_risk, where defect_profile lands.
• MCP overview for the rest of the tool surface.