Appearance

Characterization Testing ​

Pinning the current behavior of legacy code with tests so change becomes safe

Introduction ​

This article builds on Working with Legacy Code. When Synapse adopts a codebase for maintenance, the first safety problem is knowledge: the system's real behavior lives in production, in workarounds, and in edge cases nobody documented. Characterization tests capture that behavior in executable form, so later maintenance, refactoring, and modernization work proceeds with a net underneath it.

The material distills Michael Feathers' "Working Effectively with Legacy Code" (Prentice Hall, 2004), Martin Fowler's bliki, Emily Bache's writing on approval testing, and "Patterns of Legacy Displacement" by Ian Cartwright, Rob Horn, and James Lewis. Inline links point to the source for each claim.

What a Characterization Test Is ​

Feathers defines legacy code as code without tests. Tests are the dividing line because without them nobody can know quickly and verifiably whether a change made things better or worse. Synapse extends the definition pragmatically in Working with Legacy Code: legacy code is code below our current standards, since standards evolve faster than systems can keep up.

"A characterization test is a test that characterizes the actual behavior of a piece of code" (Feathers, "Working Effectively With Characterization Tests"). Feathers states the purpose in "Characterization Testing": "The purpose of characterization testing is to document your system's actual behavior, not check for the behavior you wish your system had." The same article explains why: "When a system goes into production, in a way, it becomes its own specification." Deployed behavior may have dependents, so a change to it is a change to the system's contract, whatever the original intent was.

Golden master tests, approval tests, and snapshot tests are community names for the same technique (Carlo, "What's the difference between Regression Tests, Characterization Tests, and Approval Tests?"). Fowler adds in SelfTestingCode that the value of a test suite lies in having the tests, whatever route produced them, so a suite built by characterization counts fully as a safety net.

The Legacy Code Dilemma ​

Feathers states the dilemma at the center of his book: changing code safely requires tests, and putting tests in place often requires changing code. The book resolves it with a catalog of conservative dependency-breaking refactorings (ch. 25).

Our recommended first pass on an adopted codebase stays on the observation side of that dilemma: write tests against the seams that already exist and leave production code byte-for-byte unchanged. This keeps the first pass free of changes to production behavior, and every unit that turns out to be unreachable becomes a documented finding with a suggested dependency-breaking edit for a deliberate second pass. Feathers' change algorithm (identify change points, find test points, break dependencies, write tests, make changes) still frames the larger effort; the observation-first pass performs "find test points" and "write tests" for everything reachable today.

The Sensing Algorithm ​

Feathers' procedure, from "Working Effectively With Characterization Tests", applied per behavior:

  1. Write a test that calls the unit with one specific input and an assertion you know is wrong.
  2. Run the test and read the actual value from the failure output.
  3. Replace the wrong expectation with the observed value.
  4. Rename the test to describe the behavior it documents.
  5. Rerun to green, then repeat with the next input.

Every expected value comes from observed execution. An expected value derived from documentation, comments, commit history, or a reader's judgment of intent records a guess about the system. The deliberately wrong first assertion also demonstrates that the test can fail. Carlo recommends two further checks that the suite actually pins behavior: run coverage to find inputs the tests never exercise, and introduce a deliberate bug to confirm at least one test catches it.

Seams ​

A seam is "a place where you can alter behavior in your program without editing in that place" (Working Effectively with Legacy Code, ch. 4, excerpted at InformIT). Every seam has an enabling point, the place where the decision between behaviors is made. Feathers names three kinds in that chapter: preprocessing seams (macros and conditional compilation), link seams (implementations swapped at link or import resolution time), and object seams (polymorphic calls, enabled where the object is constructed or passed in). He calls object seams "the best choice" in object-oriented languages because they are the most explicit and the easiest to maintain.

Fowler's LegacySeam (2024) shows the practical mechanisms in JavaScript and TypeScript: parameter passing (his preference), service locators, and module exports. For an observation-first pass, a seam qualifies only when its enabling point already exists in the code: function and constructor parameters, dependency injection registrations, exported and public interfaces, HTTP and CLI entry points, environment variables, and configuration files.

Sensing and Separation ​

Feathers gives two reasons to break dependencies. Sensing covers cases where the test needs access to values the code computes. Separation covers cases where the code will run inside a harness only after a dependency is removed. In an observation-first pass, sensing comes from existing seams and test-side interception (HTTP interception, fake timers, in-memory transports, temp directories). A separation problem is a stop condition: record the unit as untestable as-is and name the smallest production change that would unblock it.

Depth Heuristics ​

Feathers' guidance on how much characterization is enough, from "Working Effectively With Characterization Tests":

  1. Write tests for the area where changes will land, with as many cases as it takes to understand the behavior.
  2. Look at the specific things scheduled to change and write tests for those.
  3. When functionality will be extracted or moved, write tests that verify the existence and connection of those behaviors case by case.

He also observes in the book that many characterization tests end up as sunny-day tests. Boundaries, error paths, and odd inputs the code visibly handles deserve deliberate attention.

Effect analysis, described in the same book, locates the observation points. A change propagates along three paths: return values, mutation of parameters, and mutation of static or global state. An effect sketch, a small hand-drawn graph of what can affect what, exposes pinch points: narrow places where tests against a couple of methods can detect changes in many methods. Pinch points are high-leverage sensing locations when a whole area needs pinning.

Fowler's TestCoverage sets the role of coverage: "Test coverage is a useful tool for finding untested parts of a codebase," and a weak numeric statement of test quality, since he notes that high numbers are easy to reach with low quality testing. Read a coverage report as a list of unpinned branches.

Approval Testing Workflow ​

In "Approval Testing", Emily Bache describes the approach: arrange and act as usual while deferring the expected value. The assertion becomes a comparison against an approved artifact, and the working loop is run, diff, approve. Plain text output makes the artifacts simple to diff, version control, and merge. She calls it "a particularly powerful technique for characterization tests of legacy code" because coverage arrives fast, before anyone fully understands the code. Reference tools are TextTest (Geoff Bache) and ApprovalTests (Llewellyn Falco). Her Gilded Rose kata is a practice exercise for the workflow: run the code across a wide input range, including edge and degenerate inputs, record the golden output, then diff after every change.

Scrubbing Nondeterminism ​

A scrubber is a function that normalizes output before comparison so nondeterminism stops breaking the diff. The pattern comes from TextTest and the ApprovalTests libraries, which ship scrubbers for the common cases. Standard moves:

  • Replace timestamps and dates with placeholders, or fix the clock with fake timers.
  • Map each distinct generated id to a stable placeholder (id-1, id-2) so equal ids stay equal after scrubbing and identity relationships survive.
  • Strip or normalize absolute paths, hostnames, ports, and version strings.
  • Sort unordered collections before serializing.
  • Seed random generators through existing parameters when the code exposes one.

When a nondeterminism source has no test-side control at all, assert properties: output shape, value ranges, and invariants between fields. Record the behavior as partially pinned.

Handling Suspected Bugs ​

Characterization surfaces behavior that looks wrong. Feathers' guidance in the book: keep the test, mark it as suspicious, and find out what the effect of fixing it would be. Behavior in production may have dependents, so the fix decision involves stakeholders. Tests that pin suspected bugs stay green against the current behavior and carry a marker comment pointing to a findings list the maintainers own.

The Fallible Oracle ​

"Patterns of Legacy Displacement" (Cartwright, Horn, Lewis) warns against treating current behavior as desired behavior. Their Feature Parity pattern is the cautionary tale: "people greatly underestimate the effort required" and "archeology is often needed to fully understand what a system does." They cite a 2014 Standish Group report finding that about half the features in a typical legacy system go unused, and they warn that "workarounds for past bugs and limitations have become 'must have' requirements." In one of their airline examples, staff deliberately followed a wrong process, including printing wrong baggage tags, because the system demanded it.

The implication for characterization work: pin the behavior faithfully so change is safe, and keep a running list of characterized-but-suspect behaviors so nothing gets blessed as a requirement by accident. Their Divert the Flow pattern extends the warning to replacement projects: "many legacy reports contain undiscovered issues and bugs," so exact matching can reproduce defects. The same page recommends parallel runs and reconciliation checks, with worked examples built alongside end users to decide which system is right when outputs diverge.

Fowler's StranglerFigApplication supplies the reason the safety net matters at all: rewrites fail partly because "it's hard to figure out the details of existing behavior." A characterization suite records those details in executable form.

Dependency Breaking ​

The observation-first pass produces a list of units unreachable through existing seams. Feathers' catalog (Working Effectively with Legacy Code, ch. 25) supplies the vocabulary for the follow-up edits, each a small, deliberate production change that opens a seam:

  • Parameterize Constructor / Parameterize Method: have the caller supply the collaborator through the constructor or the method signature.
  • Extract Interface: pull the needed methods into an interface a fake can implement.
  • Subclass and Override Method: a test-only subclass replaces the problem behavior.
  • Extract and Override Call / Factory Method / Getter: isolate one problematic call behind an overridable method.
  • Adapt Parameter: wrap a hard-to-fake parameter type behind a simpler interface.
  • Break Out Method Object: move a long method into its own instantiable class.
  • Introduce Static Setter / Replace Global Reference with Getter: make global access replaceable.
  • Sprout Method / Sprout Class and Wrap Method / Wrap Class: add new, tested code around untested code when future feature work lands in the area.

Tooling ​

The characterization-test-writer skill in the synapse-claude-workflows plugin applies the observation-first discipline described here: it accepts files, functions, classes, or endpoints as targets, writes characterization tests through existing seams, and reports suspected bugs and untestable units while production code stays unchanged.

Further Reading ​