Bloom*

Engineering

SDK compatibility reports: the diff that makes migrations reversible

· 4 min read · Bloom

  • sdks
  • migration
  • compatibility

A generated SDK has a public surface that customers depend on. Method names, error classes, pagination helpers, README examples — change any of these and customer code might silently break. The way to avoid the silent break is a compatibility report: a structured diff between the SDK you're about to publish and the SDK that's currently in customers' lock files. This is short post on what's actually in a good report and how to read it.

What a compatibility report compares

For each language, the report walks the public surface of both SDKs and produces three buckets:

1. Missing things

Anything in the old SDK that doesn't exist in the new one. Examples:

  • A method (client.messages.archive) that no longer exists.
  • An exported class (AcmeAPIError) that's been renamed.
  • A pagination helper (AsyncCursorPage) that's been replaced.
  • A field on a request type (SendMessageParams.priority) that the generator dropped.

Anything in this bucket is a customer-facing break and must be resolved before publishing. The fix is either to put it back, or to publish a deliberate major-version bump.

2. Changed things

Things that exist in both but with different shape. Examples:

  • A method signature where one parameter changed from required to optional, or its type changed from string to string | null.
  • A response type that gained or dropped a field.
  • An error class whose constructor signature changed.

This is the bucket where human review matters. A required-to-optional change is usually safe. A type-narrowing change can break customer code that relied on the wider type.

3. New things

Things in the new SDK that didn't exist before. New methods, new fields, new optional parameters. These are mostly safe — they're the value-add of the new release.

The compatibility report doesn't gate on "new" items; it just lists them so you know what you're shipping.

Why teams skip the report

Three reasons, in roughly decreasing order:

  1. The previous toolchain didn't offer one, so nobody built a habit around it.
  2. Time pressure during a launch — "the new SDK works in tests, ship it."
  3. The diff is in someone's local terminal, not in the deploy workflow.

The fix to all three is to make the report a build artifact, not a one-off command. Bloom emits a compat-typescript.md and compat-python.md on every report run, and they're attached to the release in the dashboard. Reading one before a publish becomes the path of least resistance.

How to read one in five minutes

Open the markdown file. It has three sections per language (the three buckets above) plus a summary header. The five-minute scan looks like:

  1. Summary. If "missing methods" is 0, you're already in good shape. If it's not 0, the rest of the report tells you what to do.
  2. Changed methods, type-narrowing. Anything that went from a wider to a narrower type. These are the silent-break candidates.
  3. Changed errors. Error class signatures and inheritance. Customer code that catches exceptions cares about this.
  4. README example. Did the example endpoint change? Did the parameter shape change? If yes, the README example needs an update so copy-paste still works.

You can read a clean report in five minutes. A messy one takes longer, but that's the point — the report is doing the work that would otherwise show up as a customer support ticket.

A real example

From a recent Bloom report on a messaging API migration:

TypeScript: 34 methods covered, 0 missing methods, 4 changed method signatures, 12 changed response types (all additive). Error classes unchanged.

Python: 34 methods covered, 0 missing methods, 42 changed method signatures (mostly Optional[X] → X | None, which is the modern Python 3.10+ idiom). Error class hierarchy unchanged.

README example: unchanged. The example endpoint is post /api/send-message with the same parameter shape.

Verdict: ready, with 42 Python signature changes to spot-check.

The verdict is the call to action. "Ready" means a customer who upgrades will not see a breaking change. "Needs review" means there's at least one diff that requires a human decision. "Not ready" means the diff has missing methods and the migration is paused until fixed.

Where to find one

If you don't use Bloom but would like the same artifact for your own SDK, you can hand-roll the diff using tsc --declaration on both packages and a script to walk the resulting .d.ts files. We may publish a small open-source helper for this — if that would be useful, let us know.