Skip to content

Latest commit

 

History

History
189 lines (149 loc) · 13.3 KB

File metadata and controls

189 lines (149 loc) · 13.3 KB

Breaking Changes and Changelog

As your API evolves, it undergoes changes. Some of these changes may be "breaking" while others are not.
The oasdiff breaking command displays the changes that break existing API clients.
The oasdiff changelog command displays the changes that can affect API consumers, both breaking and non-breaking. Documentation-only edits, such as descriptions and examples, are not included; use oasdiff diff to see every difference in the API definition.
These commands are typically used in the CI to report or prevent breaking changes.

Note: breaking and changelog run on the diff engine described in DIFF.md. The flags and concepts there — extension tracking, path matching, allOf flattening, the path-prefix family, header case, --allow-external-refs, --fail-on-diff — apply here too, in addition to the breaking-specific options below.

For teams: oasdiff.com wraps these commands with a per-change PR comment, one-click approve/reject buttons, and commit-status checks — so reviewers can act on each breaking change directly from the pull request.

Example: display breaking changes

oasdiff breaking https://raw.githubusercontent.com/oasdiff/oasdiff/main/data/openapi-test1.yaml https://raw.githubusercontent.com/oasdiff/oasdiff/main/data/openapi-test3.yaml

Example: display a changelog

oasdiff changelog https://raw.githubusercontent.com/oasdiff/oasdiff/main/data/openapi-test1.yaml https://raw.githubusercontent.com/oasdiff/oasdiff/main/data/openapi-test3.yaml

Side-by-side review in your browser (--open)

Both breaking and changelog accept an --open flag. After printing the usual terminal output, the CLI uploads the comparison to oasdiff.com and opens the rendered side-by-side review in your browser, with the diff anchored at each change site. The approve/reject buttons are shown but disabled; they are enabled only with oasdiff Pro.

oasdiff changelog HEAD~1:openapi.yaml HEAD:openapi.yaml --open

The comparison is encrypted on your machine before upload, with a one-time key that lives only in the review link (the part after the #) and is never sent to our servers, so what we store we cannot read. There is no sign-in and no account.

The resulting URL works for 7 days. Paste it into a PR comment or Slack and reviewers can open the review without installing the CLI themselves. Anyone you give the full link to can open it (the part after the # is the decryption key), so treat it like a secret.

Checks

Oasdiff supports hundreds of checks (run oasdiff checks changelog for the current list, or browse the full catalog at oasdiff.com/docs/breaking-changes), categorized into three levels:

  • ERR - Errors are definite breaking changes which should be avoided
  • WARN - Warnings are potential breaking changes which developers should be aware of, but cannot be confirmed programmatically as breaking
  • INFO - Non-breaking changes

oasdiff breaking detects changes with level ERR and WARN only.
oasdiff changelog detects changes with levels that are greater or equal to the --level argument. The default level, INFO, includes all checks.

To see the full list of checks and their descriptions, run:

oasdiff checks changelog

See also Customizing Severity Levels

How oasdiff decides what is breaking

oasdiff judges a change against the API contract your OpenAPI definition declares, not against what a particular server happens to accept. A change is breaking if a consumer that followed the old contract can stop working under the new one.

This matters because an OpenAPI definition declares which requests and responses are valid, but most servers do not enforce it at runtime, and a server may quietly accept a request that the contract says is invalid. oasdiff still reports the change as breaking, because other consumers of the same contract do enforce it: API gateways and validators reject non-conforming requests, and generated client SDKs turn the contract into typed code that no longer compiles. Whether your own server is lenient is your choice to make; it does not mean the contract is unchanged.

Two consequences worth knowing:

  • A change that makes a previously valid request invalid is breaking even if your server would still accept it. For example, adding a required request property is breaking: a request that omits it is invalid under the new contract, whether or not the property has a default. A default is a server-side fallback value; it does not make an omitted required property valid.
  • WARN is reserved for changes where the definition genuinely does not contain enough information to decide, such as a change to an object parameter whose serialization the definition leaves unspecified. It is not used for changes that break some setups and not others; those are errors.

If a check's default severity does not match your API's compatibility policy, change it with Customizing Severity Levels.

Preventing Breaking Changes

A common way to use oasdiff is by running it as a step the CI/CD pipeline to detect changes.
In order to prevent changes, oasdiff can be configured to return an error if changes above a certain level are found.

  • To exit with return code 1 if ERR-level changes are found, add the --fail-on ERR flag.
  • To exit with return code 1 if ERR-level or WARN-level changes are found, add the --fail-on WARN flag.
  • To exit with return code 1 if any changes are found, add the --fail-on INFO flag.

For example:

oasdiff breaking --fail-on ERR data/openapi-test1.yaml data/openapi-test3.yaml

Checking many specs at once

To check a list of changed specs against their versions in a git ref, one comparison per spec, see BREAKING-FILES.md. That is what the pre-commit hook uses.

Output Formats

By default, oasdiff displays changes in a human-readable colorized text format.
Additional formats can be generated using the --format flag:

  • json
  • yaml
  • githubactions: suitable for integration with github
  • junit: suitable for integration with gitlab
  • html: see example
  • markdown: see example
  • text: the default, human-readable, format
  • singleline: displays each change on a single line, this can be useful to prepare ignore files

For example:

oasdiff breaking -f yaml https://raw.githubusercontent.com/oasdiff/oasdiff/main/data/openapi-test1.yaml https://raw.githubusercontent.com/oasdiff/oasdiff/main/data/openapi-test3.yaml

JSON Schema

oasdiff schema prints a JSON Schema for the json output of breaking and changelog (the yaml output has the same structure):

oasdiff schema

Color

When outputting changes to a Unix terminal, oasdiff automatically adds colors with ANSI color escape sequences.
If output is piped into another process or redirected to a file, oasdiff disables color.
To control color manually, use the --color flag with always or never.

API Stability Levels

Assigning stability levels to APIs allows fine-grained control over how APIs are allowed to change based on their maturity.

Deprecating APIs

Before deleting an endpoint, it is recommended to give consumers a heads-up in the form of "deprecation". Oasdiff allows you to deprecate APIs gracefully without triggering a breaking-change error.

Version Bumps

If you use semantic versioning, oasdiff can report a breaking change that was released without a major version bump. See Version Bumps and Breaking Changes.

Nullability Changes

A schema can allow null in three equivalent ways, and whether a nullability change is breaking depends on whether it appears in a request or a response. See Nullability Changes.

Ignoring Specific Breaking Changes

Sometimes, you want to allow certain breaking changes, for example, when your spec and service are out-of-sync and you need to correct the spec.
Oasdiff allows you define breaking changes that you want to ignore in a configuration file.
You can specify the configuration file name in the oasdiff command-line with the --warn-ignore flag for WARNINGS or the --err-ignore flag for ERRORS.
Each line in the configuration file should contain two parts:

  1. Method and path (the first field in the line beginning with slash) to ignore a change to an endpoint, or the keyword 'components' to ignore a change in components
  2. Description of the breaking change

For example:

GET /api/{domain}/{project}/badges/security-score removed the success response with the status '200'

Or, for a component change:

components removed the schema 'rules'

The required parts may appear in any order, in lower or upper case, and the configuration line may contain additional text, like this:

 - 12.01.2023 In GET /api/{domain}/{project}/badges/security-score we removed the success response with the status '200'
 - 31.10.2023 Removed the schema 'network-policies' from components

The configuration files can be of any text type, e.g., Markdown, so you can use them to document breaking changes and other important changes.

Breaking Changes to Enum Values

Oasdiff supports special rules for enum changes using the x-extensible-enum extension.
This method allows adding new entries to enums used in responses which is very usable in many cases but requires clients to support a fallback to default logic when they receive an unknown value. x-extensible-enum was introduced by Zalando and picked up by the OpenAPI community. Technically, it could be replaced with anyOf+classical enum but the x-extensible-enum is a more explicit way to do it.
In most cases the x-extensible-enum is similar to enum values, except it allows adding new entries in messages sent to the client (responses or callbacks). If you don't use the x-extensible-enum in your OpenAPI specifications, nothing changes for you, but if you do, oasdiff will identify breaking changes related to x-extensible-enum parameters and properties.

Localization

To display changes in other languages, use the --lang flag.
Currently English, Russian and Brazilian Portuguese are supported.
Please improve oasdiff by adding your own language.

Customizing Severity Levels

Oasdiff allows you to change the default severity levels according to your needs.
For example, the default severity level of the api-security-removed check is INFO. You can verify this by running oasdiff checks changelog.
To change the api-security-removed check's severity level to ERR use the following command:

oasdiff changelog data/checker/api_security_added_revision.yaml data/checker/api_security_added_base.yaml --severity-levels oasdiff-levels.txt

Where the file oasdiff-levels.txt contains a single line:

api-security-removed    err

Checks can be customized with the following levels:

Custom Level Check Status
err Enabled with level ERR
warn Enabled with level WARN
info Enabled with level INFO
none Disabled

When oasdiff reports a change below its check's level

The level a check declares, which is what oasdiff checks changelog lists, is what the change means in general. For a specific change, oasdiff may report a lower level, with a comment explaining why. This happens for two reasons:

  • The comparison was not exact. A change inside an allOf is the common case: without --flatten-allof the branches are compared one at a time, so another branch may still guarantee what one branch dropped. Such a change is reported at WARN even where its check declares ERR, and the comment says what was missing and what to run to get the exact answer.
  • The spec shows the change cannot break anyone. A property marked readOnly never appears in requests, so restricting it (adding a pattern, setting a maximum) cannot invalidate a request; the same holds for writeOnly properties in responses. Such a change is reported at INFO even where its check declares ERR, and the comment states the reason. The conditions a check recognizes are listed in its GUARDS column in oasdiff checks changelog (see CHECKS.md).

Setting a level yourself turns this off for that check. The level you set is the level you get, whether or not one of the reasons above applies. This matters in both directions: pinning a check to err means it will report ERR even where oasdiff would otherwise have lowered it, and pinning one to the level it already has is enough to opt out.

Customizing Breaking Changes Checks

If you encounter a change that isn't reported, you may:

  1. Run oasdiff checks changelog to see if the check is available, and customize the level as needed.
  2. Add a custom check

Known Limitations

  • no checks for context instead of schema for request parameters
  • no checks for callbacks