Schema versioning

This document describes how schemas produced by fmu-dataio are versioned, under what circumstances version numbers change, and provides criteria for the kind of version change (patch, minor, major) a schema change causes.

Note

The version of fmu-dataio is decoupled from the versions of schemas it produces. This means that a new version of fmu-dataio may cause no schema changes at all. Similarly, a new version of fmu-dataio may cause just one schema version to change, or even all of them.

There is no direct relationship between an fmu-dataio version and schema versions aside from the fact that a schema version may have changed with a new fmu-dataio version.

Schema version changes

Schema versions follow semantic versioning. This gives every schema version a specific number in the form X.Y.Z where

  • X is the major version number,

  • Y is the minor version number,

  • Z is the patch version number (sometimes called the micro version).

Schema version numbers change when a schema is changed. When deciding what version a changed schema should become the primary concern should be whether or not the change being made is backward compatible.

Contractual fields

Fields that are marked as contractual in the main metadata schema require extra considerations before changes can be applied to them. These fields are used both for indexing in Sumo and may have a larger importance to difference consumers.

Backward compatibility

For metadata generated by fmu-dataio backward compatibility can mean multiple things:

  • Metadata generated by a newer version can validate metadata generated by a previous version

  • Metadata generated by a newer version remains compatible with logic possibly implemented by consumers of this metadata

These two properties are in some cases not possible to maintain at the same time. Example:

  • A new version makes a required field optional

    • ✅ Newer versions of the schema can still validate older versions

    • ⛔️ Consumers depending on the existence of this field now break if it is nulled

In the general case a breaking change is taken to mean that consumers of data no longer have the same guarantees about data they depend on.

Backwards compatibility is broken if some metadata valid with the new schema is not valid against the previous version of the schema.

Major

Any schema change that breaks backward compatibility with metadata created using the previous schema version. These scenarios are candidates for a major version change:

  • Making a required field optional

  • Removing a required or optional field

  • Removing a regular expression or validation from a field

  • Changing the name of a field

    • Example: datetime is changed to timestamp

  • Changing the type of a field

    • Example: A float field is changed is changed to a float string

  • Changing the data format of a field

    • Example: A date field is changed from Unix timestamp to an ISO 8601 string

  • Splitting or merging fields

    • Example: A version field is changed to major, minor, patch (or vice versa)

  • Removing a value from a controlled vocabulary

    • Example: OWC is no longer a valid contact (unlikely, but an example!)

  • Changing the cardinality of field

    • Example: An array field of length 4 becomes an array of length 8

Minor

Any schema change that retains backward compatibility with metadata created using the previous version.

  • Making an optional field required

  • Adding a new required field

  • Adding an optional field

  • Adding a regular expression or stricter validation to a field

    • Example: A string field applies a maximum length validation check

  • Changing a field from a controlled vocabulary to free text without changing the field type

  • Adding alternative formats for the same data

    • Example: "YYYY-MM-DD" exists, and now "YYYY/MM/DD" is allowed

  • Adding a computed or derived field that is not required to be present.

    • Example: A sha1 hash is computed on an object metadata is representing

    • Example: A deterministic UUID is computed on an object and its metadata

Patch

Any change to auxiliary information that does not affect the structure or semantics of the schema itself. Also, any bug fixes to the schema.

  • Adding or updating the field description to improve readability, add context, or correct an error

  • Adding or updating the field example, comment, or user-friendly name

  • Extending a controlled vocabulary enumeration

  • Fixing an incorrect regular expression or validation