We are excited to announce the new Build Config Validation feature. It is now in beta, and available for you to activate.

When active, the build config validation feature will validate and normalize any incoming build config sources, your .travis.yml file and configs from build requests created via API.

We are hoping to give you more clarity about the build config format and ultimately a better experience when activating a repository and enabling certain features.

You can enable this feature using the repository setting “Build config validation” in the Travis CI UI, or by specifying the version in your .travis.yml file:

version: ~> 1.0

The validation process produces validation messages that you can review on the respective build’s “View Config” tab in the Travis CI UI. This gives you direct insights into how your build config has been processed, what issues Travis CI might have found, and how to resolve them:

It uses a formal config format specification which comes in the well known and widely supported format JSON Schema.

This feature replaces any previous linting and validation tooling.

The API endpoint /lint, which is used by our CLI command travis lint has been updated to use the same endpoint that is also used by our system in order to validate and normalize your build configs.

The same specification is also used to autogenerate reference documentation, which can be found at config.travis-ci.com.

Because our system uses the same endpoint and specification to process your build configs, both the travis lint command and the reference documentation are guaranteed to always be up to date.

We have also added an experimental Travis CI build config explorer that you can use to try out and check how your build config will be normalized and validated, and eventually expanded into build job configs.

For questions and discussion, we would love to see you in the Travis CI community forum. If you see any missing functionality or find bugs please open an issue here.

Changes

The goal is to make the build config processing service 100% transparent. Ideally, not a single build would break or behave differently. On top of coming with an extensive unit and acceptance test suite, the library is also validated against over 9000 actual build configs from power user repos.

However, our build config format allows for many different forms on various config keys, and not all of these behave completely consistently throughout the system. Therefore, while we now introduce a normalization layer that applies the same patterns to all keys consistently, there is a slight chance that some case might have slipped through.

One change that has been made intentionally is moving from matrix to jobs. So far, both the keys matrix and jobs were functionally equivalent, and jobs was an alias to matrix. We are now settling with jobs being the main, recommended key, and matrix being an alias for it. If you are using matrix in your build config you will see an informative validation message.

In order to make things more consistent, we have also introduced the key env.jobs, and made the previous env.matrix an alias for it.

Release Plan

We normally do not publish release plans in detail, but as this feature affects everyone’s build configs we want to give full transparency. The following plan is still preliminary, and can be halted at any time if needed.

• Starting today, the build config validation feature is available for opt-in as a beta feature.
• Starting November 15th, we will enable this feature for new repositories, allowing to opt-out in the event of major issues.
• Starting January 15th, we will start enabling it incrementally for existing repositories, rolling back in time.

We will communicate the current status of this on our docs, changelog, and via Twitter.

Highlights

The Build Config Validation:

• Produces a JSON Schema specification for Travis CI’s build config format
• The specification is defined using a user friendly, descriptive and readable DSL, hopefully making it possible for external maintainers (e.g. language maintainers) to contribute easily.
• Uses this same specification to normalize and validate actual build configs as part of the build configuration process.
• Enforces well known structural patterns everywhere, rather than hopefully relying on various components to apply the proper normalizations (e.g. if a node accepts an array of strings, always also accept a string, and wrap it into an array, etc.)
• Is rigorous with the structure, but lenient with the content.
• Keeps unknown keys, unknown value, and values missing required keys by default. Dropping such keys and values is parametrized, and can be turned on.
• Emits structured messages that clients can use to display rich information about error and warning conditions, deprecations, and and info level notices. These include line numbers and the config source name (in case of multiple config sources, e.g. with imports or API config payloads). [1]
• Supports minimal spellchecking and correction for typos, potentially sparing users a lot of frustration when typing language correctly (see keys.dict for just a few common typos).
• Alert messages when a plain string is used on a node that expects a secure.
• Heavily tested: ~4000 acceptance and unit tests, ~1100 tests against YAML examples from our documentations, ~9000 tests against public .travis.yml files from actual builds, ~200 tests against the deployment tooling.
• Custom YAML Psych parser, using Psych’s official api for safe YAML parsing: puts an end to users (and our docs) having to quote version numbers that would be parsed into floats by standard YAML parsers (e.g. go: "1.10"), and the key deploy.on being turned into deploy.true.
• New and improved shell variable parser, tested against 20K actual env vars from abovementioned public build configs.

We are excited to see how you are using the new build config validation feature and are looking forward to reading your feedback in the community forum. Happy Building!