@megabytelabs/codeclimate-yamllint

CodeClimate and Standalone Linter for YAMLLint

A performance-optimized CodeClimate engine maintained by Megabyte Labs

A slim YAMLLint container that can also serve as a GitLab CI-ready CodeClimate engine

Table of Contents

• Overview
  • Standalone
  • Slim Build
  • Versioning
  • Testing
• Requirements
  • Development Requirements
• Examples
  • Integrating with GitLab CI
    • CodeClimate GitLab CI Example
    • CodeClimate CLI Walkthrough
    • Standalone Integration
• More Information About YAMLLint
• Pairing with ESLint
  • Building the Docker Container
  • Building a Slim Container
  • Build Tools
• Philosophy
• Contributing
  • Affiliates
• License

Overview

This repository is home to a Dockerfile project that generates a GitLab CI-ready CodeClimate engine for YAML Lint and a compact slim container suitable for standalone linting with YAMLLint. YAMLLint is arguably the standard for linting YAML files. It is written in Python and checks for syntax validity, key repitition, and cosmetic problems such as line lengths, trailing spaces, and indentation.

This project was created to integrate YAMLLint results into the GitLab web GUI. GitLab accepts results in a format that is very similar to the CodeClimate format. This repository also serves a second purpose and that is to be able to generate a tiny Docker container that is capable of running YAMLLint very quickly in any CI setting.

Standalone

If you are interested in using the tool and have no need for CodeClimate integration, you can get the latest, slim, and versioned images without CodeClimate-related code by removing the codeclimate- string from the image name. For example, if the image is named megabytelabs/codeclimate-yamllint, then you can get the same image without CodeClimate-related code by using the megabytelabs/yamllint image. For a full listing of images to choose from, check out the DockerHub page for this project.

Note: The DockerHub page for the CodeClimate engine version of this project is located on the aforementioned DockerHub page and also on a separate DockerHub page prefixed with codeclimate-.

Slim Build

All of our CodeClimate engine projects include two variants. One is the regular build that is created using Docker and the Dockerfile that is in the root of this project. The other build is a slim build that is built via DockerSlim. If you are looking for stability and want to take advantage of the slim build, then you should specify an exact version when referencing the image (e.g. megabytelabs/codeclimate-package:5.4.0-slim instead of megabytelabs/codeclimate-package:slim).

Versioning

Whenever possible, we tag our Docker image versions with the corresponding version of the software that the Docker image is used for. For instance, if you are using the ESLint CodeClimate image, then the megabytelabs/codeclimate-eslint:5.4.0 image means that it likely includes ESLint at version 5.4.0. There may be some exceptions but this is what we strive for.

Testing

Multiple methods of testing are used to ensure both the latest and slim build function properly. The Dockerfile-test.yml file in the root of the repository is a container-structure-test that ensures that each container is working properly. On top of that, we also run commands on sample projects stored in test/compare/ to ensure that the output from the latest image matches the output from the slim image. In some other scenarios, we also include unit tests for custom code written for the CodeClimate engine.

Requirements

• Docker

Development Requirements

• DockerSlim - Used for generating compact, secure images
• jq - Used for interacting with JSON
• Node.js (>=14.18) - Utilized to add development features like a pre-commit hook and maintenance tasks
• Many more requirements that are dynamically installed as they are needed by our Taskfile.yml via our custom go-task/task fork named Bodega

If you choose to utilize the development tools provided by this project then at some point you will have to run bash start.sh (or npm i which calls bash start.sh after it is done). The start.sh script will attempt to automatically install any requirements (without sudo) that are not already present on your build system to the user's ~/.local/bin folder. The start.sh script also takes care of other tasks such as generating the documentation by calling tasks defined in the Taskfile.yml. For more details on how the optional requirements are used and set up, check out the CONTRIBUTING.md guide.

When you are ready to start development, run task --menu to open an interactive dialog that will help you understand what build commands we have already engineered for you.

Examples

There are several different ways you can use the Docker container provided by this project. For starters, you can test the tool out locally by running:

docker run -it -v ${PWD}:/work -w /work --rm megabytelabs/yamllint:latest --help

This allows you to run YAMLLint without installing it locally. It also removes the image from your system when you are done. This could be good for security since the application is within a container and also keeps your file system clean.

You can also add a bash alias to your ~/.bashrc file so that you can run the YAMLLint command at any time. To do this, add the following snippet to your ~/.profile file (or equivalent):

yamllint() {
  docker run -it -v ${PWD}:/work -w /work --rm megabytelabs/yamllint:latest "$@"
}

Note: Some CLI tools run without any arguments passed in. For example, the CLI tool ansible-lint runs by simply entering ansible-lint in the terminal. Our Docker images default command is to show the version so to get around this quirk you would run ansible-lint ..

Integrating with GitLab CI

The main purpose of this project is to build a Docker container that can be used in CI pipelines and during GitLab CI CodeClimate analysis.

CodeClimate GitLab CI Example

If you are interested in improving the GitLab web UI for your merge requests, then you can take a peek at our GitLab CI configuration for CodeClimate. It has to run in a single CI stage that includes all the linters you want to report on because GitLab CI only accepts one CodeClimate report artifact.

CodeClimate CLI Walkthrough

If you want to incorporate CodeClimate into your project, you need to ensure that you have a .codeclimate.yml properly setup in the root of your project. At the very minimum, the file might look something like this:

---
engines:
  yamllint:
    enabled: true
exclude_paths:
  - 'node_modules/**'

Also, before you run the CodeClimate CLI, you need to ensure that this project's CodeClimate image is pulled to your local cache and properly tagged for CodeClimate. You can accomplish this by running:

docker pull megabytelabs/codeclimate-yamllint:latest
docker tag megabytelabs/codeclimate-yamllint:latest codeclimate/codeclimate-yamllint:latest

After that, you need to invoke the CodeClimate CLI by passing the --dev parameter. This may seem hacky but it is the only way of using CodeClimate engines that are not officially hosted in the codeclimate namespace on DockerHub. Your CLI command might look something like this:

brew install codeclimate/formulae/codeclimate
codeclimate analyze --dev

Or if you want to see an HTML report:

brew install codeclimate/formulae/codeclimate
codeclimate analyze --dev -f html > codeclimate-report.html

Standalone Integration

If you want to incorporate this CI pipeline tool into GitLab CI project without CodeClimate integration then your first step would be to create a .gitlab-ci.yml file in the root of your GitLab repository. Your .gitlab-ci.yml file should look something like this:

---
stages:
  - lint

include:
  - remote: https://gitlab.com/megabyte-labs/gitlab-ci/-/raw/master/lint/yamllint.gitlab-ci.yml

That is it! YAMLLint will now run anytime you commit code (that matches the parameters laid out in the remote: file above). Ideally, for production, you should copy the source code from the remote: link above to another location and update the remote: link to the file's new location. That way, you do not have to worry about any changes that are made to the remote: file by our team.

More Information About YAMLLint

For those of you who are unaware, YAMLLint is a CLI tool that lints YAML files. It is well-received by the community and should be in anyone's arsenal when it comes to linting YAML files. It reports various issues that, when tended to, help reduce errors and improve the readability of YAML files.

Pairing with ESLint

Sadly, YAMLLint does not have the ability to auto-fix/sort *.yml files. For that, you can use another JavaScript-based tool called ESLint. It can automatically fix some of the issues that YAMLLint reports. It can also sort your YAML files to your liking. It does require a decent amount of configuration though. If you would like to see some great examples that are ready for use, check out our ESLint shared configuration named eslint-config-strict-mode and our GitLab-friendly CodeClimate engine for ESLint.

Building the Docker Container

You may have a use case that requires some modifications to our Docker image. After you make changes to the Dockerfile, you can upload your custom container to Docker Hub using the following code:

export DOCKERHUB_USERNAME=Your_DockerHub_Username_Here
export DOCKERHUB_PASSWORD=Your_DockerHub_Password_Here
docker login -u "$DOCKERHUB_USERNAME" -p "$DOCKERHUB_PASSWORD" docker.io
docker build --pull -t "$DOCKERHUB_USERNAME/yamllint:latest" .
docker push "$DOCKERHUB_USERNAME/yamllint:latest"

After setting your DockerHub username and password, the commands above will build the Docker image and upload it to Docker Hub where it will be publicly accessible. You can see this logic being implemented as a GitLab CI task here. This GitLab CI task works in conjunction with the .gitlab-ci.yml file in the root of this repository.

Building a Slim Container

Some of our repositories support creating a slim build via DockerSlim. According to DockerSlim's GitHub page, slimming down containers reduces the final image size and improves the security of the image by reducing the attack surface. It makes sense to create a slim build for anything that supports it, including Alpine images. On their GitHub page, they report that some images can be reduced in size by up to 448.76X. This means that if your image is naturally 700MB then it can be reduced to 1.56MB! It works by removing everything that is unnecessary in the container image.

As a convenience feature, we include a command defined in package.json that should build the slim image. Just run task docker:build after running npm i (or bash start.sh if you do not have Node.js installed) in the root of this repository to build both the latest and slim builds.

To build and publish a slim Dockerfile to Docker Hub manually, you can use the following as a starting point:

export DOCKERHUB_USERNAME=Your_DockerHub_Username_Here
export DOCKERHUB_PASSWORD=Your_DockerHub_Password_Here
docker login -u "$DOCKERHUB_USERNAME" -p "$DOCKERHUB_PASSWORD" docker.io
docker build -t "$DOCKERHUB_USERNAME/yamllint:latest" .
docker-slim build --tag $DOCKERHUB_USERNAME/yamllint:slim {{#if (eq (typeOf dockerSlimCommand) "string")}}dockerSlimCommand}}{{/if{{#if (not (eq (typeOf dockerSlimCommand) "string"))}}dockerSlimCommand[slug]}}{{/if $DOCKERHUB_USERNAME/yamllint:latest
docker push "$DOCKERHUB_USERNAME/yamllint:slim"

It may be possible to modify the DockerSlim command above to fix an issue or reduce the footprint even more than our command. You can modify the slim build command inline in the package.json file under blueprint.dockerSlimCommand. Some of our repositories have multiple build targets in the Dockerfile so those repositories will have multiple dockerSlimCommands.

If you come up with an improvement, please do open a pull request. And again, make sure you replace DOCKERHUB_USERNAME and DOCKERHUB_PASSWORD in the snippet above with your Docker Hub username and password. The commands in the snippet above will build the slim Docker image and upload it to Docker Hub where it will be publicly accessible.

Build Tools

You might notice that we have a lot of extra files considering that this repository basically boils down to a single Dockerfile. These extra files are meant to make team development easier, predictable, and enjoyable. If you have a recent version of Node.js installed, you can get started using our build tools by running npm i (or by running bash start.sh if you do not currently have Node.js installed) in the root of this repository. After that, you can run task --list to see a list of the available development features. Alternatively, you can run task --menu to view an interactive menu that will guide you through the development process.

Note: We use a custom-built version of go-task/task so if you already have it installed then you should either replace it with our version or use a different bin name for task.

For more details, check out the CONTRIBUTING.md file.

Philosophy

Utilizing Continuous Integration (CI) tools can improve developer efficiency drastically. They allow you to do things like scan new code for possible errors and automatically deploy new software.

This repository is home to the build instructions for a Docker container that is just one piece to the CI puzzle. Nearly all of our CI pipeline Docker projects serve a single purpose.

Instead of using one of the countless pretty_name public Docker images available, we create it in-house so we know exactly what code is present in the container. We also ensure that all of our CI pipeline images are as small as possible so that our CI environment can download and run the specific task as quickly as possible. Using this repository as a base, you too can easily create your own in-house CI pipeline container image.

At first glance, you might notice that there are many files in this repository. Nearly all the files and folders that have a period prepended to them are development configurations. The tools that these files and folders configure are meant to make development easier and faster. They are also meant to improve team development by forcing developers to follow strict standards so that the same design patterns are used across all of our repositories.

Contributing

Contributions, issues, and feature requests are welcome! Feel free to check the issues page. If you would like to contribute, please take a look at the contributing guide.

Sponsorship

Dear Awesome Person,

I create open source projects out of love. Although I have a job, shelter, and as much fast food as I can handle, it would still be pretty cool to be appreciated by the community for something I have spent a lot of time and money on. Please consider sponsoring me! Who knows? Maybe I will be able to quit my job and publish open source full time.

Sincerely,

Brian Zalewski

Affiliates

Below you will find a list of services we leverage that offer special incentives for signing up for their services through our special links:

License

Copyright © 2020-2021 Megabyte LLC. This project is MIT licensed.