Validating CloudFormation Templates in AWS CodePipeline

This article explains how to add cfn-lint in AWS CodePipeline using a custom AWS CodeBuild project.

CloudFormation Linter () is a static code analysis tool that validates CloudFormation YAML and JSON templates against the CloudFormation Resource Specification. Engineers who work with CloudFormation normally run this tool locally on their machine to fix any issues with the templates and correct them immediately.

However… there are use cases where you work in bigger teams and need to manage hundreds of templates. Integrating in a CI/CD pipeline helps to enforce rules over shared CloudFormation templates, which in turn makes standardizing rules and guidelines over bigger teams easier. We are going to be integrating in AWS CodePipeline using a custom AWS CodeBuild project.

Overview

In this example, we have an AWS CodePipeline that contains the source code including the CloudFormation templates in an AWS CodeCommit repository. Next up we have the Validation stage where we have an AWS CodeBuild action that contains the tool. The templates will be scanned and the report will be sent to an encrypted S3 bucket. The last stage contains the actual deployment after the validation has finished.

Pipeline

If we’re building the pipeline it looks as follows (snippet):

CodePipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
ArtifactStore:
Location:
Ref: ArtifactStore
Type: S3
Name: !Ref CloudFormationPipelineName
RoleArn: !GetAtt CodePipelineRole.Arn
RestartExecutionOnUpdate: False
Stages:
- Name: Source
Actions:
- Name: GetSource
ActionTypeId:
Category: Source
Owner: AWS
Version: 1
Provider: CodeCommit
OutputArtifacts:
- Name: source
Configuration:
BranchName: master
RepositoryName: cloudformation-codecommit-repo
- Name: Validation
Actions:
- Name: Linter
RunOrder: 1
InputArtifacts:
- Name: source
ActionTypeId:
Category: Build
Owner: AWS
Version: 1
Provider: CodeBuild
Configuration:
ProjectName: !Ref CodeBuildLinter
- Name: Deploy
Actions:
- Name: Dev
RunOrder: 1
InputArtifacts:
- Name: source
ActionTypeId:
Category: Build
Owner: AWS
Version: 1
Provider: CodeBuild
Configuration:
ProjectName: !Ref CodeBuildDev
ArtifactStore:
Type: AWS::S3::Bucket
Properties:
BucketEncryption:
ServerSideEncryptionConfiguration:
- ServerSideEncryptionByDefault:
SSEAlgorithm: aws:kms

The CloudFormation template snippet contains the three stages that were mentioned before including an artifact bucket to save the artifacts from the build.

CodeBuild

The most important part of this pipeline is the validation stage which contains the CodeBuild step:

CodeBuildLinter:
Type: AWS::CodeBuild::Project
Properties:
Artifacts:
Type: CODEPIPELINE
Description: Linting CloudFormation templates and sending reports to S3
Environment:
ComputeType: BUILD_GENERAL1_SMALL
EnvironmentVariables:
- Name: AWS_ACCOUNT_ID
Value: !Sub ${AWS::AccountId}
- Name: CFN_LINT_BUCKET
Value: !Ref CfnLintBucket
- Name: CFN_LINT_KMS_KEY_ARN
Value: !Ref CfnLintKmsKeyId
- Name: CODE_PIPELINE_NAME
Value: !Ref ProvisioningPipelineName
Image: aws/codebuild/standard:3.0
Type: LINUX_CONTAINER
ServiceRole: !GetAtt CodeBuildRole.Arn
Source:
Type: CODEPIPELINE
BuildSpec: buildspec-linter.yml
TimeoutInMinutes: 20

The tool is added as a CodeBuild action in the validation stage of this pipeline. We add additional environment variables like the AWS Account ID, S3 report bucket, KMS Key and the pipeline name. The next step is to set up the buildspec of the CodeBuild project and add the container with the commands that need to be executed:

version: 0.2
phases:
install:
runtime-versions:
python: 3.8
commands:
- apt-get update -y
- apt-get install -y uuid-runtime
- pip install cfn-lint
build:
commands:
- |
REPORT=$(uuidgen)-cfnlint-report.json
set +e; cfn-lint -f json > $REPORT
aws s3 cp $REPORT s3://$CFN_LINT_BUCKET/$AWS_ACCOUNT_ID/$CODE_PIPELINE_NAME/$(date +'%Y')/$(date +'%m')/$(date +'%d')/ --sse=aws:kms --sse-kms-key-id $CFN_LINT_KMS_KEY_ARN --acl bucket-owner-full-control

First, we install the required packages in the container. Next, we run and make sure the command won't exit ( ) when it reports an error. In this situation, we don't want to stop exit CodeBuild if we discover an error because we use only for reporting.

In this setup, I’ve included a configuration file which specifies what templates need to be analyzed.

After the analysis, it will generate a report in JSON format which will be uploaded to an encrypted S3 bucket. As you can see the S3 CLI command is being used to transfer the report to the S3 bucket. We’ve included the variables which we passed from the resource to dynamically create the object and enable KMS Encryption. After the report is uploaded to S3 it is ready for consumption for other applications.

KMS & S3 Bucket

If you enable AWS KMS, make sure to include the following policies for the CodeBuild role:

The CodeBuild role is allowed to put the report in the destination bucket (CfnLintBucket). We also allow the role to encrypt the object using the KMSKey we created for the S3 Bucket. Then we need to explicitly allow the CodeBuildRole on the KMS Key policy, we can do that as follows:

To Summarize

We’ve set up a CodePipeline which adds cfn-lint as a validation stage to analyze your CloudFormation templates and send the reports to an encrypted S3 Bucket. This allows auditors for example to check how well the templates are built according to best practices.

👋 Liked this article? Follow me on Twitter to stay updated!

Originally published at https://dannys.cloud.

AWS Cloud Consultant, Writer, Community Builder.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store