20200203 - CfgMgmtCamp 2020 - Head in the Clouds: Testing Infra as Code, Peter Souter

Head in the Clouds: Testing Infra as Code, Peter Souter

Treating the tooling that manages your infrastructure the same way as you would manage your code.

gives the following benefits:

• easy to regenerate from scratch
• code review and collaboration
• easier to conceptualise as a code model
• auditing and polices
• iteratively improve over time
• modularise
• testing: makes all the above easier

a little bit of bias towards Puppet and Terraform

Configuration management tools install and configure software on machines that already exist.
Terraform is not a configuration management tool => provisioning tool

How do you test things you don't have control over (cloud) and without spending too much money.

Two types of people interested in testing IaC and it's interactions with DA Cloud: Producers and Consumers.

Testing pyramid vs Testing Ice-cream Cone (anti-pattern)

Unit Tests: if you know Cornetto, at the bottom you have the small cone with chocolate, this is what you want.

Let's start with the most simple "tests" ...

compiling and type checking
- terraform validate
- ansible-playbook --check

Linting

• tflint
• ansible-lint

Ok, so now let's get deeper: Unit Tests

Unit Tests ensure you build the thing right.
Acceptance Tests ensure you build the right thing.

What is not a unit test: see @mfeathers rules

Don't just write tautological unit tests, test the edge cases.

Producers and consumers have different intents for unit tests.

for Terraform: not yet something official

Some possible contenders:
- clarity: Go-lang, gherkin style, BDD, parses HCL to run its checks
- terraform-compliance: Python cli, parses plan outputs for its checks
- terraform-validate: Python, standard unit testing, parses HCL for its checks

pros: fast
cons: doesn't test the actual outcome

The hard part: integration testing and acceptance testing

How can you test something you don't control?

approach 1: Mock the cloud API's - fixture mocking

pretty close to unit tests, they are not really interacting with the real thing

example: ACME provider - we don't want to fire requests to a real CA provider
- Go lang has an HTTP api + expect some HTTP response

pros: fast, close to unit testing
cons: you don't test the real thing

approach 2: simulate cloud

• Azurite
• DynamoDB Local
• GCP Emulator
• Localstack: Cloud in a Box, provides real working versions of AWS API, free version has 80%

cons: won't exist for every service, relies on simulated service to keep in line with real cloud, still not really testing an actual cloud

approach 3: don't mock anything, use the real thing

this is what Terraform does internally
takes a pretty long time, and it will cost some amount of money (depending on how you do it)

as a producer, you test the tool
as a consumer, you test the final outcome and ultimately the state

• terratest: full acceptance framework for Terraform, developed by Gruntworks

• Test Kitchen: tests after deployment and logs in into the System under Test

pros: pretty good end-to-end testing
cons: ...

Two additional tests above conventional testing

• Observability
• Governance: naming conventions, tagging, price guides, legal requirements and regulations, ...

Hashicorp Sentinel => Policy as Code, example: you may not allow security groups from 0.0.0.0