Tweetscriber
 Get Tweetscriber

Sign in with Twitter

20200204 - CfgMgmtCamp 2020 - I got 99 problems and a bash DSL ain't one of them, John Willis

by Thierry de Pauw on

#cfgmgmtcamp

I got 99 problems and a bash DSL ain't one of them, John Willis

@botchagalupe

https://twitter.com/botchagalupe/status/1143489712176422912?s=20

In Defense of YAML, Rod Johnson, Creator of Spring

I feel like there is a large gap between shell scripts and Ansible.
Make -> CFEngine -> Chef/Rubby -> Ansible

The Irony of the Configuration Management 30 Year Arc

  • we started out with Shell Scripts (Before)
  • next, we created macro like definition files (first generation)
  • next, we created primitive based declarative languages (second)
  • then we move back to definition files (third generation)
  • here we are ... Shell Scripts are back in vogue

The Drunken History of Configuration Management

Scripts -> Config Values (first generation) -> Rules Engine (second generation) -> DSL (third generation) -> Scripts

Before Configuration Management

a bunch of wires in a datacenter

The Three Virtues of a Great Programmer according to Larry Wall, creator of Perl
- laziness
- impatient
- hubris

Summary of configuration management

  • procedural
  • manual and ad-hoc
  • no desired state
  • pets not cattle
  • inconsistent environments (divergent)
  • not repeatable or disposable

First Generation

The four:
- Tivoli
- IBM
- ...

  • loosely declarative
  • not manual but still ad-hoc
  • no desired state
  • not pets or cattle
  • inconsistent environments (mostly divergent)
  • not repeatable or disposable

to apply a configuration change, you basically exported something from the tool that you thought worked and modified that code

  • @fengor: "you didn't far out spaces or tabs in this stuff. The compiler would go where no human would dare to go..." @botchagalupe on the history of what came before #cfgmgmtcamp

  • @boffbowsh: This history of Devops from @botchagalupe just makes me frustrated at how easily newer infrastructure engineers dismiss these literal decades of infra experience. So many of these problems have been solved before #cfgmgmtcamp

Second Generation

The new four:
- CFEngine, 1993 - Mark Burgess
- Puppet, 2005 - Luke Kanies
- Chef, 2009 - Adam Jacob
- Ansible, 2010 - Michael DeHaan

Is anybody else confused about Chef?, Kris Buytaert

Why Chef Exists:
- Adam Jacob, look I'm going to rewrite the product (Puppet)
- @felis_rex: “If Luke had accepted that Pull Request, Chef would not exist today.” - @botchagalupe with an angle on history #cfgmgmtcamp

Trinity of Configuration Management
- service
- template
- events - notify

CM Tools are only as good as their authors, Jerome Peazzoni
- @felis_rex: RT @WouterSchoot: Haha this is so true #cfgmgmtcamp https://t.co/tMBC4hu6op

Infrastructure as Code pro's:
- abstraction DLS's are very powerful
- self-documenting
- high reusability
- easier to provide data-driven models
- generally more consistent than scripted patterns
- most major IaC products have good testing abstractions

Infrastructure as Code con's:
- abstraction DSL's have higher learning curve
- complex edge case scenarios/failures
- script/shell primitives are used often
- integration interfaces are more complex
- infrastructure is built Just in Time (JIT)
- knowns are not always known
- builds are convergent, not congruent

Summary:
- mostly declarative
- fully automated
- desired state
- cattle not pets
- consistent environments (convergent)
- repeatable and disposable
- not immutable

Please, do not rewrite another configuration manager! @botchagalupe

Enter "Kid" Midas

looks like a 12 year old kid and looks like a person that has been in the enterprise for 25 years
- Vagrant
- Packer
- Terraform
- Vault

Third Generation

In all fairness, Solomon made a serious change. It was big.
- pull/push, modelling git into it
- the hub
- commoditised shared images

Dockerfile, how could we go back in time? @botchagalupe

Creating consistency in the Pipeline
- immutable infrastructure,Netflix - immutable servers, Kief Morris

Paper: Why Order Matters: Turing Equivalence in Automated Systems Administration,

The least-cost way to ensure that the behaviour of any two hosts will remain completely identical ...

=> argument for immutability

  • First-generation: divergence
  • Second generation: convergence
  • Third generation: congruence

see following picture:
- @f3ew: Configuration management history. #cfgmgmtcamp https://t.co/g1JQHMSEmK

Why order really matters (SPC 68-95-99.7 Rule) => Variation:
- converged infra
- immutable infra
- immutable delivery

Immutable pros:
- least variation pattern
- faster provision model
- fits well with Microservices architectures
- less reliance on IaC
- binary consistency from dev to prod

cons:
- DSL abstraction not as mature as IaC
- small changes are harder to manage
- debugging is harder
- ...

Summary:
- partially declarative and partially descriptive
- fully automated
- disposable target state
- cattle not pet
- congruent vs convergent env
- repeatable and disposable
- immutable

"The New Normal" Configuration Management

  • @fengor: "terrible feeling as a consultant but you are still going to cache that check" @botchagalupe about his Tivoli experience #cfgmgmtcamp

Scripts -> Config Values -> Rules Engine -> DSL -> Scripts

Puppet was a game-changer to me.
Pulumi looks fantastic. Haven't tried it yet.

Everything is old is new again, @botchagalupe