20200204 - CfgMgmtCamp 2020 - Nomad: Kubernetes, without the complexity, Andy Davies

by Thierry de Pauw on

#cfgmgmtcamp

Nomad: Kubernetes, without the complexity, Andy Davies

Nomad doesn't seem

I have problems with Kubernetes. I've used it and had even more problems with it.

Disclaimer:
- I don't like non-essential complexity

required complexity is what you need to fulfil a business need

From how I see it: K8s is accidental complexity. @Pondidum

Cindy Sridharan @copyconstruct

Docker succeeded not because of “the tech” (containers) but because it was a great developer tool.

Kubernetes built atop Docker, and decided to be all things tech and not much by way of UX. It was and remains a hostile piece of software to learn, run, operate, maintain.
https://twitter.com/copyconstruct/status/1194701905248673792?s=20

If you are going to use K8s, please use a cloud service. @Pondidum
Don't waste time managing K8s.

Nomad comes with one binary: cli + server mode @Pondidum
- cli
- server mode

features:
- health checks
- container management
- storage orchestration
- rollouts & rollbacks

  • vault for secrets storage
  • nginx for load balancing
  • etcd or consul for service discovery ...

Nomad: No YAML! This is the best reason for not using Kubernetes. @Pondidum


...
task {
cpu: 500 #Mhz instead of .2 of cpu
memory: 256 #Mb
}
...

Nomad assigns a job version and checks for that at deploy time

anyone had problems with Helm state? runtime state different from Helm known state

Secrets

Kubernetes secrets are not secret. They are obfuscated.

... secret data is stored in ectd; therefore: Administrators should limit access to etcd.

Nomad uses Vault

Vault supports AWS IAM, Github authentication, Active Directory

Docker Only?

as fas as I know Kubernetes can only run docker containers

Nomad can run any binary: exec, Java, Docker, Qemu
- artefact can be fetched via HTTP packaged as an archive

downside: you lose the isolation that docker gives you

use case: .NET application, small size, can only be run in K8s using a Windows Docker image that is several Gb big => run as executable in Nomad

Questions

Which placement scheduler does Nomad support?

there are several ones, I use always the default one

Do you have the .NET runtime installed on every machine that has Nomad?

Nomad has the feature of "affinities": these nodes support .NET, these not