# 2019/12/12 YOW! Melbourne Day 1. 3a

by Gene Kim on

#Yow19

2019/12/12 YOW! Melbourne Day 1. 3a

  • @RealGeneKim: RT @yow_conf: The talk in the Blue room with @thattommyhall contains interesting theory that is resolutely practical & with concrete examples in several languages and distributed computation frameworks. #YOW19
    https://t.co/frdu3ah176

  • @jchyip: At @thattommyhall’s data pipelines talk at #yow19

  • @RealGeneKim: #Yow19 @thattommyhall: https://t.co/rVlWviok4p

  • @RealGeneKim: #Yow19 @thattommyhall: epochal time model (from Clojure); very applicable for data pipelines https://t.co/OArWORegu7

  • @thattommyhall: epochal time model (from Clojure); very applicable for data pipelines

  • @thattommyhall: "The weird trick about data: the data will always be there... as long as we don't overwrite or delete it" πŸ˜‚πŸ˜‚πŸ˜‚ The awesomeness of immutable data models β€”- don't change it. So great!!

  • @thattommyhall: "Great feature of most bucket stores (AWS, Google Cloud, Azure?): you can publish it, and pass on transfer charge to your users; great for sharing scientific data"

  • @thattommyhall: Apache Airflow?

  • ran-on vs. ran-for

  • @thattommyhall: "Don't as part of your filename versioning do: data-FINAL.csv... it's never final. You'll have data-FINAL-FINAL.csv, data-FINAL-FINAL2.csv" πŸ˜‚πŸ˜‚πŸ˜‚

  • @thattommyhall: airflow, AWS Stack Functions, AWS Fargate (serverless k8s; slow start: 1m...); AWS batch; Sparklens; DeltaLake (data diffs on S3)

  • @thattommyhall: Bencode from BitTorrent

Up: Cost of a Dependency with @LeeRyanCampbell

  • @UrcherAus: Cost of a Dependency with @LeeRyanCampbell at #YOW19 https://t.co/NUYqaoG0tI

  • @LeeRyanCampbell: "the technology we build will often outlive the technology it's built on (e.g., programming language, libraries, OS)". 🀭🀭🀯🀯

  • @ineffyble: "Often the systems that we build will outlast the technology that they're based on" #YOW19

  • @lizthegrey: TypeScript can let us statically typecheck for safety.

HTML must be used in ways that render correctly, have good accessibility, but are easy to read for developers. Use semantic tags like <section>, <nav>, <aside>, not just <b> <div> etc. #YOW19

  • @LeeRyanCampbell: "you have to build, test, and package

  • @LeeRyanCampbell describing the problems when anytime you want to change something, the changes are splattered across the entire repo... "the loss of locality decreases focus, flow, and joy." So wonderfully described!!!!

  • @RealGeneKim: #Yow19 @LeeRyanCampbell: "Do these change together?" Yes! The questions that isn't asked often enough! "Consequences: build times longer, testing harder, friction to innovate, blurry boundaries, lack of responsibility" (SO GOOD!) https://t.co/3m44vr8KIW

  • @LeeRyanCampbell: "Do these change together?" Yes! The questions that isn't asked often enough! "Consequences: build times longer, testing harder, friction to innovate, blurry boundaries, lack of responsibility"

  • @LeeRyanCampbell: "look at this function... 11 things being injected into this function. how can you possibly understand this?"

  • @KenScambler: Lee Cameron is telling the truth, listen to this man. Every dependency has a cost. #YOW19 https://t.co/LeF8fVLuE1

  • @ineffyble: Deep layering... #YOW19 https://t.co/KIe5kBDT2F

  • @LeeRyanCampbell: "The dev is trying to right thing. They're trying to bind to an abstraction that they don't own: 983 lines in interface... a mock needs to implement all 90 methods." (this is like a horror story that we should all tell each other to scare each other!)

  • @ineffyble: #YOW19 https://t.co/mzDvoImsdi

  • @ineffyble: You get non linear growth when you have layers of dependencies. #YOW19 https://t.co/kyOK6InRPg

  • @LeeRyanCampbell is describing the problem that we have all sleepwalked our way into... In fact, in many ways, this sums up so much of my learnings from Clojure and functional programming. This is such a great talk!

  • @LeeRyanCampbell: "Each build brought in 17GB of data" (!!!)

  • @RealGeneKim: #Yow19 @LeeRyanCampbell: A call stack. See how few layers are actually the business logic!!! "Pity the cognitive load of the dev or ops person, or @troyhunt delighted at the attack surface area!" https://t.co/im0KmS4URB

  • @LeeRyanCampbell: call stacks. See how few layers are actually the business logic!!! "Pity the cognitive load of the dev or ops person, or @troyhunt delighted at the attack surface area!"

  • @simonbrown: An alternative view: look at all of the functionality that is done for you ... imagine having to code this yourself. #YOW19 https://t.co/ETrF3NoZTb

  • @LeeRyanCampbell: "So look at this function. Do I try/catch, or is something above me doing it? Do I log, or is the job of my caller?

  • @RealGeneKim: #Yow19 @LeeRyanCampbell: "All of these things create coupling. It's like Velcro. Once you put it together, it's very difficult to pull apart." https://t.co/rTom1NtqrX

  • @LeeRyanCampbell: "All of these things create coupling. It's like Velcro. Once you put it together, it's very difficult to pull apart."

  • @LeeRyanCampbell: "To compensate for our long build times, we build magic build scripts, which only one person understands... Like Brent in Phoenix Project"

  • @lizthegrey: stop doing height: 100%. instead use 100vh for "100% of viewport height.

Okay, now onto flexbox layout. display:flex;align-items:center;justify-content:center

also try space-between, space-around to apply to contained elements. and flex-direction:column #YOW19

  • @lizthegrey: p.s. you avoid this cognitive problem if your APM or distributed trace spans... start with the actual application logic rather than in upper/middleware as crash/callstacks do...

  • @RealGeneKim: RT @ineffyble: "I depend on abstractions but they depend on abstractions, so you get a deep call stack" @LeeRyanCampbell #YOW19 https://t.c…

  • @RealGeneKim: RT @simonbrown: An alternative view: look at all of the functionality that is done for you ... imagine having to code this yourself. #YOW19 https://t.co/ETrF3NoZTb

  • @RealGeneKim: RT @UrcherAus: @LeeRyanCampbell Cognitive load increases super-linearly with each layer

  • @LeeRyanCampbell at #YOW19

  • @RealGeneKim: RT @UrcherAus: @LeeRyanCampbell The problem is that our code is too large and our solution is that we only add to it. This is not good

  • @LeeRyanCampbell at #YOW19

  • @RealGeneKim: #yow19 @LeeRyanCampbell: β€œbefore vs after: division by technology area vs. features.

β€œthe second groups files that change together.”

Reminds me so much about the great work of @AdamTornhill https://t.co/nFk1a5zkPq

  • @UrcherAus: @LeeRyanCampbell Loose coupling is one end of a spectrum. It needs to be balanced with high cohesion

  • @LeeRyanCampbell at #YOW19

  • @LeeRyanCampbell: "it's not just about loose coupling; we want high cohesion"

  • @LeeRyanCampbell: "Instead of reuse, aim for encapsulation"

  • @RealGeneKim: RT @UrcherAus: @LeeRyanCampbell Let's aim for 'use' first before aiming for 're-use'

  • @LeeRyanCampbell at #YOW19

  • @LeeRyanCampbell: "Build times went from 1 hour to 5m; tests ran in minutes; it's now so obvious when changes introduce things that shouldn't be there -- change failure rate

  • @LeeRyanCampbell: "every dependency can increase cognitive load,

TODO

  • after Save Scribe, focus back to editor
  • pull into notes: need to increment one more cur POS?