Log Like you Mean It - Cloud Austin

by James Wickett on

#cloudaustin

@tristanls: RT @ernestmueller: #cloudaustin Log Like You Mean It - coming to you live via Hangouts On Air! http://t.co/hd8G6K4ezt

Log Stash demo

Pro tips for logging http://theagileadmin.com/2010/08/20/logging-for-success/ @ernestmueller
LogStash written in ruby and better to use jruby to run it
@filler: Listening to @pczarkowski tell us that he is lmao’ing if we don’t logstash.
RT @filler: “If a new user has a bad time, it’s a bug.” @pczarkowski #cloudaustin < that's deep man
Nice how logstash parses into json
@filler: “grok takes complex regular expressions and transforms them into easy, repeatable patterns.” @pczarkowski #cloudaustin #logstash
200 tweets per minute for Beiber, 4% of his fans misspell his name
Logstash will function as a syslog endpoint
http://tech.paulcz.net/ACUG-Logstash are the slides from the #logstash portion of the evening
@jordansissel: Regarding "logstash losing connectivity to elasticsearch" - ask on logstash-users@googlegroups.com for help!
@jordansissel: I've got a demo lof kibana 3 up for logstash at http://t.co/FbwCzQuhhR

Graylog2

Ian Richardson is now up at cloud Austin
Graylog is written in Java, web front end is in ruby on passenger
Moving from slides to demo, running in vagrant
Graylog2 even has a #pagerduty plugin
Now @lennart is talking new features for graylog2
graylog2-radio helps you buffer > http://blog.torch.sh/post/43651465177/introducing-graylog2-radio
No longer using ruby and or rails going forward (so sad)
@filler: #graylog2 0.20.0 to no longer use Ruby, Rails per @
lennart. Going Java, Scala instead.
@campbellmcneill: Logging pipeline between LogStash and Graylog2 seems to be a popular combo
Follow the project at @graylog2

Sumologic

Lots of people like @splunk but it seems like the cost is keeping them away or causing customers to abandon
Using rsyslog with Sumologic
Sumologic looks a lot like splunk to me... If I squint I don't think I could tell them apart

Splunk

National Instruments boys are talking @splunk
They showed the difference between classic metal implementation versus the cloud implementation of splunk
@michaelwilde: @dabrownkid84 very nice presentation about Splunk this evening at #austin #CloudAustin
The splunk ninja is in the crowd @michaelwilde
@campbellmcneill: Anyone using their log management systems for detecting security threats?
@jordansissel: the logstash/graylog2/sumologic/splunk demos at #cloudaustin showed all the same features, but the costs are wildly different.

Project Meniscus

@everett_toews: Next up at #CloudAustin is the #Rackspace Meniscus logging team. Check out the open source project at http://t.co/EFiF1Reh6E #OSS
Stole lots of good ideas from the other tools when building meniscus
@campbellmcneill: Complex Event Processing is key to building extra value on top of logs #ProjectMensicus
@campbellmcneill: How many of these logging systems have a means to prevent log repudiation? #CloudAustin

Logging Architecture

@alexcorley uses rsyslog and logstash and kibana
Rsyslog is just the way to go