That was me. The first five mornings of my seven odd days at LISA’14 in Seattle started between 0400 and 0430 in the morning, as my body insisted it was time to rise and shine. The most fun, though, was the wake-up due to SMS, when our garage door failed to open for Marcia. Turns out the spring catastrophically unsprung itself overnight Sunday night (though Marcia couldn’t figure out the source of that noise until morning). Fortunately, “catastrophically” didn’t include damage to either house or cars. She consulted, then called Overhead Door, who came out and made it all better, for money. Well spent, that.
* * *
The LISA conference was useful, educational, and fun.
On Sunday, I partook of two half-day tutorials: Statistics for Ops: Making Sense Out of Data (presented by Kyrre Begnum and Nicole Forsgren Velasquez); and R for Sysadmins (presented by Jason Maughan and Nicole Forsgren Velasquez). R is a language/environment for statistical computing and visualization. In a nutshell, Sunday was Math Day. And it’s surprising how much more interesting statistics are when I care about the data and results. Using the vast amounts of machine-generated data I collect, I can use freely available tools to baseline and model the behavior of my systems and infrastructure. Why? To catch out-of-the-ordinary trends before they become problems, of course. These two classes also helped me to understand the underpinnings of the other tools that we utilize for monitoring at $FIRM. And, since my goal for LISA is always to spend my time hanging out with and learning from people smarter than I am, it was a very good first day, indeed.
I would have spent more time in the evening sessions, formal and informal, as well as the late-day “hallway track”, but my body for the first three days was done by eight or so in the evening. I’d hang out in the freenode #lisa14 IRC channel, have a bite of supper, and fall asleep in the bedside chair, with the TV running. Party animal, I’m not.
On Monday, I sat in on a full-day training on the topic of Testing Your Automation Code, presented by Nathen Harvey. While Nathen works for Chef, and I use another automation suite for my environment, the principles of test-driven development for operations management are sound and cross the boundaries I need them to cross. I’m learning to be more effective with the tools I’ve chosen, and this training gave me better leverage for planning, execution, and documentation.
Tuesday I was absorbed in the Defending Against the Dark Arts tutorial offered by Branson Matheson. I’ve taken training from Branson previously – he’s primarily a computer and operational security bloke, but with strong cred across a lot of the fields that comprise modern enterprise information technology. This was a first-time-offered course by Branson, which discussed how well-funded offensive actors are driving into and through most organizations. Topics included how to offer organizational training to oppose trivial social engineering attacks, and how to focus on the layered defenses in a manner so as to shorten the time between a successful attack, and the detection thereof. Since the bad guys generally always have access to a successful attack against a vulnerability before there’s a chance (or even a hope) of patching, it’s better to detect and mitigate their presence as quickly as possible as a primary goal. It was an excellent and stimulating day.
That’s the quick survey of my three training days of LISA 14. As a result, I didn’t get to attend any of the other 18 full- and half-day courses offered during the same time span. Sigh. I guess there’s always next year. And this holds doubly true for the second three days of the conference: Talks (vendor and invited), refereed papers, mini-Tutorials, and more: Oh My! The fun kept happening…
Wenesday opened with remarks by Program Chair Nicole Forsgren Velasquez (you’ll recognize her name from my stats trainings on Sunday). Nicole welcomed us to the more academic (and more crowded) half of LISA14. Attendance is up quite a bit: 1100+ this year. She was followed by opening keynoter Ken Patchett of Facebook, who talked to us about Open Compute and the Changing Data Center. Very interesting stuff, especially (mostly?) to the two or three dozen companies that deploy systems at those scales. Fun to learn about, but way beyond what I can apply in my environment.
Other things I sat in on Wednesday: H. Wade Minter presenting, “You code like a SysAdmin” – on the importance of coding for ops. Dave Cliffe on the topic of “Best Practices when the s*IT hits the fan.” – planning and rehearsing for the inevitable outages. Lunch was spent with the vendors on the expo floor. Caskey Dickson discussing “while (true) do; How hard can it be to keep running” – resilience at the service layer. “LISA Build: Mind. Blown” was presented by Branson Matheson and the rest of the volunteer LISA Build team, who assembled and kept the conference network going for the entire 6 days. Wednesday’s talks wound up with Lars Lehtonen, on the toopic of “Burnout and Ops.”
Thursday’s keynote was by Gene Kim, entitled “DevOps Patterns Distilled: A Fifteen Year Study of High Performing IT Organizations.” Gene is well know in IT circles for a variety of commercial and academic reasons. He talked to us about what makes the best performing IT organizations distinctive. There’s a paper coming (not out yet) that should substantiate much of what he discussed. The rest of my Thursday included: Steven Murawski on “Building PowerShell Commands.” An hour or so sitting at the LOPSA table supporting our community. Finally, Steven Murawski again, on the topic of Windows Desired State Configuration.
On Friday, the talks I sat in on included: Chris Stankaitis on the utility of checklists to enhance human reliability. Ben Rockwood: “I am SysAdmin (and so can you).” Luca Deri describing “High Speed Network Monitoring using ntopng.” Brendon Gregg speaking on Linux Performance Analysis. The capstone of the week was delivered by Courtney Kissler, VP of eCommerce and Store Technologies at Nordstrom. Her conference-closing keynote was an enlightening look at what a large, non-technology firm makes of, and utilizes, from the buzzword-drenched ever-shifting technology sands that our industry is today. Three thumbs up for Courtney!
There were many other events, too: An evening social sponsored by Cambridge Computers that was very pleasant. A conference party at Seattle’s EMP Museum (not Electro-Magnetic Pulse, much to my surprise). Music and art and sci fi and fantasy and horror all in a yummy mix there, in the shadow of the Space Needle. The famed hallway track, where unscheduled and awesome conversations happen nearly around the clock.
It was a superb, if hugely tiring, week of smart people sharing their knowledge and wisdom. I loved it.
* * *
Now it’s time to utilize some automation tools to do some Linux patching. Be well.