Patch Sunday, plus Valentine and Banks

A week or two ago, the Oracle quarterly CPU (Critical Patch Updates) notification went out. So I scheduled my Solaris patching for the maintenance day I have: Sundays. The first round went fine today. Next Sunday, I’ll do the rest. Solaris patching is just about the least troublesome thing I have to do. It takes a fair bit of time, but the reversion path on Solaris 10 with the Boot Environment (BE) feature is as safe as houses (well, not houses badly built in an earthquake zone, or ramblers built on slabs on a flood plain, but you know what I mean).

Boot environments enable me to use the ZFS file system tools to make a copy of all the important bits of the operating system, mount that copy, and patch that. Then I can set it to be the new active BE, and reboot. Once done, I test. If all is good, and patching didn’t break things, I move on. If patching breaks things, I simple set the prior BE to active, and reboot again. Then I’m back to the state of the system prior to patching. It’s an excellent feature.

On the home front, I’m still happily running FreeBSD 10 as my main OS. I think there’s an update available, to 10.1, but I’m not going to try for that this evening. I’m tired – I pushed really hard on the exercise front this last week, and my knees ache a bit. No yardwork at all this weekend. The lawns could have stood to be mowed, but next week will be fine for that.

*      *      *

Recent Reading

Today I finished reading Genevieve Valentine’s SF novella, Dream Houses. You may recall my mentioning that Ms. Valentine read from this work at Capclave ’14 a couple of weeks back. It’s a bit eerie reading a story someone wrote, and hearing the words in her voice … wowsers! A note: You can get Dream Houses in eBook format. Mine is an inscribed trade hardcover edition.

I really liked this story. From the opening words straight through to the end, I was hooked – if it had been as long as this week’s second book, I’d be half dead from sleep deprivation. That’s one of the joys of the novella length. You can get the intensity of the short story form, and add in the missing character development that there isn’t room for 3000 words or so. Ms. Valentine has written shorts that I’ve read and enjoyed in Clarkesworld, but Dream Houses gets under the skin. It’s not a happy tale, I’ll give you that much. It seems that many of the novella length stuff I like, isn’t. (See Scalzi’s The God Engines, for example.) I care about Amadis, last of the crew alive on this run to Gliese. I pondered the motivations of the ship’s AI, Capella. I still wonder how many times Capella binge-watched 2001: A Space Odyssey.

Dream Houses is dark, exquisitely crafted, and deeply creepy. I’m going to have to read it again, sooner than later, to get more out of this marvelous confection. Highly Recommended.

A couple of days ago, I finally finished reading Iain M. Banks’s Against a Dark Background. I’ll give you this: If you’d read two or more of the Culture novels, it’s trivial to identify this tome as one by Banks. Against a Dark Background is deeply embedded in the Banks school of SF, but without any of the redeeming (IMO) quirks of humor that spice up the Culture books.

I guess it was one of those weeks, since this book is dark, dark, dark, too. And at about ten times the word count of Dream Houses, it was more of a workout, too, in trade paperback format. More to the point, it was a mental and emotional workout, almost more than I wanted. I very nearly put this book down. And by the end, only the protagonist, Lady Sharrow, continues to be damaged goods. Damn near everyone else is dead. It’s as if this story was Banks doing a dark, humorless SF version of a season of Black Adder. Everyone dies at the end there, too.

I wanted to like Against a Dark Background. I’ve enjoyed all the other SF written by Banks that I’ve laid hands on. The intricacy and attention to detail that are the mark of Banks are present. And in the details of sections here and there throughout, I was hauled into the story, against my will. Oddly, both books this week are about a woman as (eventual) sole survivor, ending badly even so. But while I loved Dream Houses, Against a Dark Background was a slog for me. I keep books I plan on reading again. I have SHELVES full of books I plan on reading again. This book isn’t staying. And that’s a darn shame.

Current reading:

Clarkesworld Magazine: Issue 97. This is my “five free minutes, I’ll read a story on my phone” target. I’ve been a subscriber (through the Kindle store) to Clarkesworld for a couple of years now. The quality and curation of the fiction is superb, the non-fiction is usually interesting and enlightening, and the cover art is awesome. I can always recommend Clarkesworld!

I’ve just started (as in, I’ve finished the prologue) of Paolo Bacigalupi’s newest novel, The Doubt Factory. At Capclave, Paolo referred to this as his “Public Relations Thriller.” I enjoyed The Windup Girl immensely, so I’m please to be able to add this one to my collection. I’ll let you know in a week or two how the ride was.

On Deck:

Unidentified Funny Objects 3, edited by Alex Shvartsman.

*      *      *

Our condolences to the families, friends, and units of these fallen warriors:

  • Lance Cpl. Sean P. Neal, 19, of Riverside, California, died Oct. 23, in Baghdad, Iraq, from a non-combat related incident.
  • Cmdr. Christopher E. Kalafut, 49, of Oceanside, California, died Oct. 24, in Doha, Qatar, of a non-combat related incident at Al Udeid Air Base.

Backup Discussion

On the topic of backups…

I like redundancy and backups. Redundancy is this: I have more than one copy of the data on more than one disk in my important systems. So: Mirrored disks == good. This protects me from one of the disks going bad. If the system blows up, or if I simply need a copy of a file I deleted by accident, I need a copy that’s either not part of the mirror, hidden from view, or NOT on the system. Here’s how I go about that.

Having a local copy of something so that when it gets deleted, I can restore it … this is a good thing. For that, on my main workstation these days, I use automated ZFS snapshots. This pulls a painless snapshot of the data in hand every 15 minutes, every hour, every day, every week, and every month. So I can restore in 15 minute increments for the last hour, hourly increments for the last 24 hours, etc. That’s awesome. I’ve also got those applied to my local backup from Marcia’s Windows 7 system, so that I can restore from various points in time (her baseline backup period is daily, however).

But also there is disaster recovery to consider. What happens when my main system bursts into flames, or more likely just corrupts both disks due to an extreme overvoltage event that overwhelms the UPS? If both sides of my mirror pair are gone, I need to be able to restore my system. The operating side troubles me not – I can rebuild that from scratch. But I have active data in my home directory, archived (and other backups) data in a /data directory, and why not make life easy by having an offsite copy of my configuration (/etc) directory as well? What? Hey, you noticed the key word there: offsite.

For theft, fire, extreme stupidity: these problems require a remote copy of the data that can’t be destroyed by the same event that takes out the originals. Now … I’m not really over-protective of this for obvious reasons. I keep my offsite backups at work, which is less than 20 miles away. For a large regional event – basically anything involving the words “blast radius” – my offsite backups don’t qualify as far-enough offsite. But since in that eventuality, I’m likely also permanently wiped, I’m unlikely to care about the state of my offsite backups. But for every reasonable risk, copies of my data at my work site are good enough.

Now, to other risks: If my data is on disks not at home, is it well enough protected there? The answer is, “Sure.” I use encryption. These days I’m using geli whole disk encryption on my FreeBSD 10 system. Oh, and I have three rotating copies of the data, so the offsite stores get refreshed weekly on a three week cycle. That’s all rockin’, but there’s one final issue that I’ve been dealing with: Heat.

I’d previosly been using an eSata shelf installed in the system case, but for a variety of reasons, it wasn’t really working well as a hot-plug solution, so I was power-cycling the system (twice!) every time I wanted to refresh the current week’s offsite disk. I broke down and bought a lay-flat USB 3 hard drive dock from Plugable (via Amazon) a few months back. This worked really well for me for one reason above all others: I don’t have a lot of headroom between the top of the system and the top of the cabinet that houses it. So normal, upright, “toaster” configuration docks won’t work for me. But, like the “toaster” versions, the lay-flat still suffers from heat issues.

These docks aren’t inside the system chassis with managed airflow removing much of the long-term damaging heat from continuously running drives. Now … in many cases, that’s not a problems with docked USB drives: You slot a hard disk, briefly write or read, spin it down, and you’re done. But I’m synchronizing over 500 GB of data. While I’m only writing 10-30 GB on any given Monday, there’s still a lot of back and forth read and write activity that runs for the better part of 45 minutes. That’s a lot of time for the heat to build up in the disk, and not be dissipated quickly enough. To improve the long-term lifespan of these offsite disks, I wanted to remove more of the heat. While doing the initial, 6+ hour synchronization, I borrowed Marcia’s AC desk fan. It worked well enough, but was awkwardly big, and noisy, too. For the long term solution, I recently picked up a Gino USB-powered mini-fan (also via Amazon). I can plug it in, set it pointing directly down on the drive, and run my backup job without overheating the disk at all. See?

USB fan cooling USB docked drive

USB fan cooling USB docked drive

Works like a champ. Both products are Highly Recommended.

*      *      *

The unasked (as yet) question that I’m about to hear is this: Why am I not just using the bog-standard and dirt cheap USB drives that one can pick up for pennies a gigabyte at the corner market?

Dirt-cheap pre-packaged USB hard drives have several strikes against them in my book. First: I’m paying for a cord, a housing, USB and power supply electronics, etc, all just to support ONE disk. With a USB dock, I can buy as many bare  hard disks as I want and use them interchangeably, with less overhead on all that other cruft.

Second: Dirt cheap means the electronics are cheap. And maybe sketchy. Or long-term unreliable. Or ? I don’t know. I can’t know. But I don’t want any issues with any part of a single, complex (yet cheaply produced) product compromising my backups.

Third (and most importantly): Disk quality. My understanding is that large system vendors and manufacturers (think Dell, IBM, HP, Fujitsu, etc) get the best quality disks – the ones that scored at the top of all the quality control checks. The second tier vendors, and the large disk resellers (think NewEgg, Amazon, and the like) get the pick of the rest of the best. And I’m told (meaning I read an article on the Internet, so it must be true) that the vendors that churn out cheap, fully packaged USB drives get the stuff from the lower third of the barrel.

Now, I’m not saying by any means that any of those disks didn’t pass quality control tests. What I’m saying is that they didn’t pass them with as much of a margin as the best disks. What does this mean for long term data storage? I’m not willing to run that experiment with my data. I’ll spend more money for higher quality disks. I actually buy the “Enterprise-grade” versions of the disks in the size and speed I require for various purposes. The price bump is on the order of 50-100% over consumer-grade disks, but the reviews and benchmarks tend to indicate that the Enterprise gear is an order of magnitude more reliable. That’s also corroborated by the manufacturer’s warranty on this grade, with is generally 3-5 years, rather than just a year.

So, buy quality products, keep them cool when running, and use encryption: the data will live a long time. That’s my story and I’m sticking to it.

Back in the Groove

After several weeks of very intermittent exercise, and some backsliding on the dietary controls, I’m back in the groove, I think …

Good exercise days

Good exercise days

Good exercise days – two days running at almost precisely the same pace. Excellent!

*      *      *

I use OpenSSH for nearly all of my computing platform access, except for that OS from Redmond. Secure access to commandline environments meets about  90% (a made up number, gut check says “true”) of my requirements. It’d be more, but I access a lot of monitoring services via a browser. But close to 100% of actual work is done via commandline remotely over SSH.

Every time I set up a new server, I’ve been logging into the new box and setting up the first user account with the appropriate public key. First I’d copy the key up to the new system, then I’d log in on the new box and run commands like these:

[user@box] ~# mkdir .ssh && chmod 700 $_ && touch $_/authorized_keys && chmod 600 $_
[user@box] ~# cat id_dsa.pub >> $_

How was I to know that in the intervening years, some one of the smart contributors to the OpenSSH project added the ssh-copy-id program. All I need to do from the client system is type something like this:

[user@client] ~# ssh-copy-id -i .ssh/id_dsa.pub user@box

It prompts me for my password, and the work is done. All of the directory and file work, correct permissions included (replacing those chmod commands), all done in one swell foop. Very handy.

Ciao!

A Lovely Day for an Anniversary

Yep. Sixteen wonderful years I’ve been married to Marcia. To celebrate, she worked on a t-shirt quilt for a customer, while I … went to work. But this evening, on the way home …

Got my gal some roses

Got my gal some roses

She seems to like them. She also got a sewing thing accessory as a pressie, while I’ve got a set of E. E. “Doc” Smith paperbacks headed my way. We’re so romantical. We’ll go out to supper to complete our celebration one of these days.

*      *      *

Marcia got a FitBit thing a while back, and she likes it very much. It helps her keep track of a lot of the things she needs to. Since I’ve been trying to do better on that front, like this (this evening):

Burned a kilocalorie

Burned a kilocalorie

I could also do better at keeping track of such things, as well as managing my portion control through better records keeping. But I chose a different bit of gear: a Jawbone UP24. I executed the purchase through Amazon, which saved a few bucks, and I have had very few problems puzzling out how to make the App interface operate. I’ll report further on the product, the app, and how/if it’s helping me with any of my goals at some ill-defined later date.

Trip Report: LOPSA East 2014

As promised, if a few days later than expected, here is my trip report for LOPSA East 2014.

For those who don’t know, LOPSA is the acronym for the League of Professional System Administrators. LOPSA is the entity that emerged from an attempt by SAGE to gain independence from USENIX back in 2005. You can get more of the back story on the LOPSA history page (https://lopsa.org/about_history).

Four years ago, PICC (the Professional IT Community Conference) was first held in New Jersey; it was organized by William Bilancio and Tom Limoncelli. I missed that one, but have attended ever since. Last year, the organization decided that if there was going to be a renaming to more closely associate the conference with the LOPSA “brand”, the time was ripe. Thus, LOPSA East.

Before the breakdown of my trip, let me present the value. I was $1100 all in: conference with two days of training, talks, and networking, plus food, lodging, and fuel. That’s a hell of a deal. I could have peeled off another 300 bucks by getting up at 4 AM on Friday, and driving home after the end of things late on Saturday, but life is short. Frankly, there is no better bang for the buck than a LOPSA regional conference.

Don’t get me wrong, I love the USENIX LISA conference as well. But there are so many simultaneous things going on at a conference of that size that I am ALWAYS missing one thing I want to do, to do something else instead. LOPSA East is small enough that I only felt that regret a little bit, since there are only three main tracks, not ten or a dozen. And if the price per hour of conference at LISA were the same as LOPSA East, LISA would be half the price. EVERY ticket at LOPSA East is a golden ticket, IMO.

The venue for the conference is the Hyatt Regency New Brunswick, in New Jersey. It’s about three hours and change for me to drive, in the middle of the day when the traffic is light. It’s a good hotel, pleasant and well-kept. The staff are polite and helpful. Stuff is expensive (as hotel stuff usually is), but the conference block rate for rooms is good, and I was comp’d the WiFi. (Dudes, seriously, Internet access is like air and water – build the charge into the room rate, mmmm-kay?)

*** Thursday ***

I drove up the day before the conference started, as has been my habit since I first started attending this conference. It lets me get settled in and be rested and ready for technical material – talks and trainings – first thing the next morning. I also found, my first year at LOPSA East (then PICC) that the conference volunteers and organizers also are on site the night before, and have a dutch treat supper, followed by assembly of the conference totes, etc. I’ve always been welcomed and been happy to help out in any way I can. This year, same thing, though we finally had more people than the small “private” dining room at the back of the restaurant in the hotel could hold.

The materials were all onsite by 1900, and we’d gotten everything assembled by 2000. Oh, yeah – I can definitely recommend the lobster macaroni and cheese. Just sayin’ … Folks often head out for a beer or three thereafter … but I usually skip that part,  not being a drinker.

*** Friday ***

One of my prime objectives in this conference was to get my head better wrapped around tools and utility of configuration management systems. I’d attended an Intro to Puppet training a couple of years ago, and while I “got it”, I wasn’t working with any CM at the time, and needed more personal experience with the concepts and products. For a variety of reasons, this has become my “year of configuration management”. I’d spent a considerable amount of personal time here at home working though issues with Puppet, and experimented a bit with Chef and CFEngine … and I was headed down the Puppet path when Erik Fitchner (former cow-orker at NFR/CP) suggested that I simply must check out Ansible or Salt prior to committing. So I started working with Ansible at home, too. This work informed my talks and trainings selections at the conference.

I started on Friday morning with a half-day Intro to Puppet tutorial presented by Thomas Uphill. Within the context of the work I’d already done, I understood everything that was placed in my brain, and picked up a few things I’d missed in autodidact mode. As with every training I attended this year, every demo and example presented actually worked. Thomas’s slides for the presentation are here: https://goo.gl/tZLMQX

Lunch (both days) is provided by the conference, at the hotel. I’ll give the Hyatt credit for having a first-rate menu, and kudos to the conference organizers for finding it in the budget to feed us so well, and having yet another space and time to meet and talk and network. With a small conference, there isn’t as much of a “hallway track” as there is at LISA-scale events, so these lunches are an important part of the overall LOPSA East experience.

Friday afternoon, I attended Mark Harrison’s tutorial on Vagrant: Not Just For Developers. I learned quite a bit about the speed and utility of spinning up and down test systems for any variety of purposes. I’m looking forward to implementing some of what I learned into my virtualization workflow at the office. Here are Mark’s slides.

The Friday evening Keynote was given by Vish Ishaya, on the topic of OpenStack in the Data Center. After last year’s keynote about our pending doom and the crisis of cloud that MJR gave us, Vish’s talk was optimistic, nearly all sweetness and light by comparison. I know more about OpenStack now, and how it plays into both external cloud vendor business models, and some of what to consider when looking for private (corporate internal) implementations.

After the keynote, a group of us went out to supper across the street at the Old Bay restaurant. I had a superb chicken and sausage jambalaya, the others in our group were equally happy with their selections. Highly recommended.

Conference BoFs (Birds of a Feather gatherings) and Lighting Talks were scheduled from nine to midnight. I attended the Mentorship Program BoF at nine. We discussed how the program was going, and what we needed to do as an organization to try to make it more effective. I’ve mentored one person through the LOPSA Mentorship program … and it just sort of, well, stopped. I’ve tried to send further emails, but no replies are forthcoming. Hmmm.

After that, I retired for the night. Twelve full conference hours is a long day for an old fart like me.

*** Saturday ***

Saturday morning, I attended the Infrastructure Talks Track, with these topics: Enlightining Technical Leadership, Using Ansible to Fill the Gaps Left Over from Puppet and mCollective, Git Hooks for Sys Admins, with Puppet Examples, and The Stack at Stack Exchange. They were, respectively: quirky, useful, interesting, and captivating. How Stack Exchange manages to be in the top fifty of web destinations, and serving that level of traffic with a single rack of windows boxes, well, it just blows my mind.

Saturday lunch was similarly arranged, and of equally high quality of food and company. I bailed out a few minutes early to take care of a few things, so I missed the public presentation of my Certificate of Professional Recognition:

LOPSA Certificate of Professional Recognition

Certificate of Professional Recognition

Saturday afternoon, I attended Thomas Uphill’s Advanced Puppet training course (slides: https://goo.gl/SeiVsa). This content included some things I’d experimented with on my own, and a lot of concepts and ways to structure a Puppet implementation that I will find very useful when I finally need to implement Puppet for real. I have no idea how long those slide decks will stay available – I pulled down a copy for myself, just to be safe. This was the most challenging half-day of my conference calendar, and I was not disappointed.

Saturday evening’s Keynote speaker was Elizabeth Krumbach Joseph, currently with HP, and working on the OpenStack project. Her topic was Universal Design for Tech: Improving Gender Diversity in our Industry. She made a great case for why we need to improve diversity (gender and otherwise) in our profession, and briefly discussed the pluses (the business world is a LOT more professional and non-harrassing than it used to be) and minuses (the online world, especially among the anonymous trolls of the open source world, is a very unpleasant place to be female, or really, different in any way). Her pitch is both true and important. It’ll be the talk most on my mind for the next two years, because of the last bit of news from this conference…

[Here’s Elizabeth’s blog post on the conference: http://princessleia.com/journal/?p=9372]

Also on Friday evening, after conversation with the organizers and next year’s Program Chair, I volunteered to be the co-chair for 2015, which means that I’ll be the Program Chair for LOPSA East 2016. Wow. Just wow. I’m honored that they think I’m up for the job, and I’ll do my best not to disappoint.

*** Sunday ***

A nice drive home, starting about 0800. I got 37.5 MPG in the 328i, too!

 

OMG Moar Wintar!!

There’s a bit of rain on the ground … about a tenth of an inch since noon. The temperatures are in the mid-to-high 30’s (F). So of course OPM has already called it for tomorrow: CLOSED. Now, depending on how reliable you regard our weather forecasting establishment to be, that might be a good move. Others believe, too: BJs was crowded early this morning, when I was there for the weekly shopping. Evidently, for some folks, a winter storm is actually Ragnarok.

I did a lot of small projects this weekend, and I’ve spent some time diving into Ansible as an alternative to Puppet for systems configuration management. It looks interesting, and a lot less programmatic in implementation. I don’t know quite enough about either to make an informed choice, yet. But soon, soon.

I also built a birdhouse for the backyard, out of shop scraps. I’ve applied some walnut stain to most of the outer surfaces. I’m going to get a bit of white paint on the roof, then apply spar urethane over the whole project. I’ll put it out in a couple of weeks (and a picture of it up, at that time).

Oh, hey. One of the people I follow on Twitter noted that they weren’t watching the Academy Awards. Those are on? Um, okay. Break a leg, film people.

*      *      *

DoD announced no new casualties in this last week, yay! Ciao!

Yardwork … errrr, Spring almost upon us.

I withstood the temptation to wash cars this weekend, which was difficult, because the weather was glorious. About 60° F both days. We did open up the house both days and get some fresh air in, which was very nice indeed. But the highs are dropping back down into the 30’s, with chances of snow a couple of days this upcoming week. So I’ll just wait. Soon it’ll be time for washing of cars and yardwork … but not just yet.

Today, I was going to cut my hair and bathe the dog this afternoon (shopping and work this morning), but instead ran Marcia over to G Street Fabrics in Rockville, to get us a little time out of the house. Pretty day for a drive.

*      *      *

Instead, I am dealing with my second SSD failure in a year. An old Crucial M4 gave up the ghost last year, and yesterday my Windows box failed to boot to the Samsung 840 I had installed there. No data lost, I have good backups, but it’s still a pain in the ass. To replace it, I’ve ordered on of the Seagate “hybrid” hard drives. I’ve not tried one of those, and I’d like to see how good they are. I’ll still probably end up with another SSD in this system, but not just yet.

*      *      *

Our condolences to the family and friends of Master Sgt. Aaron C. Torian, 36, of Paducah, Kentucky, who died Feb. 15 while conducting combat operations in Helmand Province, Afghanistan.

Really, Valve?

You folks at Valve crack me up. I’m waiting for Half Life 3. I’m waiting for Portal 3. I’m waiting for Obduction (oh, wait, I’m getting that through the Kickstarter) … and y’all ask me if I approve of greenlighting Krita for sale on Steam? [[ Krita is drawing and image manipulation software, and it’s pretty damn cool ]] But I already have Krita on my Linux install. Oh … buying the free (as in GPL, eh) software just supports the development. Why did I have to dig for that information, Krita-people? Or was it Valve that made that information go way to the bottom on their page for your project.

Hear me. I’m waiting for Half Life 3. I’m waiting for Portal 3. If you want to sell me stuff, then work on those games. If you don’t … oh, well.

Pro Puppet, Second Edition, Grumble.

Well. I’m now plowing through Pro Puppet, Second Edition, in a continuing effort to be useful in integrating configuration management into my environments at work and here at home. But the publisher website doesn’t have features promised by the book, the publisher website doesn’t seem to accept and post the errata I’ve discovered, and now the Puppet Labs blog post about the book has marked my comment on their post as “Spam”. Um, really? Here’s the comment I tried to post to the blog:

Bought the book. Quite useful, about a quarter of the way through, but…
There are still some serious editing issues, code inconsistencies, and outright errors and oversights. While I’m learning *more* than I expected by fixing problems in the examples in the book, I’m sad that the examples appear not to have actually been tested (or copied from known working configurations).

Hint 2 – ALWAYS say what file the code snippet goes into. Very frustrating.

Hint 3 – The book says that all of the code from the book is available on the Apress site. Not yet, it isn’t. Can someone get Apress on the ball about this, please?

Hint 4 – I’ve put in two or three errata entries on the Apress site so far, battling the bloody captcha. None have been reposted, and I’m tired of doing so when I must manually transcribe (I can’t copy out of the Kindle browser reader directly). No feedback, not posted … not sure why I should continue trying.

Like I said, good book, seems to be good coverage, but ragged in places and not as well supported by the publisher as I’d expect, given the amount of money I shelled out.

Best,

Brian

That doesn’t seem too spammy to me. But then, I wrote it. Oh, well.

*      *      *

Oh, yeah. Home today: I had some stuff to do with Marcia this morning, and I took the rest of the day off … because I can. Ciao!