Rather Closer to the End

Well, the beginning was a good long time ago. And one thing I can always take away from the Drake Equation is that a primary reason that we’re alone is because civilizations just don’t last long enough to get past the deeply stupid stage that follows the enlightenment. Whether we do it to ourselves, or Giant Asteroid ’20 does the job for us, the bell is tolling.

I know, I’m a cheery sort of bloke, eh? You look at the news. I’ll go back to the small shit that I have control over. Like …

The garden is alive. Almost two weeks and nothing’s dead yet. On Saturday I’ll take off the bird netting, weed out the beds, and set the tomato plant cages. Yay!

SSH Agent Persistence vs CSH/TCSH

Y’all can skip this part if you want. It’s here as much for me to find it again if I ever need it as for anyone in Greater OutThereLandia.

The problem is “simple”. I have a group of systems I’m responsible for. The developers have a whole suite of scripts and processes based on the fact that they use the modern incarnation of the C shell, tcsh. I’m migrating them to newer platforms, and newer code repositories, remote code repositories. Access to these repositories requires SSH access. SSH key pairs are the answer to the problem, but the private key requires a passphrase to meet our security requirements.

Persistence across multiple login sessions is the key for the developers. Once they have an active login session on the platform, they want to leverage their SSH authentication without re-entering the passphrase each time it’s needed, or even each time they login (simulateous sessions) on the system.

All the tools I have laying about for managing persistent SSH keys across multiple sessions are pretty much Bourne shell based, and mostly I use a tool called keychain, written a while back by Daniel Robbins of Gentoo fame. I needed to find out how other people are solving this problem for csh/tcsh users.

What I learned is that not very many people are using csh/tcsh anymore. BUT. But, I found Mark A. Hershberger, who many moons ago wrote a page about managing SSH Agent via scripts, with a link to a sub-page with solutions for alternate shells (like tcsh/csh) – http://mah.everybody.org/docs/ssh-agent-startup … yes, that’s a plain HTTP link, no SSL, and your browser will hate it. Don’t worry, you’re not logging in or doing any banking there.

This was an awesome find for someone like me that spends little or no time in tcsh, but knows that it’s a bit of a janky environment for scripting things. I read, understood, and implemented the script as written. It didn’t work. Sigh. After a couple of hours complete with gnashing of teeth and pulling of hair, it turns out I was getting clobbered by a default feature of the environment called noclobber, which effectively prevented me from overwriting a file that already exists via redirection. So, if noclobber is set (which is part of the environment for these devs), then this code won’t work, if the target file, /tmp/blue.txt, already exists:

echo "I\'ve got the blues!" > /tmp/blue.txt

Once I learned about noclobber, and determined that I could not unset it and leave it unset because of user expectations, I found that I could force the overwrite with the judicious application of a ‘!’ character to decorate the redirection. This works, whether or not /tmp/blue.txt exists:

echo "I\'ve got the blues!" >! /tmp/blue.txt

With that problem out of the way, I was able to get the code to run at login. Then I started piecing together the logic I wanted to actually apply for these development users. For login, look for a file that defines an existing SSH Agent session. If that file doesn’t exist, or if it contains information about a defunct session, it will start a new SSH Agent session. IF there’s a running SSH Agent session, it’ll check to ensure the key is loaded, and prompt to load it if needed. So, this code goes into the users .login file in their home directory:

set sshAgent=/usr/bin/ssh-agent
set sshAgentArgs="-c"
set tmpFile=~/.ssh/ssh-agent-info
#
Check for existing ssh-agent process
#
if ( -s $tmpFile ) source $tmpFile
  echo $SSH_AGENT_PID
  if (! $?SSH_AGENT_PID ) then
    # echo "No $tmpFile, starting new agent…"
    $sshAgent $sshAgentArgs | head -2 >! $tmpFile
    source $tmpFile
    echo "ssh agent started [${SSH_AGENT_PID}]"
    ssh-add
else
  # the tmpfile was present with data, check it…
  # echo "Found $tmpFile, check data"
  set this=`ps -elf | grep ${SSH_AGENT_PID} | grep ssh-agent`
  # start ssh-agent if status is nonzero
  if (( $? != 0 ) && ( -x "$sshAgent" )) then
    # tmpFile exists, but stale data
    $sshAgent $sshAgentArgs | head -2 >! $tmpFile
    source $tmpFile
    echo "ssh agent started [${SSH_AGENT_PID}]"
    ssh-add
  else
    # Agent running, ensure a key is present
    set sa_data=`ssh-add -l`
    if ( $? != 0 ) then
      # need to add key
      ssh-add
    endif
  endif
endif

The original script for exiting the session would kill the SSH Agent outright. This is not so useful if you still have other login sessions running. So I wrote a few lines to attempt to ensure that only when the last running login was being exited, would the SSH Agent be reaped. This code goes in the user’s .logout file in their home directory:

set tmpFile=~/.ssh/ssh-agent-info
set sessCount=`w | grep $user | wc -l`
if ( $sessCount == 1 ) then
  # last user, clear out the ssh-agent
  eval `ssh-agent -c -k`
  /bin/rm $tmpFile
endif

And the solution works. The developers are minimally unhappy about the increased security wrapped around access to the code base, because they know I worked to make it as painless as possible while meeting policy requirements. It could always be more robust, but I tried to get all the common failure cases, and mostly the resolution to something I didn’t catch is for the user to log out of all of their sessions, then log back in again to reset.

Winding Down

Our condolences to the family and friends of 1st Lt. Trevarius Ravon Bowman, 25, from Spartanburg, South Carolina, who died on May 19, 2020, in Bagram Air Force Base, Afghanistan, from a non-combat-related incident.

Our hearts go out the families who have lost loved ones to COVID-19. Y’all, socially distance, wear a mask when you need to (indoors around other people for sure), and WASH YOUR DAMNED HANDS way more often.

Please don’t be one of the people who want to go into retail (or any other) businesses without a mask. Don’t be one of the people trying to up your chances of being DEAD by the time the General Election rolls around. Even if you don’t care for yourself, do you want to get ill, pass it on to an elderly parent (or a young child) and have them DIE because you’re behaving like a petulant child yourself? No, no, I understand that your role model in this case does indeed behave much like a petulant child. Don’t follow that particular lemming off the cliff. Stop. Think. Listen to medical professionals and keep safe.

I love you ALL.