Warning: dumb joke modified from a dumb joke that used to contain a character of a certain ethnicity. Also note: this is not representative of lawyers in general so please do not sue me.
A lawyer and an SysAdmin are sitting next to each other on a long flight. The lawyer is thinking that SysAdmins are so dumb that he could get an advantage over on them easy…
So the lawyer asks if the SysAdmin would like to play a fun game. The SysAdmin is tired and just wants to take a nap, so he politely declines and tries to catch a few winks. The lawyer persists, and says that the game is a lot of fun. “I ask you a question, and if you don’t know the answer, you pay me only $5; you ask me one, and if I don’t know the answer, I will pay you $500,” he says.
This catches the SysAdmin’s attention and to keep the lawyer quiet, he agrees to play the game.
The lawyer asks the first question. “What’s the distance from the Earth to the Moon?”
The SysAdmin doesn’t say a word, reaches in his pocket, pulls out a five-dollar bill, and hands it to the lawyer.
Now, it’s the SysAdmin’s turn. He asks the lawyer, “What goes up a hill with three legs, and comes down with four?” The lawyer uses his laptop and searches all references he could find on the Net. He sends e-mails to all the smart friends he knows, all to no avail. After one hour of searching he finally gives up. He wakes up the SysAdmin and hands him $500. The SysAdmin pockets the $500 and goes right back to sleep.
The lawyer is going nuts not knowing the answer. He wakes the SysAdmin up and asks, “Well, so what goes up a hill with three legs and comes down with four?”
The SysAdmin reaches in his pocket, hands the lawyer $5 and goes back to sleep.
Don’t mess with SysAdmins!
I’m all for performing service management using mechanisms built-in to a configuration management system (i.e. you change a config file and the config management system automatically restarts the daemon), but occasionally you get into a situation where you want to interactively do the same exact thing on several machines.
For example, I recently wanted to bounce the ClamAV daemon running on four of my incoming mail servers simultaneously. This process would require the exact same commands on all four machines, ideally run at the same time.
Rather than type the same thing four times, let me tell you about a few programs that can help in this sort of situation where you need interactive input to multiplex to different sessions. The one I used for the last example was OSX specific (since most of my time is spent typing at a Mac laptop):
- iTerm – iTerm is a spiffy rewrite of the native OSX Terminal application that has a bunch of cool features. The one I used is called “Send Input to all Tabs” (under the Shell menu). iTerm lets you run multiple sessions in different tabs of the same window. When you turn “Send Input to all Tabs” anything you type in one tab is automatically echoed to every other tab. I ssh’d to each of the mail servers, turned on this feature, and typed the commands I needed a single time, yet all four machines obeyed these commands. If you run OSX and you haven’t played with iTerm, you really should.
Before I discovered this feature in iTerm, I used to use cssh, from ClusterSSH. It opens up a smaller input window and as many other xterms as you need. Anything typed in the input window is echoed to those windows. This program isn’t OSX specific, it can be used any place you can build the right Perl modules. I stopped using it a while back because I had issues with it building under MacPorts (if I built the modules it needed using the Apple-provided Perl libraries, everything was peachy, but building it into MacPorts in an attempt to avoid polluting the OSX install of Perl lead to a script that would segfault upon running).
I recently noticed a new rewrite of the same idea for OSX called csshx. It uses the native OSX terminal vs. X11. I haven’t tried it, but it looks promising. I suspect there are other utilities like this available. I wouldn’t be surprised if there was a way to do this with GNU Screen or tmux. Are there any you use for this sort if interactive multiplexing that I’m missing?
Sorry I left this blog languish for a bit. Let me catch you up on a few things that have happened in my professional life in rough chronological order:
- I was quite surprised (stunned really, ask me about it some time) and tremendously honored to receive the 2009 SAGE Outstanding Achievement Award at the last LISA. For those who were not at the 2009 LISA, you missed my 1985 Sally Fields acceptance speech imitation.
- It was great to have the opportunity to present to the other sysadmins in the ‘hood at the January 2010 BBLISA meeting.
- At a friend’s urging, I ran for and was elected to a seat on the USENIX’s Board of Directors. If you haven’t heard of USENIX before, it is the organization that sponsors the LISA conference (and some other great ones). Come June, I look forward to serving USENIX and the wider sysadmin community in that capacity to the best of my ability. Thanks to those who voted to give me that opportunity.
- I’m also very honored to be asked to give the opening keynote session at the upcoming PICC Conference sponsored by LOPSA-NJ. I’ve also been asked to teach a class there based on this website’s favorite book. PICC is the first regional conference I’ve attended in a while and I think it is going to be great.
That’s what is up with me. How are things with you?
Here’s a quick tip on how to make an fsck (or specifically an fsck_hsfs) run much faster. I learned this as part of debugging some corruption with the backup image on my Time Capsule.
/bin/fsck_hfs -f -d -c {%mem} /dev/rdisk{N}
where {%mem} is ½ to 1⁄3rd the amount of memory you have in your computer. If you need to determine the amount of memory in a machine from the command line, a great blog post on Mac OS X Command Line Goodies suggests you can use:
/usr/sbin/system_profiler SPHardwareDataType | grep Memory
For example, you can use -c 512m if you have a 1GB machine.
and
{N} is the correct device node. This command will show you the list of disk nodes:
diskutil list
For example, you might use /dev/rdisk0s2.
The -d turns on debugging and -f instructs fsck_hsfs to force a repair if it finds any problems.
In case you were curious, the -c setting is the part of the incantation that gives it its real magic. It lets the command cache a considerable amount of information in memory and that speeds up any scan a tremendous amount. If you ever run a disk repair from Disk Utility, you’ll notice the fsck_hsfs command it spawns lacks this option and hence is much slower.