Here’s a quick tip on how to make an fsck (or specifically an fsck_hsfs) run much faster. I learned this as part of debugging some corruption with the backup image on my Time Capsule.

/bin/fsck_hfs -f -d -c {%mem} /dev/rdisk{N}

where {%mem} is ½ to 1⁄3rd the amount of memory you have in your computer. If you need to determine the amount of memory in a machine from the command line, a great blog post on Mac OS X Command Line Goodies suggests you can use:

/usr/sbin/system_profiler SPHardwareDataType | grep Memory

For example, you can use -c 512m if you have a 1GB machine.

and

{N} is the correct device node. This command will show you the list of disk nodes:

diskutil list

For example, you might use /dev/rdisk0s2.

The -d turns on debugging and -f instructs fsck_hsfs to force a repair if it finds any problems.

In case you were curious, the -c setting is the part of the incantation that gives it its real magic. It lets the command cache a considerable amount of information in memory and that speeds up any scan a tremendous amount. If you ever run a disk repair from Disk Utility, you’ll notice the fsck_hsfs command it spawns lacks this option and hence is much slower.

Just had to deal with a total swap out of all of my SSL certificates at work thanks to the ipsCA screwup that also nailed Bob Plankers and Chris Siebenmann and I’m sure lots of other sites (especially .edu). I’m really peeved too, but rather than hissing and spitting like I really feel like doing, let me pass on a few SSL resources that I found useful while debugging the new chained certs we started using:

• Debian’s SSL keys page is a lovely resource for where and how each common open source package keeps its particular SSL-related configs. Your file system locations may vary but this at least gives you a start for the 30+ packages they list
• UnixCraft’s How To Verify SSL Certificate From a Shell Prompt is a swell tutorial on the various openssl command line options you can use to debug certs.
• Paul Heinlein’s OpenSSL Command-Line HOWTO is a great HOWTO for all sorts of openssl operations.
• The OpenSSL verify manual page lists the error messages you might get when attempting to verify, for example, a chained cert.
• SSLShopper.com, in addition to being a good place to compare certificate authorities, has a surprising amount of really good technical information about all things SSL. For example, The Most Common Java Keytool Keystore Commands and The Most Common OpenSSL Commands are genuinely useful.

Hope this are helpful to you should you get put in the same bind.

Back in the days of yore (when sysadmins still had to bang two rocks together to do their job), we didn’t have command line history. No, we typed every command by hand. If we had to type the same 110-character command-line several times, we typed those 550 characters ourselves, mistakes and all, and by gum, we LIKED it!

And then shells became available with history functionality and, well, we kind of liked that too. And, um, then came along libraries like GNU readline that were written to provide even more pleasant, vizz-eee-ul history recall and editing to anything that presents a command line prompt, and well, ok, we REALLY that too. Because, quite frankly, typing long command line strings over and over again, sometimes because our typing skills aren’t always sharp, is a real drag.

But alas, not every program that provides a command-line prompt or handles interactive input provides this new-fangled user experience (I’m looking at you sqlplus, maybe if scott had a real tiger…). In the past, a sysadmin had to look admonishingly at one of these programs, perhaps working around the lack of interface using something like Expect.

Maybe I’m the last admin on the planet to hear about this idea, but it came as a pleasant surprise to find out that there are a number of programs out there whose whole purpose in life is to add the readline-yumminess to programs that don’t have it already. The first one I encountered was rlwrap by Hans Lub. Now you can type:

$ rlwrap sqlplus scott/tiger@oracle_instance

And you get a reasonable command-line environment even from within sqlplus.

I know I would have offered at least a vestigial body part several times in the past to have the “add readline to command X” super power several times during my career. (apparently this tip has been making people in the Oracle community happy for a while now).

The rlwrap README mentions two other similar packages: rlfe which even ships with the GNU readline distribution and cle. I note that cle hasn’t been touched for over ten years (the last release’s 10th birthday was just over a week ago). I can’t speak to how well rlfe works, but kudos to Lub for continuing to update the rlwrap package.

Hope this tip brings you a similar amount of joy.

I spent the good part of my morning today debugging an issue with a user’s access to a Subversion repository/trac service we provide. That person could not get even a basic svn checkout to work as expected. It would keep throwing errors like this:

Server sent unexpected return value (403 Forbidden) in response to OPTIONS request for {url}

I went up and down the stack, I poked and prodded the two web servers in question (Apache 2 and Lighttpd) and their authentication sources with multiple clients. I watched packet traces. I checked and rechecked every config I could lay my hands on to no avail. It was driving me crazy.

Finally, I realized that the mod_dav_svn file semantics for paths in an svn.access file distinguished between:

[repos:/dir/]
user1 = rw
user2 = rw

and

[repos:/dir]
user1 = rw
user2 = rw

The first only works if you’ve explicitly granted the same permissions to the parent directory as well, i.e.:

[repos:/]
user1 = rw
user2 = rw

while the second “does the right thing” to allow the users to have access to that path component without having to permit the parent directory as well.

Hopes this saves you some time.

We just purchased a few new phones to replace our aging tin cans on a string at home. On the sage advice of some other techies I know, we purchased a few phones based on the DECT standard. IMHO, one of the coolest things about the DECT phones is they freakin’ talk to each other.

I know this sounds like one of those “Primitive man fascinated by shiny objects” little exclamations (it sounds so ridiculously naive to even me as I say it, especially given my background), but hey, it is sooo COOL that the headsets sync stuff like phonebook information amongst themselves (I guess via the base station) and I didn’t have to do jack to make that happen besides provide one POTS connection and plug in a few power bricks. I guess I’m so used treating every single handset in my life as a completely separate entity that this idea really impressed me. I wanted to walk into the kitchen and stare reproachfully at all of the appliances there and say “So, what’s wrong with you guys? How come you aren’t sharing information? The phones can do it.”

The one other cool, though less unexpected thing these phones do is treat the caller id information that comes in every time the phone rings as something I might care about. I think it is a no-brainer that I might want to use this information to add information to the phones’(!) phonebooks or block subsequent calls from a certain caller and now that’s available. They also keep their internal clock in sync with the telco’s signals, another nice touch.

I suppose you get all of this cool stuff if you start to play with VoIP (e.g. get into the Asterisk game), but for a consumer experience it was a surprise.

No, really. They talk to each other.

P.S. Warning: DECT phones have some relatively recently discovered security vulnerabilities (I know, what doesn’t? but they were new to me, I hadn’t seen them in my usual infostream). It looks like the best source of info about these vulnerabilities is this site here. See the thesis published (right hand column) for the best description of the issue and a list of phones they tested.

Well, I think the book has hit the big time because I randomly stumbled on not one, not two, but four separate places where people had uploaded my book to some of the grey matter (both kinds) sharing sites. In particular, there’s a Thai-speaking user whose been very busy being naughty. At least one site I found had a very pretty web gui for downloading the book. And while I admire the pluck of people who work that hard to make it easy for other people to engage in less than savory activities, it still leaves me feeling a little uneasy and perhaps a bit sad.

I’ve ready my fair share of articles about how “illegal” distribution can have all sorts of positive effects on book sales and other promotional activity which is why I’m not advocating the villagers grab their pitchforks and smokey torches and join me in storming these sites to demand justice. But you won’t find me giving my blessing any time soon.

If you happened to have stumbled upon this site through a chain of events related to one of these ehemm, distribution points, I’d ask two things:

• Enjoy the book. There’s a Yiddish phrase that roughly translates to “If you are going to do something wrong, at least enjoy it.” I didn’t write the book for the money, I wrote it so people could learn from it and take a little pleasure reading it. (though the small royalty check is kind of nice too).
• Keep karma neutral. You don’t even have to right things by me by buying a copy of my book or attending one of my classes. Just go do something giving to offset the taking. It will make you, me and the person you give to happier.

Thanks.

’bout time I add some sysadmin content here.
If you ever have a trac instance go boom with an error like “DatabaseError: database disk image is malformed” you can often fix the problem by doing an sqlite dump/reload:

$ cd /path/to/trac-sqlite-db-directory # find the trac.db file
$ su www-data   # switch to the owner of the db file
$ mv trac.db trac.db.damaged
$ sqlite3 trac.db.damaged .dump|sqlite3 trac.db

If you’ve read Jeffrey Friedl’s Mastering Regular Expressions¹ you’ll recall that one of the things that is a lot harder than it looks to get right using regexps is extracting the content from within some text-delimited string, especially when there are opening and closing delimiters².

Text::Balanced originally by Damian Conway and now maintained by Adam Kennedy makes short work of this extraction for:

• strings delimited by the same single character
• strings delimited by brackets of some sort (parens, square brackets, etc.)
• strings delimited by XML or other kinds of tags
• strings delimited by Perl quoting operators
• code blocks

Each kind of delimited string has its own function, e.g. extract_bracketed(). For the most part, they all work like this:

my ($extracted,$remainder) =
        extract_something($extract_from, $delimiter, $opt_prefix);

where you have a function called extract_something() that takes the string to extract the data from, the delimiter that surrounds the data in question, and an optional prefix. That last argument reveals a part of Text::Balanced that tends to confuse people so we’ll look more at it in a second. The extract_something() functions return (in a list context) the first extracted piece of data and what was left over or just the extracted data in a scalar context (with alas, another twist).

There are two things about this module that tend to trip up newcomers:

1. By default (i.e. without the optional third argument), the extraction functions all expect the data they are going to extract to be found either right at the beginning of the string or right at the position in the string that the last extraction left off. If you expect it to skip over non-delimited data that isn’t just whitespace, you will have to provide that third $opt_prefix argument.
2. When called in a scalar context, the extraction functions eat the text. Extracted strings are removed from the input text. Fans of functional or immutable data structure programming will not be pleased.

If you do want to do split() on steroids kind of stuff, Text::Balanced also offers a extract_multiple() function that takes a list of extraction functions, each of which gets run over the string, returning what they collectively find. Text::Balanced can also make Friedl proud by generating optimal regular expressions for balanced matches that you can use with a minimum of head scratching.

If you need something to extract data matched delimiters of almost all sorts, this module will be spot on for you and will do its job like a laser beam.

Today’s interesting Perl module of the Day:  Devel::Deprecate by Curtis “Ovid” Poe.

Back in 2000, I started teaching a class called “Perl Saves the Day: Writing Small Perl Programs to Get Out of Big SysAdmin Pinches” which was essentially a class about Perl hacks and how to use them responsibly in System Administration. One of my slides was “How to Get Rid of Hacks,” which started with:

It can be hard. You may have to wait for the rewrite.
Step one: remember you did it.

• Go back and document the code after the crisis.
• Send yourself mail.
• Set up an AT job.
• Put it in your calendar.

Now there’s an even cooler way: Devel::Deprecate. Devel::Deprecate provides a deprecate() function that let’s you write code like this (to quote the doc):

deprecate (
    reason => 'Please use the set_name() method for setting names',
    warn   => '2008-11-01',    # also accepts DateTime objects
    die    => '2009-01-01',    # two month deprecation period
);

deprecate() only comes into play when the code is run during a test (which you are writing, right?). Each time it is run under this condition (and only this condition, it never comes into play when the code is run outside of a test), it produces output that is crystal clear:

# DEPRECATION WARNING
#
# Package:     Our::Customer
# File:        lib/Our/Customer.pm
# Line:        58
# Subroutine:  Our::Customer::name
#
# Reason:      Please use the set_name() method for setting names
#
# This warning becomes FATAL on (2009-01-01)

After the due date, it blows up just as promised with an equally verbose message. Very cool.

Kudos goes to Thomas Leyer of Germany for finding the very first typo in the printed version of the book. Books like these have an amazing amount of detail so it is not uncommon for a few small things to slip through (in this case it was a single missing dash).

I’m very thankful when readers take the time to report errors they find so I can correct them in a future printing. I really appreciate it when others are willing to help me make this book better in whatever fashion I can. If you do spot something, please report it through the book’s errata page. I’ll get a little note if you do and then I can make sure it is dealt with appropriately.

Thanks, Thomas!