Living fearlessly with a laptop

My Thinkpad X61s is tiny and powerful, so I like to take it places with me. Of course, when you go places with a laptop, there is a risk of it getting lost, broken, or stolen. For me (as for most people, I would guess) the integrity of the data on my laptop is far more valuable than the cost of the laptop itself. In fact, there are two measures I take which almost entirely mitigate the risks of getting my laptop stolen (lost, broken, etc.). Consequently, I have become more willing to bring my laptop places. Here is what I do:

Use hard disk encryption. The Alternate installers in Ubuntu and Debian give you the option of easily configuring an encrypted hard disk. Everything (except the /boot partition, but even your swap) that goes onto the disk is transparently encrypted. You just need to type a password whenever you boot up your computer. There is some overhead associated with encrypting everything, but if you have more than one core you will rarely notice it.

This means that, should your laptop fall into the wrong hands, no useful information whatsoever can be extracted from the hard disk.

For this to work well, you need to lock your screen when you are not using it, and your computer needs to be configured to lock the screen when you wake from suspend. It should be noted that some attacks have been described on hard disk encryption techniques. While the risk of these attacks remains low for most targets, if you are paranoid you would have to shut down (not just suspend) your computer before taking it places. You would also have to consider either overwriting the most sensitive areas of memory before you shutdown, or leaving your computer powered off for a couple of hours before taking it anywhere.

Keep backups. I do my backups over the internet so that I can backup from anywhere. I use a variant on the following script:

rsync -vaxE --delete --ignore-errors --delete-excluded --filter="merge excluded-files" /home/phil/ remotehost:/path/to/backup/destination/

where excluded-files is a file that looks like this, and contains some paths that I don't want backed up (usually, local cache-like places that are generally space-consuming and not terribly useful):

- /.local/share/Trash/
- /.mozilla/firefox/*/Cache/
- /.thumbnails/

I run this about as often as I can remember to, and before I shut down my laptop to take it somewhere. That's all there is to it.

With this measure, I can be quite confident that were my laptop to vaporize, I would lose nothing at all. It has the fortuitous side effect of making it super easy to reinstall an operating system.

Linus Torvalds on Git

I finally got around to watching the video of the tech talk that Linus gave at Google discussing the design of Git.

In this video, Linus explains a lot of the advantages of using a distributed system. But it is also enlightening because it's a window into Linus's motivations: he discusses the ways in which his own needs— as a system maintainer— drove the design of the system, in particular in the areas of workflow, speed, and data integrity.

One interesting idea is that in DVCS, the preferred development workflow (you pull from a small group of people you trust, who in turn pull from people they trust...) mirrors the way humans are wired to think about social situations. You cannot directly trust a huge group of people, but you can transitively trust many people via a web of trust— a familiar concept from security. A centralized system cannot scale because there are n2 pairs of conflicts waiting to happen, and they will happen, because groups of people are distributed (not everyone is in the same room at the same time on the same LAN). But a DVCS workflow can scale, because it is fundamentally based on interactions between people and not on the artificial technical requirement that there has to be a single canonical place for everything.

Warning: Linus has strong opinions. I think he refers to at least three different groups of people as "ugly and stupid" in the course of his 70-minute talk.

A million lines of Lisp

People rave about Lisp, and one reason why is because you can use macros to make new kinds of abstractions. Sure, in other languages you can write functions to define new procedures, but in Lisp you can write macros to define new control flow constructs (for starters). You can write functions that write functions and programs that write programs, a power which is pretty much unimaginable in most other languages. In other languages you can only stack your abstractions so high, but in Lisp, the sky's the limit. Because of macros, and others of its features, Lisp is sometimes called the programmable programming language.

I suspect that because of macros, Lisp programs grow in a way that is qualitatively different from programs in other languages. With every line of additional code in Lisp, you can do more, more quickly; in ordinary languages, the return on additional lines of code is constant if not decreasing. Various people have tried to estimate the advantage that Lisp has over C++ (in the ratio of the number of lines needed to do a particular task). Whatever the figure, I believe that Lisp's advantage should increase for larger and larger programs.

In pretty much any language, a program with a million lines of code is regarded with considerable respect. So if Lisp can do so much more with so much less, what would a million line Lisp program look like? Could we even imagine it? (Would it be sentient, or what?)

It turns that such a program exists, and is widely available.

As of right now (23 June 2008), a checkout of GNU Emacs has 1,112,341 lines of Lisp as well as 346,822 lines of C.

It is somewhat astonishing that Emacs, a program with more than 30,000 built-in functions— in a single namespace&mdash can keep from falling apart under its own weight! In Emacs, activating minor or major modes, or setting variables, can have effects on how particular buffers look, what is done when Emacs saves a file, and how different keys (or other events) are interpreted, among other things. Unlike operating systems, which are broken down into programs (which only talk to each other in limited ways), Emacs has many parts which all have the potential to interact with each other.

In some cases it is necessary to add special-case code to say what should happen for particular pairs of interacting features, but a robust system should be able to make the right thing happen most of the time even for features which are totally oblivious of each other. One way that Emacs tames this complexity is with the use of hooks.

Hooks (sometimes referred to elsewhere as listeners) are a way of telling Emacs to run a particular piece of code whenever a certain situation happens (e.g. when a file is opened). Many Emacs modules add hooks in order to do their thing. For example, VC checks every time I open a file whether it is in a version-controlled directory, so that it can report the file's version and status. Another example: Emacs activates follow-mode— if I have set a particular option— whenever I open a file. The variable find-file-hook is a list containing, among others, two functions responsible for performing the tasks described above; each function in that list is executed whenever Emacs opens a new file. I were to add a function of my own to one of these hooks, then Emacs would happily run it, too.

As an alternative, you might consider implementing such an extensible system using polymorphism and inheritance: e.g. to add behaviors, you would create a subclass of a particular class which implemented some method differently. The chief advantage of using hooks over that approach is that with hooks, changing behavior is very obviously additive: if you can call for either A, B, or C to happen in a particular situation, you can also easily call for any combination of those things, but A, B, and C can very well be implemented completely independently— a key requirement for systems of a million lines or more.

My Top Ten Essential Emacs Tips

I wrote a brief article detailing what I consider to be the top ten can't-live-without-them Emacs features.

Switching to Openbox

I switched window managers recently, to Openbox. I also switched panels from gnome-panel to pypanel.

My new setup has these chief advantages over the old setup:

  1. Speed. Going from GDM to a desktop that is ready is much faster than it was under Gnome. I blame gnome-session and the gnome-panel.
  2. Openbox is much more flexible.
    • I can rebind the window management keys, so I'm now using the Windows key (which was previously not seeing much use). I've bound W-Tab to serve the same purpose that Alt-Tab used to. I've bound W-j, W-k to switch virtual desktops, and W-1, W-2, and other keys to do window management tasks (maximizing, minimizing, etc.). No more reaching for the arrow keys or the function keys for complete common tasks. And it frees up Alt-Tab, which Emacs uses.
    • I can bind keys to other actions: for example, W-c starts up a terminal and W-b starts up a web browser.
    • I can bind mouse actions, too. I've set up W-drag to move windows (just as Alt-drag does usually), and W-right-drag, which resizes my windows, is a lot better than trying to drag window borders (hello, Fitt's Law!).
    • The configuration files can be easily version-controlled, so I synchronize my Openbox settings everywhere. This is not true of my Gnome/Metacity setup.
  3. Space. I turned off window decorations to save space. Who needs window decorations? I don't need them to see what application I'm using because I have a panel. I don't need them to move windows because W-drag is faster. And I don't need them for window context menus because W-SPC already gets me one.

To install it on Debian/Ubuntu: apt-get install openbox

The kicker is that Openbox was apparently designed with Gnome interoperability in mind. And it is easy to switch back and forth while you are testing. All you need to do is log out to get back to GDM and then select either the "Gnome" or "Openbox" sessions from the Session menu. No futzing with files in your home directory to select your session. Moreover, the default Openbox setup loads a bunch of stuff that your GTK+/Gnome apps need to work. So your GTK themes and Gnome settings will be just fine, D-Bus and other session services will be launched as normal, and all that. It was suprisingly painless to switch.

To configure Openbox, after installing it, copy the XML files in /etc/xdg/openbox to ~/.config/openbox. Check out the default configuration to see what you are in for. Remember that when you log in to Openbox, right-clicking on the Desktop will probably get you enough to get out of a jam. Enjoy!

Installing Ubuntu from hard disk + grub

Previously, I praised Debian for supporting an installation method ("hard disk booting") that only requires an existing filesystem and Grub, and can be kicked off just by downloading two files and asking Grub to boot from them. This is really convenient because you can install a full system without a CD (or USB key, or any media), without PXE (or configuring any other hosts on your LAN), and with a download that is quite small.

Well, it turns out that Ubuntu supports this installation method too (which is, now that I think about it, not surprising). It just doesn't seem to be advertised anywhere! (I suppose that instead of praising Debian I should praise the Debian documentation.) I just used it to install Ubuntu with encrypted LVM on my Thinkpad X61s, a machine which would otherwise be nontrivial to reinstall because it has no optical drive.

Here are the links to the downloads (the "netboot" installer): i386, amd64. Download initrd.gz (disk image) and linux (the kernel). Make sure you put them somewhere Grub knows how to get to (i.e. not to a networked or encrypted volume; /boot is a good place). Then do the following to boot into your new installer (more complete instructions from Debian):

  1. Restart your computer and wait for Grub to load.
  2. Find some existing boot entry and press e to edit it.
  3. Edit the root line to make sure that it corresponds to the partition where you downloaded the files. It might already be correct.
  4. Edit the kernel line so it reads kernel /boot/wherever/you/put/linux (on recent grubs, you may have to use linux instead of kernel).
  5. Edit the initrd line so it reads initrd /boot/wherever/you/put/initrd.gz
  6. Press b to boot.

Enjoy your new installer!

Update, 19 Jun 2008: I have documented this procedure in the Ubuntu Wiki here.

Using wget or curl to download web sites for archival

wget is useful for downloading entire web sites recursively. For archival purposes, what you want is usually something like this:

wget -rkp -l3 -np -nH --cut-dirs=1 http://web.psung.name/emacstips/

This will start at the specified URL and recursively download pages up to 3 links away from the original page, but only pages which are in the directory of the URL you specified (emacstips/) or one of its subdirectories.

wget will also rewrite the links in the pages it downloaded to make your downloaded copy a useful local copy, and it will download all page prerequisites (e.g. images, stylesheets, and the like).

The last two options -nH --cut-dirs=1 control where wget places the output. If you omitted those two options, wget would, for example, download http://web.psung.name/emacstips/index.html and place it under a subdirectory web.psung.name/emacstips of the current directory. With only -nH ("no host directory") wget would write that same file to a subdirectory emacstips. And with both options wget would write that same file to the current directory. In general, if you want to reduce the number of extraneous directories created, change cut-dirs to be the number of leading directories in your URL.

Bonus: downloading files with curl

Another tool, curl, provides some of the same features as wget but also some complementary features. One thing that curl can do is to download sequentially numbered files, specified using brackets [..]. For example, the following string:

http://www.cl.cam.ac.uk/~rja14/Papers/SE-[01-24].pdf

refers to the 24 chapters of Ross Anderson's Security Engineering: http://www.cl.cam.ac.uk/~rja14/Papers/SE-01.pdf, http://www.cl.cam.ac.uk/~rja14/Papers/SE-02.pdf, etc., http://www.cl.cam.ac.uk/~rja14/Papers/SE-24.pdf.

You can give curl a pattern for naming the output files. For example if I wanted the files to be named SE-chapter-01.pdf, etc, then the appropriate curl incantation would be:

curl http://www.cl.cam.ac.uk/~rja14/Papers/SE-[01-24].pdf -o "SE-chapter-#1.pdf"

In addition to specifying consecutively numbered files, you can also use braces {..} to specify alternatives, as you would in a shell, e.g. http://web.psung.name/page/{one,two,three}.html. Specifying output patterns with "#1" works with braces too.