These were the most linked-to posts on my blogs in the last year:
Happy new year, peace on earth, and may all your builds be green!
The Intel SSDs were on sale around Thanksgiving, so I picked up one for my Thinkpad.
There is little that was remarkable in getting it set up. I swapped out the old spinning rust device. (The X201, unlike any other laptop I've opened up, has rubber shock absorbers that fit between the drive and the chassis. Not that the thing they're guarding is particularly susceptible to mechanical damage any longer.) Everything just worked.
Boot times and application load times are dramatically improved. Though, the way I work, my laptop is essentially just a device for running Emacs, Chrome, and gnome-terminal, all of which I just open and keep open. All of those apps load fairly quickly, so the speed improvement from the SSD, while nice, is not by any means life-changing. (And, how often do you reboot a computer, anyway? But, for the record, my Thinkpad goes from LUKS password prompt to login screen in 10 seconds, and to a ready-to-use desktop in about 3 more.)
What's really nice about the SSD is that my laptop is now nearly silent, or at least quieter than ambient noise. Perhaps I just have unusually low noise tolerance, but I find the steady-noise of a HDD to be somewhat annoying, and the sound of a HDD spinning up to be even more annoying. Unfortunately, I actually didn't correctly attribute those noises to the HDD until recently. If this sounds like you, it's time to pay a visit to the computer store.
One caveat is that if you use LUKS full disk encryption (and if you're using a laptop, you really ought to; no, I mean, you really ought to), then Ubuntu (10.10) doesn't issue TRIM commands to the device. Which will eventually lead to decreased throughput, but I believe that may just be the price of security. (IANA security expert...) LUKS goes to great lengths by default to foil cryptanalysis, including initializing encrypted partitions with random data to obscure which sectors of the device are actually being filled with interesting data. This would all be for naught if the OS was periodically telling the device exactly which sectors were no longer being used.
This is a classic, or, as much of a classic as a 3-year-old graphics paper can be.
Seam Carving for Content-Aware Image Resizing (Avidan and Shamir, 2007) describes a technique for resizing photographs along one dimension in a way that allows the aspect ratio to be changed while reducing the amount of distortion in perceptually important parts of the image. Watch the video, if you haven't seen this before:
The algorithm translates into identifying the shortest path in a DAG and it can be easily implemented by a dynamic programming algorithm. As Dmitry pointed out to me some years ago, this makes an ideal teaching example for dynamic programming, for a number of reasons. The algorithm is easily visualized because the space of values to be computed maps straightforwardly onto the pixels of the image. Both the naive exponential-time algorithm (testing all 3#rows paths) and the dynamic programming algorithm are fairly easily expressed without too much additional machinery. And, unlike most classroom dynamic programming questions I have encountered, which seemed either banal or contrived, the applications of this one are visually appealing and fairly interesting. For example, as someone pointed out to me, it makes the fallout of breaking up much easier to deal with.
Before... and after (Source: video, above)
I've been using a Nexus One as my primary phone for about a year now. Here are some tips and suggestions I've accumulated. Most of these tips should also apply to the Nexus S and the G2 (which have stock Android builds), as well as to other Android phones (Froyo+) modulo any vendor customizations.
ZXing's Barcode Scanner Android app (the most popular barcode app for Android; it's available on the Market) can now configure a wifi network based on information encoded in a QR code.
So you can say goodbye to having guests/patrons/visitors ask you how to log in to your wifi network every time someone new visits. Instead of having to select the correct network and type in the password (I thought computers were supposed to relieve us of this kind of drudgery), all they have to do is scan a barcode, and boom, they're online.
ZXing's QR Code Generator will help you make such a barcode (select "Wifi network" in the dropdown), after which you can just print it out and leave it in the living room.
If you wish, you can also create such a barcode yourself just by encoding the SSID and password in the barcode payload. Here are a couple of examples that illustrate the encoding method:
|
WEP: WIFI:S:mynetwork;T:WEP;P:00deadbeef;; |
WPA: WIFI:S:mynetwork;T:WPA;P:mypassword;; |
Hat tip to Vikram Aggarwal who implemented this concept, originally as a standalone app, WyScan, and then later implemented it in the Barcode Scanner app.
bash has a lot of shortcuts to save you typing and thinking time— globs, shell aliases, and history expansion, for starters; and a bunch more.
When you use bash shortcuts extensively, though, you often end up with inputs that are rather, er, opaque, e.g.
$ !-3 -r [mnop]* !-3$
You have to be fairly brave to press ENTER without looking over that a few times. There is, though, a way out. Instead of trying to analyze what the hell it is you just wrote, you could just ask the computer to tell you exactly what it's going to do. bash has a number of features that facilitate this by expanding shortcuts for you to inspect before you execute them. I refer to these features collectively as inline expansion.
There are three really useful inline expansion commands you can use at a bash prompt that are essentially complementary to each other:
History expansion
If you add the following to your .inputrc,
$if Bash
Space: magic-space
$endif
then whenever you press space, any history expansions (e.g. !!, !grep, !-2, !-3:1, etc.) are expanded.
You can also, at any time, press M-^ to perform history expansion (no configuration required). I prefer to set up magic space because you can just set it up and forget it about it, and subsequently you have one less thing to think about.
Glob expansion
C-x * expands the previous word into any filenames that match the glob. For example, /* gets expanded into /bin /boot /cdrom /dev ....
Shell expansion
C-M-e expands shell aliases, history, variables, commands, and arithmetic. For example:
* * * * *
I find these features useful for the same reason I use GNU Parallel the way I do: they really improve the interactivity of my work— in the sense that I get much faster empirical feedback on whether I'm doing a complex task correctly (the computer tells me what it thinks I'm trying to say), rather than me having to reason (unreliably) about what I wrote and recall a bunch of history and read the man page for the umpteenth time because I'm second-guessing myself.
Inline expansion is useful in a different scenario as well, namely, once you expand a glob or a shell alias you're free to (interactively) make changes to it. For example, if you want to delete all the files but one in a directory, start with rm *, press C-x * (which expands the glob into a list of all the files in the current directory), and edit out the name of the file you want to keep. Or if you want to run one of your shell-aliased commands, but without one of the flags, expand it with C-M-e and then edit it.
Further reading: A nice bash tips slide deck by Simon Myers; "Miscellaneous [Readline] Commands" in the bash manual
My post Thank you, Ubuntu attracted some attention (hello, Hacker News!). A couple of commenters were skeptical of my assertion that GNU/Linux is more usable than the major proprietary operating systems— and rightly so, since I didn't really elaborate on it at the time. For example, commenter Dubc wrote:
I think Ubuntu is a fine choice indeed, but you are going to either have to use another word than usability, or redefine the term to meet your presumptions.
So I'd like to expand on a few of the things that make Ubuntu such a pleasure to use, for those who aren't familiar with its ins and outs, especially because I so rarely see people take the time to articulate some of these nifty constructs at a high level.
I will happily grant that at the application level, Windows and Mac OS have a much more consistent base of polished applications. And Apple has an attention to detail and psychology that others would do well to learn from.
It's often said that Apple takes the time to get the little things right. Unfortunately, at times I think they should have instead spent a bit more time thinking through the big things in Mac OS. There are some fairly glaring architectural/design/HCI problems on the Mac (and on Windows, too) that hamper users needlessly. And because these are issues with how one interacts with the system at its most basic levels, no amount of polish on the "little things" can really satisfactorily outweigh them.
What that boils down to is this: yes, you will wrestle with GNU/Linux for a few hours or days or weeks when you get it set up. But if you use Windows or Mac OS, you will wrestle with it every minute of every day you're using it. Forever. And that's just not what "usability" means to me.
Here are some illustrative examples:
apt
An anonymous commenter wrote:
in Mac installing software is a single drag-and-drop function
Oh, if only that were the case. The flow for installing software on Windows/Mac looks something like this (some steps omitted for brevity):
These steps are error-prone, and unsophisticated users will quickly find themselves with malware on their systems. We should help users to do the right thing by default when installing software, and subjecting them to the vagaries of the wide-open web isn't the right thing.
On Ubuntu:
Whatever the heck you want, you can find it in Synaptic. And there's no distinction between software that was preinstalled with your system and software that you choose to install later. It all works the same way and gets security updates in the same way. And it looks like Mac OS is finally starting to head in this direction with the App Store for Mac software.
But, what's more, each of the thousands of pieces of software available in the Ubuntu (or whatever distro you use) repository has an audit trail, and— unlike the apps you get in the Mac App Store, or any "App Store," really— Ubuntu has the means, motive, and opportunity to do whatever it takes to make them "just work," no matter where and when and for whom it was originally written. There is a vast body of software that is just considered to be part of your operating system and is maintained alongside it, and Ubuntu will vouch for that software just as it will vouch for the software at the core of your system.
It's not clear whether you will ever see that sort of large-scale integration work in a proprietary OS. Package management is really one of the highlights of the free software ecosystem.
Remote access
X11 has pretty much decoupled where a program runs and where the program can be used. You have a few basic composable tools— SSH, X forwarding, and port forwarding— and all of a sudden the vast majority of applications can be invoked remotely. So you can, very easily, interact with a program that's using the resources (CPU, filesystem, network bandwidth) of a computer sitting somewhere else. There are a few gotchas with respect to X11 (watch out for high-latency network links, for example), but by and large it works.
"But wait," you say, "Mac OS has SSH and X11 too." Except that most of the apps you want to use can't be invoked remotely, because they aren't X11 apps. So you have a litany of tricks. VNC and NX are appropriate sometimes, but they're clunky. Mac OS has "Back to My Mac" which is a slightly more general tool (and costs $99/year). But typically, whenever you work remotely you have to work differently too. It's a far cry from the situation on GNU/Linux, where you have a critical mass of applications you can use remotely and they just work, more or less transparently.
The result is that Mac and Windows users are typically highly dependent on a single piece of hardware. If you have a nice light laptop, more power to you. They say the best computer is the one you always have with you, after all. But even better than that is not having to carry a computer around. It's incredibly liberating, not to mention convenient. You can actually go places without having to pack a bag.
On GNU/Linux, one barely even perceives the concept of "other computers." Every computer you use, no matter how far away it is, is so accessible to you, so immediate, that it might as well be the one sitting on your lap right now.
Of course, Microsoft and Apple don't really have any incentive to make remote access terribly effective. Since they generally sell you a license to use a particular instance of their software on a particular computer, it would cut into their bottom line to make remote access actually useful. So they won't.
Window managers
The Windows/Mac window managers (the part of your OS that lets you manipulate windows and switch between them) are pretty bad. They weren't always this way— they have become noticeably clunkier on the large displays that are so common today.
On large displays, maximizing (zooming) windows is not useful much of the time. Maximized windows are just too big. Instead you have to move and resize windows to obtain a useful multi-window arrangement on your screen. But Windows and Mac OS give you very little help in this regard. To produce these arrangements you usually have to click on some tiny button/target and manually move windows around.
For example, moving or resizing windows on Windows or Mac OS typically requires you to point at the titlebar or resize handle or window border— both of which are just a few pixels wide.
Fitts's Law? Bueller? Bueller? Having enabled alt-dragging to move or resize on any capable X window manager (I use Openbox, but even the Ubuntu default window manager supports this), the entire window becomes your drag target. The entire window. (Alternatively, if you use a tiling window manager, even the whole concept of manually moving windows around becomes basically moot.)
In fairness to Apple and Microsoft, they have taken some steps in the right direction recently. Windows 7 "docks" windows to one side of the screen when you ask it to. You can configure some recent Macs to let you drag windows with three fingers (or so I've heard). And Mac recently gained Spaces (i.e. virtual desktops) too.
Unfortunately, I think these are the exceptions that prove the rule. Mac OS and Windows users have waited for years only to get a fraction of the window management facilities that you could have set up in your X11 window manager. Supposing you spent an hour setting up your WM to your liking, I figure you would earn back that time in improved productivity in just a few weeks, rather than having to wait years for Microsoft or Apple to implement a sensible workflow. But more importantly, your blood pressure would go down, immediately.
People really are more effective when they can set up their computers to work they way they want. But on Windows and Mac OS they aren't given the tools they need to do so. I wouldn't disparage these operating systems so much if they came with window managers that could at least be configured to be minimally adequate. But they don't.
The window manager is the one program you're using all the time, even when you're using other programs. It's your primary vehicle for multitasking and your command center for managing your work on a second-by-second basis. So it's really important to get this right.
Conclusion
Having read this post, you might decide that you would brand these issues not as "usability" issues but as something else. Which is completely fine by me. A rose by any other name, after all…
Many of the cumbersome rituals that Windows and Mac OS have whittled away at over the years are completely unnecessary in Ubuntu. And this is why, while I've had occasion to use a Mac (at various times since 2004 or so) and Windows (only occasionally), sitting down at one always feels like death by a thousand cuts. They're superficially simpler, but really quite tiresome once you get beneath the surface. It seems that hey are optimized for learnability at the expense of usability. That is is exactly the wrong optimization to make, if you use a computer for a living.
Life is just too short for that.
Now, it's not like Ubuntu is a paragon of usability out of the box, either, though some of the things I mention above do just work by default. The difference— the key difference— is that you can make it into one without much hassle.
I've been using a Nexus One as my primary phone for the last few months now, and I quite like it! Upon reflection, the thing I most like about it, however, is not something I had anticipated at all when I got the phone.
Being able to search the web, read email, run apps, look at maps, and listen to podcasts are all very convenient. Though, there is not much to say about those things: I use those tools in more or less the same way I would use them if I had a laptop with me.
However, being able to send and receive IMs from my phone has been one of a small number of applications that has led to a qualitative change in the way I do things. I'm using IM a lot more now, at the expense of pretty much every other form of communication. And that's because instant messaging is the only mode of communication that actually feels convenient.
You can write and reply whenever you want, not just when it's convenient for the other person (unlike phone calls). You can have rapid back-and-forth conversations (unlike e-mail and voice mail). You can write messages of whatever length you want (unlike SMS). You can communicate unobtrusively, e.g. in a library (unlike phone calls and voice mail). You can read messages without going through an absurd interface (unlike voice mail). Your messages reach you on whatever device you're using— desktop, laptop, or phone (unlike phone calls and SMS). It doesn't cost an exorbitant amount of money (unlike SMS). (As for video calling, it has pretty much all the disadvantages and restrictions of voice calls, plus some more. It's a cool tech demo, but not something I expect to use on a day-to-day basis.)
If you think about where and when and how you can use each mode of communication, IM matches or surpasses pretty much everything else on most axes. Sometimes you need the phone or email for a high-bandwidth conversation or to deliver a large payload. But those occasions are getting to be few and far between.
Part of it is just the medium— IM is as synchronous or as asynchronous as you want it to be— but a good part of the goodness here is thanks to Android's implementation. You can dictate messages with your voice; for short common messages (e.g. "call me when you get home") it works quite well, so you can dash off quick messages using whatever modality is more convenient, keyboard or voice. And the Android notification system notifies you of new messages and lets you bring up conversations easily— but discreetly, without interrupting whatever else you are doing on the phone.
For me, IM is the killer app for carrying a smartphone in my pocket, ranking significantly above "browsing the web" and a fair amount above "making phone calls".
(Incidentally, the words "phone" and "smartphone" seem so inadequate after you have come to fathom this super-communications capability you have on your hands.)
I'm pleased to announce the release of Zeya 0.5 (also— Hello, Reddit!).
Major changes since Zeya 0.4:
Known issues:
See http://web.psung.name/zeya/ for more information about Zeya, installation, getting started, and reporting bugs.
Ubuntu 10.10, the Maverick Meerkat, will be released in just a couple of weeks. That got me reflecting on the fact that I have been a happy user of Ubuntu for what must be over 5 years now. That's a long time!
The GNU/Linux variants are the only OSes I've used where I really have the flexibility to define my own workflow (example). So they are a pleasure to use (ok, most of the time). I use a computer for many, many hours a day nearly every day. And the time spent customizing software and learning it is a drop in the bucket when it's amortized over the months and years I'm going to spend using it. Sure, Windows and Mac OS are a bit more learnable and easier to get started with— but they are much less usable. And for me, and most other people who sit at a computer for a living, that is precisely the wrong optimization to make.
There's plenty more to love about Ubuntu: for starters, that it runs on every piece of hardware you throw at it; how with a modest amount of effort, you can make all the computers you use behave exactly the same; and how great apt is (really, it takes the fear and hassle out of installing software, and it's an experience that no proprietary desktop OS comes close to).
Ubuntu is far from perfect, but it is pretty marvelous, and all the GNU/Linux operating systems have come a long way in the last 5 years. When I step back, I'm a bit astonished that Ubuntu or anything like it even exists at all. It works, it's powerful, it's free of charge, and, with small carve-outs, all of it is free for anyone to do anything they wish with it.
One thing I rarely stop thinking about is how technology can be made to be an instrument of empowerment. And I believe that one necessary step in that direction is ensuring that you are the master of all these amazing devices you carry around with you all the time: that they serve you and carry out your will, and not the other way around. Ubuntu has this vast collection of software you can use as the substrate for doing anything, and the question isn't "Will the creators of this software give you permission to do this?" but rather "Who the hell is going to stop you?".
I find this an incredibly heartening idea, almost a cousin of the concept of Turing's universal machine— the possibility, realizable in software, that you are limited by nothing other than your imagination. Unfettered computation is really a magical thing. And Ubuntu is a wonderful demonstration of that assertion, though by no means the only one.
So, to everyone that helped to make this possible (Canonical; the Ubuntu community; Debian developers; kernel developers; upstream maintainers and contributors of all stripes; and yes, even the folks working on other downstreams, like RH/Fedora— your code makes its way into Ubuntu too):
You have truly helped to make something wonderful, and it's a real gift to humanity. Thank you.
A public service announcement for Java programmers.
Google has released as Free software a bunch of convenience libraries used internally in Google Java code. The project is (very cleverly) called Guava.
The libraries span a huge range of functionality, but I think that in general they help to promote a more functional programming style and they paper over some of Java's warts.
Here are three of my favorite things from Guava, but if you read the Javadocs you will undoubtedly find more cool stuff. Looking at all of this, you might decide that it's all simple stuff that you could implement yourself in about five minutes. Of course you could (in some cases, anyway), but why would you?
Immutable collections
Create an immutable list (as you might recall, every Java array is mutable, which can be a huge source of pain):
List<String> answers = ImmutableList.of("yes", "no", "maybe");
There's also a Builder pattern for more complex constructions, and analogous classes Immutable{Map,Multimap,Set,SortedMap,SortedSet}.
Collection factory methods
Here's some standard Java code to instantiate a collection:
List<Foo> foos = new ArrayList<Foo>()
Using the Lists class, you can rewrite that as the following:
List<Foo> foos = Lists.newArrayList()
Usually factory methods are a way for you to give the callee the flexibility of returning different classes at runtime. That's not the case here— Lists.newArrayList will always return an ArrayList (unsurprisingly). But Java does type inference for a call of this type, so you save the hassle of having to repeat yourself by writing the generic type Foo on both sides of the assignment, every time you create a new collection.
There are analogous classes Maps and Sets, which also contain other useful utilities in addition to these factory methods.
Splitter
Some simple illustrative examples for splitting a string using Guava's Splitter class:
Splitter.on(",").split("foo,bar,baz");
// ==> an iterable containing "foo", "bar", "baz"
Splitter.on(",")
.trimResults()
.omitEmptyStrings()
.split("foo,bar , baz,,,");
// ==> an iterable containing "foo", "bar", "baz"
But wait, doesn't Java already do string splitting? Yes! Yes, it does. And this is how (emphasis mine):
String.split
public String[] split(String regex, int limit)
Splits this string around matches of the given regular expression.
The array returned by this method contains each substring of this string that is terminated by another substring that matches the given expression or is terminated by the end of the string. The substrings in the array are in the order in which they occur in this string. If the expression does not match any part of the input then the resulting array has just one element, namely this string.
The limit parameter controls the number of times the pattern is applied and therefore affects the length of the resulting array. If the limit n is greater than zero then the pattern will be applied at most n - 1 times, the array's length will be no greater than n, and the array's last entry will contain all input beyond the last matched delimiter. If n is non-positive then the pattern will be applied as many times as possible and the array can have any length. If n is zero then the pattern will be applied as many times as possible, the array can have any length, and trailing empty strings will be discarded.
This is a bizarre and unmemorable edge case in the API. I guarantee that upon finishing this paragraph, you will promptly forget about it until about two hours into debugging a failure caused by it. You can save yourself some grief if you just use Splitter. Stay away from String.split.
Update: these steps seem to work on Ubuntu Natty/11.04 for me, as well.
My fellow citizens, our long national nightmare of having to use bad pointing devices is over.
Starting with Ubuntu Maverick/10.10, configuring scrolling with the middle button and the TrackPoint is now really easy:
(gpointing-device-settings is not new but Maverick is the first time it has really worked well for me.)
Update: some people are reporting that g-d-s settings are lost after suspend. If you run into this, you might try these alternative instructions.
Public service announcement: some 8% of men have some form of colorblindness.
A couple of people have pointed me to this nice article on colorblind-friendly design, by Masataka Okabe and Kei Ito. Useful reading for anyone who designs for print, presentations, or UIs.
It's a good overview of the underlying theory/physiology of colorblindness and gives many tips for conveying information through channels other than color.
The authors also provide this handy colorblind-friendly color palette:
I always thought nearlyfreespeech.net's advice on backups was good advice:
You should adopt a backup policy that assumes we are storing crates of sweaty dynamite on top of the servers that hold your important data. (Even though we aren't.) [NFSN FAQs]
If you only have one copy of something, stop what you are doing, obtain a disk, and replicate it.
Here are some brief notes on my backup setup, including some things I've learned since I last wrote about backups. (I had a disk failure last December and restored everything from backup. No tears, no sweat. In fact the exact same thing seems to happen about once every year, which I suppose is a good testimonial. I'm probably due for a disk failure real soon now.)
(I am not on Lenovo's payroll, just a happy user.)
I just got a new Thinkpad X201, and will comment on it more in due time. But while I was waiting for it to arrive, something happened that reminded me why I don't have to think twice about buying or recommending Thinkpads.
A friend of mine had spilled a glass of water on her Thinkpad. Well, Thinkpads have drainage holes under the keyboard so the bulk of the water is routed harmlessly out of the bottom. So this was a minor incident. The laptop wouldn't boot for about a day. But once it had dried off, it was back in business. No prepaid shipping box, no trips to the repair store. No discernible lasting damage. None.
Water damage must be one of the most common causes of laptop death. And yet, most computers (Thinkpads excepted) ship with a keyboard that goes zap when you spill something on it. It is astonishing that the spill-through keyboard (or equivalent) is not a standard feature on laptops. It does significantly increase the expected lifetime of your computer.
And it's not just water damage, either. My own X61s has also been through quite a few nasty drops and falls. I fell on it once when I slipped on ice. No ill effects. For the laptop, anyway. And the thing is everyone just expects that level of durability from Thinkpads. There is some serious quality engineering going on at Lenovo.
For crying out loud, the Thinkpads pass the freaking military spec tests for ruggedness. And just for fun, here's a video of a guy driving a motorbike over a Thinkpad.
These things are unbelievable. They are really built to last.
Google Web Toolkit 2.0 was released some months back. As I've said before, I think GWT or something like it is really the right way to write web apps. The optimized/compressed/inlined Javascript you want to be running in any nontrivial web app is so far from the readable and modular code that you want to actually be writing that it really demands an intermediate translation layer. This most recent release really just cements that fact.
For example, new in GWT 2.0 is code splitting, a feature that helps to reduce page load times. You define some split points in your app— points during execution where the client can make a round-trip to the server to fetch more code. GWT breaks off parts of your application that aren't needed when the app initially loads, and only downloads them when the app needs to cross the split point. Sure, you could compute, by hand, the transitive closure of all the components of your application that aren't past any split point, and shuffle your code around between files, and dread having to redo all that every time you do some refactoring. But you have better things to do.
It's true that Javascript engines are getting faster every week, so the value of highly optimizing your Javascript ahead of time may seem rather uncertain. But GWT also does things like automatically creating image sprites on your behalf, so your app can pull down all its images without the overhead of multiple HTTP requests. No Javascript engine can make excessive image requests fast. And in version 2.0, GWT does something similar for stylesheets. Again, all stuff you could do by hand— if you were highly disciplined, or a robot.
Further reading: More new features in GWT 2.0 and stuff slated for 2.1.
Many random notes (some for my own reference), each of which is too short to warrant a full blog post:
PSA: remote tab completionIf you have set up passwordless SSH login to some host, then bash-completion in Debian/Ubuntu (which is enabled by default, I think) automatically completes remote paths as arguments to scp. For example, pressing TAB after the following does exactly what it ought to, namely, display files within /path/to/whatever on the host example.com:
scp example.com:/path/to/whatever/Emacs: dired-jump
Ever find yourself wanting to rename or move a file while you're editing it? M-x dired-jump brings up a dired view of the directory containing the current file (you may need to (require 'dired-x) first). Press R ("Rename") and type the new path.
The best part is that the original editing buffer gets its path updated automatically (and its buffer name too, if you changed the basename), so you can get straight back to work— no need to close and reopen the file.
Suspending from the command lineOn Ubuntu 10.04 Lucid, you can suspend the machine programmatically by installing acpitool and then running the following:
To avoid having to enter my password every time for acpitool, I whitelisted that command in sudo for use without a password. You can do this by running sudo visudo and adding the following line at the end of the file:
%admin ALL = NOPASSWD: /usr/bin/acpitool -s
Sources: Linux Living. Also see: instructions for previous Ubuntu versions; pm-suspend(8).
VirtualBoxI've been using VirtualBox (aptitude install virtualbox-ose) to run virtual machines of various GNU/Linux flavors (for me, it's primarily useful for making sure Zeya works on different platforms and configurations), and I am generally pleased with it. A couple of tips for setting up VMs for maximum effectiveness:
(For the uninitiated, wdired is an indispensable emacs mode that lets you rename files in a directory just by editing their filenames in a buffer.)
Joakim Verona wrote a nifty patch against wdired that allows you to rename even to paths in directories that don't exist yet (emacs creates them for you when you finalize the operation). As an example of where this might be useful, suppose you have a directory like this:
/home/phil/tempfiles: total used in directory 60K available 413675080 -rw-r--r-- 1 phil phil 318K 2009-12-31 19:00 20091231.jpg -rw-r--r-- 1 phil phil 320K 2010-01-01 19:00 20100101.jpg -rw-r--r-- 1 phil phil 305K 2010-01-02 19:00 20100102.jpg [...]
Say you change that to the following (for instance, using M-x string-rectangle, among other alternatives):
/home/phil/tempfiles: total used in directory 60K available 413675080 -rw-r--r-- 1 phil phil 318K 2009-12-31 19:00 2009/12/31.jpg -rw-r--r-- 1 phil phil 320K 2010-01-01 19:00 2010/01/01.jpg -rw-r--r-- 1 phil phil 305K 2010-01-02 19:00 2010/01/02.jpg [...]
When you save, your files are now neatly organized in a year/month/day directory structure. This is the closest thing to magic I've done on my computer this month.
I've made some minor updates to Joakim's patch to fix bit-rot and conditionalize this behavior on a variable.
wdired-rename-to-nonexistent-directories.patch (1.6kb)
As an aside, I think learning about wdired turned on a lightbulb in my head that changed my understanding of Emacs. Yes, emacs is a text editor. But its power comes, in part, from the scores of major modes that let you interact with many different kinds of resources (e.g. your filesystem, as with wdired) under the guise of editing text (which you can think of as supplying a common abstraction over all those resources). Once you learn how to use, say, regexp search and replace, you can take advantage of that power when editing text files, but also when renaming/reorganizing files or composing mail or whatnot. Emacs is a text editor, but text doesn't just mean files.
I figured this was worth sharing because I myself had written two (fairly lame) clones of this program before I discovered it.
Sometimes I find myself composing and running huge shell scripts, like the following:
$ cat process-files.sh sox input/foo.ogg output/foo.ogg channels 1 sox input/bar.ogg output/bar.ogg channels 1 sox input/baz.ogg output/baz.ogg channels 1 sox input/quux.ogg output/quux.ogg channels 1 # more of the same, for perhaps hundreds of lines...
(Aside: why not xargs? For complicated tasks, it can be error-prone or just plain insufficient. Moreover, there's a lot of value in being able to just look at the script and see exactly what is going to be executed on your behalf, especially for one-off tasks. If you know emacs macros, scripts like this are not onerous at all to generate anyway.)
If you have a sequence of tasks like this that can run independently (and they are CPU-bound), then it pays to distribute the tasks over all your CPU cores. Here's where GNU Parallel comes in handy. Just pipe into it the commands you want to execute:
$ parallel -j4 < process-files.sh
Now parallel runs up to 4 tasks concurrently, starting up a new one when each one finishes (just as if you had a queue and a pool of 4 workers). What an elegant interface.
GNU Parallel has a bunch of more advanced features that are worth checking out, for example, preserving the proper ordering of standard output across tasks (to maintain the illusion of sequential-ness), or showing an ETA.
GNU Parallel is not in the official Debian/Ubuntu repos (as far as I can tell) but it is a snap to build from source, and it's the sort of thing I'd want floating around in my ~/bin everywhere I work.
On Ubuntu/Debian, you can use ffmpeg (in the package of the same name) to extract the audio only from video files:
$ ffmpeg -i input.ogm -ab 128k -vn output.ogg
ffmpeg detects the input and output formats automatically, so you can also convert to an MP3 (for example) just by specifying an output filename of the appropriate extension.
When converting audio for listening while working out or while traveling, it's also useful to downmix stereo to mono so you can listen with just one earbud without missing anything:
$ ffmpeg -i input.ogm -ab 128k -ac 1 -vn output.ogg
(Thanks to Rene Moser for the pointer. Also check out sox if you are looking to apply more sophisticated audio transformations.)
Further reading: ffmpeg command-line tool documentation
A few months ago I glanced at my Google Reader Trends page:
From your 188 subscriptions, over the last 30 days you read 10,075 items...
Seeing a five-digit number there frightened me a bit, especially because I am not Robert Scoble1 and staying on top of tech news is not my day job. Since then I've made three changes:
Just the first item, dealing with the news in batch mode, is sufficient to free up a big chunk of time (and mitigate the sensation of drowning). I still get news from other sources on a more-than-once-weekly basis— mostly, Google Buzz and email lists— and I'd like to improve the way I deal with those, too, but at this time they have far better S/N ratio and lower volume.
In my ideal world, for each feed I could turn a dial to say "show me the no-more-than-N most popular items per day for this feed". But sort by magic comes pretty close to what I want.
1 Scoble famously follows over 18,000(!) people on Twitter.
Amit Saha wrote a nice article about Zeya in Linux Journal. It's pretty much Zeya's missing tutorial and design doc (not to mention, a review and a rundown of the alternatives). Thanks, Amit!
I use rdiff-backup for backups. It creates a mirror of your data, just like plain old rsync, but also stores a series of reverse diffs that lets you restore historical versions of each file. For the most part it works well, but the interface for browsing and restoring from backups is somewhat clunky:
$ # List all snapshots
$ rdiff-backup -l /backup
Found 3 increments:
increments.2010-06-01T23:00:00-07:00.dir Tue Jun 1 23:00:00 2010
increments.2010-06-02T23:00:00-07:00.dir Wed Jun 2 23:00:00 2010
increments.2010-06-03T23:00:00-07:00.dir Thu Jun 3 23:00:00 2010
Current mirror: Fri Jun 4 23:00:00 2010
$ # List contents of a snapshot
$ # (note, lists specified dir recursively)
$ rdiff-backup --list-at-time 2010-06-01T23:00:00 /backup/foo/bar
foo/bar/one
foo/bar/two
foo/bar/more/three
foo/bar/more/four
$ # Restore a file from backup
$ rdiff-backup -r 2010-06-01T23:00:00 /backup/foo/bar/one ~/one.restoreI wrote rdiff-snapshot-fs, which is an alternative to rdiff-backup's restore interface. It's a virtual filesystem (powered by FUSE) that lets you browse all your snapshots and their contents. Now you can examine each snapshot as if it were backed by a real mirror:
$ ./rdiff-snapshot-fs.py /backup /view $ ls /view 2010-06-01T23:00:00-07:00 2010-06-02T23:00:00-07:00 2010-06-03T23:00:00-07:00 $ ls /view/2010-06-01T23:00:00-07:00/foo/bar one two more $ cp /view/2010-06-01T23:00:00-07:00/foo/bar/one ~/one.restore
Some background: another tool, called archfs, provides similar functionality, but it has performance issues, and I never got it to successfully load my backup repository (granted, my homedir backup repo has 400 snapshots and a current mirror size of 450GB). I concur with Jon Dowland's assessment that the problem stems from archfs trying to reconstruct historical snapshot data up front before it's needed. (Many problems in software, as well as in life, are really problems caused by being insufficiently lazy!)
More info about rdiff-snapshot-fs, including how to obtain the source, can be found at http://web.psung.name/rdiff-snapshot-fs.
(By the way, FUSE is really interesting to play with and FUSE-Python makes it a snap to get started. More remarks about this soon, possibly.)
Disclaimer: This is something I cooked up in my spare time. rdiff-snapshot-fs is liable to eat your data and scare your pets. The file sizes and modes that it reports are known to be unreliable under certain circumstances (though the file data itself is generated correctly, as far as I can tell). But, if you know no fear, you may find it somewhat more useful than the standard rdiff-backup interface.
Update: for (easier) instructions for Ubuntu Maverick/10.10 and later, click here.
Another Ubuntu release, another set of X.org shakeups.
Some things in X changed in Lucid (xorg 1:7.5+5 and higher), breaking existing Thinkpad TrackPoint scrolling configurations that modify files in /etc/hal/fdi/policy (like those you may have seen in this previous post). You can use gpointing-device-settings to apply the same policy, but I found that even that stops working after a suspend/resume cycle.
Samson Yeung pointed out to me the following fix which can be applied on Ubuntu Lucid/10.04:
Create a new file /usr/lib/X11/xorg.conf.d/20-thinkpad.conf with the following contents (on Ubuntu Maverick/10.10 and later, save the file to /usr/share/X11/xorg.conf.d/20-thinkpad.conf instead):
Section "InputClass"
Identifier "Trackpoint Wheel Emulation"
MatchProduct "TrackPoint"
MatchDevicePath "/dev/input/event*"
Driver "evdev"
Option "EmulateWheel" "true"
Option "EmulateWheelButton" "2"
Option "Emulate3Buttons" "false"
Option "XAxisMapping" "6 7"
Option "YAxisMapping" "4 5"
EndSectionThen restart X.
The configuration above works for both Thinkpad laptops with TrackPoints and the Thinkpad TrackPoint keyboard.
Source: instructions derived from these instructions by Samson Yeung. Thanks, Samson!
When Git (or another VCS) attempts to apply a patch that doesn't apply cleanly, it will defer to you, the user, to figure out the right thing to do. Usually you get, dumped in your lap, a file that has all these funny conflict markers in it, showing alternative versions of the content:
<action name="Execute"><execute>google-chrome</execute></action>
</keybind>
<keybind key="W-s-b">
<<<<<<< HEAD
<action name="Execute"><execute>firefox-3.6</execute></action>
=======
<action name="Execute"><execute>firefox-3.7</execute></action>
>>>>>>> f50523b...
</keybind>
<keybind key="W-a">
<action name="Execute"><execute>rhythmbox</execute></action>You can edit that file by hand to get the state you want, but Emacs has a specialized mode for resolving conflicts. Type M‑x vc‑resolve‑conflicts and you get a three-paned frame like the following:
Keyboard macros are the Emacs feature that I wish every program had. Especially web browsers: having to carry out repetitive tasks in web apps always feels to me like a throwback to a less civilized era.
(More information about keyboard macros, for the uninitiated: see #2.)
Here are some useful macro-related commands that I recently learned to use:
Extending a macro. If you record a macro, then realize you forgot to add an important step at the end, you don't have to re-record it. C-u F3 replays the last macro and then lets you tack on more commands to the end of it.
For example, C-u F3 RET F4 replays the last macro and then appends RET to the end of it. (You can also use C-u C-u F3 ... F4 to extend a macro without replaying it again first.)
Editing a macro. For more intensive fixes or changes to macros, you can edit your most recent macro with C-x C-k C-e (kmacro-edit-macro-repeat). You get a buffer like the following, and you can remove keystrokes or add new ones:
;; Keyboard Macro Editor. ;; Press C-c C-c to finish; press C-x k RET to cancel. ;; Original keys: ESC c ESC f ESC f ESC f C-d RET< Command: last-kbd-macro Key: none Macro: ESC c ;; capitalize-word ESC f ;; forward-word ESC f ;; forward-word ESC f ;; forward-word C-d ;; delete-char RET ;; newline
Creating a macro from your keyboard history. If you just performed some task that you'd like to repeat, but didn't have the foresight to start recording a macro before you started, all is not lost. C-x C-k l (kmacro-edit-lossage) brings up a buffer (like the edit buffer above) containing your most recent 300 keystrokes, which you can pare down to create a new macro.
I really liked the Gmail Labs "Muzzle" feature, which hides the status messages of contacts in the Gmail chat list. Unfortunately, Google retired this lab last week.
Happily, Yojimbow has replicated the functionality of Muzzle in a Greasemonkey script. (The script is only 8 lines long, with just 2 lines of Javascript! A very elegant solution.)
Firefox users can install Greasemonkey, and then install this user script; meanwhile, Google Chrome natively understands user scripts (making them look like extensions), and you can just click on links to them to install.
Well, better late than never, I suppose.
Things in software/computing that were new for me last year:
These were the most popular blog posts I wrote last year:
And these were the most visited blog posts that were written in previous years:
Alex Bennée has written an Edit with Emacs extension for Google Chrome. It's Chrome's answer to Firefox's "It's all text" extension, which makes composing emails, blog posts, and other long-form text in a browser a lot more tolerable. (Hooray!)
Since Chrome extensions can't spawn arbitrary processes, the Edit with Emacs extension requires the cooperation of an additional edit server that can. The edit server is implemented in elisp and is bundled with the extension.
Installation instructions:
Mozilla released Firefox 3.6 last week. There are many noticeable improvements compared to Firefox 3.5, but two that stand out to me.
First, performance has improved quite a bit. And not just Javascript performance. Startup time is improved; the awesomebar responds instantly to keystrokes; Javascript and redraw/redisplay are faster. I no longer perceive a huge speed difference between Firefox and Google Chrome on most web apps, and the UI has become more responsive. (A few things—startup time, redraw/redisplay/scrolling, and creating/closing new windows and tabs—are still minorly annoying when I use Firefox alongside Chrome—but 3.5 and 3.6 are like night and day.)
Second, Firefox 3.6's implementation of HTML5 media, or at least audio, is superb. Having spent lots of time hacking on and listening to Zeya on Chrome, Firefox 3.5, and Firefox 3.6, the latter is the one where things most often just seem to work as spec'd. Moreover it's Firefox 3.6 (and only Firefox 3.6) that can consistently maintain the illusion that I'm using a native media player rather than a web-based one. That's colossal.
So, kudos to the Firefox developers.
PSA: the ubuntu-mozilla-daily PPA has nightly builds of Firefox 3.5, 3.6, and 3.7 packaged for all Ubuntu releases since Hardy. It's a low-hassle way to teach an old dog new tricks.
I learn something new every day.
First, the sort of thing you could do to mess with someone when they step away from their Python interactive shell to go use the restroom:
>>> False = True >>> False True >>> if False: print "Woo!" ... Woo!
Sure enough, True and False are just names in the builtin namespace that can be rebound. This capability seems pretty dangerous, so I'd expect to see something like it in C or in Lisp, but its presence in Python surprised me a bit.
Second, the following puzzle stumped me for a while when I first saw it. Why does the following expression evaluate as it does?
>>> "string" in [] == False False
It's not a precedence issue, because that result is not consistent with either of the parenthesizations:
>>> ("string" in []) == False
True
>>> "string" in ([] == False)
Traceback (most recent call last):
File "", line 1, in
TypeError: argument of type 'bool' is not iterable
It's actually a consequence of Python's support for chained comparison operators. That is, just as
5 < a <= 7
desugars to
5 < a and a <= 7,
the first expression above desugars to "string" in [] and [] == False, which is False.
It's good that the comparison operators (< <= > >= == != in is) work in this consistent way, but it can trip up Python novices who have not yet learned that they really have to write the more Pythonic expression "string" not in [] if they want to stay out of trouble.
I'm pleased to announce the 0.4 release of Zeya. Zeya is a web server that lets you listen to your music collection, from anywhere, using nothing more than Firefox or Google Chrome.
Major changes since Zeya 0.3:
Zeya has also now been packaged for Ubuntu! (More precisely, the Debian package has been synced to Ubuntu Lucid.) On Debian testing or unstable and Ubuntu 10.04/Lucid, install the zeya package:
# apt-get install zeya
(It may be several more days before 0.4 appears in Debian testing or Ubuntu.)
See http://web.psung.name/zeya/ for more information about Zeya, installation, getting started, reporting bugs, and development.
I donated to the following charities and organizations in the past year (although I really meant to get this post out before the end of 2009):
UNICEF, to support relief operations following the earthquake in Haiti.
The Free Software Foundation, which has a simple but lofty goal: to ensure that the computers we use and carry around with us every day are actually accountable to us. Their public awareness efforts are invaluable, but if they did nothing but maintain the GNU operating system (including, ahem, Emacs), I would still be thankful beyond measure.
Wikipedia, for being one of the things that makes the web so wonderful.
Creative Commons, for showing that we can do better than "All rights reserved."
MIT Open CourseWare, for disseminating knowledge. OCW relies on donations to keep the lights on and to fund digitization of more classes.
My local branch of Feeding America (aka Second Harvest Food Bank). There is something depraved about hunger in the midst of plenty, and so close to home.
Amnesty International, that everyone may have the basic human dignities that you and I enjoy.