Upgrading to Windows 8

I have been using Windows XP for over 10 years. I have been happy enough with it that I never really felt a strong need to upgrade: it runs all the software I need, it's reliable, and it is still receives security updates from Microsoft.

Windows XP support is slated to end in a little over one year from now. Furthermore, Windows 8 recently launched, and, through the end of 2012, the upgrade price is attractive. I tried the Windows 8 Release Preview a few months ago, and found it to be compatible and fast. Time to upgrade:

A brief hardware detour: Windows XP has weak support for SSDs, in particular it does not support the TRIM command, so I've been holding back on getting an SSD drive. Windows 8 has much better SSD support, including TRIM support and faster drivers, so I also got a Samsung 830 SSD. A nice additional benefit of the new SSD drive is that it is much simpler and easier to install Windows 8 on an empty partition.

Purchasing and downloading Windows 8 is easy. The install, however, hit a snag: it is not possible to create a bootable DVD when running the Windows 8 installer in Windows XP. The only install option is to migrate in place, which I wanted to avoid. Fortunately, I was able to run the Windows 8 installer in the Windows 8 Release Preview I had previously installed on a separate partition, and I could create the install media that way:

The Windows 8 install is fast and smooth, it is much faster and easier than the Windows XP install. It also detected all the hardware correctly, and it did not need me to provide any external drivers. The operating system does a great job finding and installing all the latest drivers, including for my printer and other external devices.

It is a good idea to tweak Windows 8 for use on an SSD, there are many good guides available. You may also want to read this excellent Ars Technica article, which describes on the important ways SSDs are different from traditional hard drives.

I am a heavy user of Google products, so the first things I installed were the native Windows 8 Google Search and Chrome applications. They work well.

One of the more controversial changes in Windows 8 is the fact that the Start button is gone, replaced with the Start screen:

There are programs to restore the Start button in Windows 8, such as Start8. After using the Start screen for a day or two, however, I found that it grew on me:

1. In Windows XP, I kept the most commonly used programs in the Quick Launch bar or on the Desktop. Windows 8 has nice support for pinning apps to Quick Launch or Desktop.
2. For the less commonly used programs, I use Win+F, type the first few letters of the name of the program, and launch it. This is different from the way I used to navigate the Windows XP Start menu, but in many ways it's faster and more efficient.
3. What I still dislike about the Start Screen is that newly installed programs clump their icons into a single large group at the right of the Start Screen, and it's awkward to reorganize them into separate logical groups.

Windows 8 runs all the software I used to run in Windows XP without problems. Windows 8 also feels faster and smoother overall, although a lot of that is because I am running it on an SSD. I ran a few heavy graphics benchmarks, and there is no performance difference from Windows XP.

I am using very few Windows 8 native ("Metro") apps:

1. I use Chrome for email, calendar, contacts, and docs; although there is a native Windows 8 Chrome app, I spend a lot of time on the Desktop, where I use the non-native Chrome
2. I use Picasa for photos
3. I use iTunes for media
4. I use Steam for games; surprisingly, Windows 8 comes with an old Live games client, which needs to be upgraded manually

Some of the native Windows 8 apps, notably the Music and Photos apps, are pretty bare-bones in terms of features, so it's difficult to justify using them over the other more mature counterparts.

The Windows games -- Solitaire, Minesweeper, and Mahjong -- have been re-written as Windows 8 native apps and are beautiful:

Shredder Chess

I recently started to play chess again. I've always been a casual player, I enjoy the game, but never really read books or got serious about it.

Some of the most enjoyable games I've played have been against players of similar skill, who probably have a similar chess background. Unfortunately, I'm not always able to find people or time to play chess in person, so I sometimes play against a chess program. Some of the chess programs I've used are:

• Colossus Chess and Cyrus Chess, on the ZX Spectrum
• ChessMaster, Battle Chess, and GNU Chess, on the PC
• Chess Free, on the Android

My general impression of chess programs is that they're imbalanced: they either play very strong or very weak chess. At the strong end of the spectrum, many chess engines play quite well, and I feel that I lose the game without really understanding why I lost or what I could have done differently. At the weak end of the spectrum, the chess engine tends to make really obvious mistakes, like nonsensical trades, that appear almost random, and it's no fun to win a game like that. If the chess engine has a "level" setting, to make it more or less strong, the levels appear discontinuous, where changing the setting even by a little makes the engine strength change by a lot.

The most recent chess program I played, Chess Free, uses a "probabilistic search engine" called TreeBeard. The game can be tuned by level, from 1 to 10. When I tune it anywhere from levels 1-4, I can win handily, as the game makes really silly mistakes; level 5 however, is far too strong, I think I've only won one or two games against it.

I spent a bit of time searching for a chess engine that doesn't suffer from these problems, and I think I found it: it's called Shredder Chess. I've played a dozen games against it at various levels, and it's amazing: not only can it match my level extremely well, but it feels like I'm playing against a human! It doesn't make obviously bad mistakes, and its level of tactical play can be very closely matched to mine: whenever I lose, the engine dials down its strength by some amount, whenever I win, it goes in the opposite direction, eventually settling at a level close to mine. The various engine strength levels are subtly different, without discontinuous jumps one way or another.

It seems that modern chess engines, like Fritz, Rybka, or Shredder are now arguably able to handily beat even the strongest human players, especially if they can run on parallel hardware. So recent chess engine research appears to be increasingly focused on making chess playing enjoyable for human players, rather than focusing on pure strength

Shredder Chess is available on all major platforms, including Android. The main difference between these platforms is the GUI and its ability to show you intermediate moves, the current positional "score" of the board, what the computer is thinking, and so on. The Android version also has the ability to also show chess puzzles, which can be turned into actual games if you wish. The price for the classic engine is modest and, in my opinion, well-worth it.

If you enjoy chess, I highly recommend Shredder Chess!

Musical archaeology

I got my first PC (a 286) in the early 90s, and one of my favorite activities at the time was playing games. In particular, I loved playing the LucasArts and Sierra graphical adventures (like Secret of Monkey Island and the Space Quest series). At the time, PC hardware was expensive, and in particular a sound card was considered a luxury. Most of these games could play rudimentary music through the PC speaker, like this:


It turns out that these games were also able to play much, much better music if you had the right hardware in your PC. Here is the same game on a Roland MT-32:


Remember that this is the same exact game, that came on 3 floppy disks, running on the same 286! There was no CD with hundreds of megabytes of orchestral scores. And yet, the music is all there, amazingly rich, deep, and polyphonic. To understand how this all worked, a little bit of musical archaeology is necessary.

Throughout the 80s and early 90s, the next best thing to the PC speaker was an AdLib sound card:

(c) Wikipedia

This sound card was reasonably affordable, and generated music by a process called FM synthesis: mix certain frequencies, produced by independent generators, to produce music. If you mixed the frequencies in the right way, you could simulate various instruments. The music still sounded pretty synthetic, but it was a big step up from the PC speaker. One advantage of the AdLib card is that the music could encoded very efficiently -- the game would simply store the frequencies, durations, and volume for the notes that make up the music, and that's it. This is how a game like Secret of Monkey Island could have reasonably rich AdLib music and still fit on 3 floppies.

At around the same time, Creative Labs started to develop the Sound Blaster series:

(c) Wikipedia

These cards could also do FM synthesis (so they were "AdLib compatible"), but they also had a digital sound chip that could play WAV files, or recorded sounds. These sound a lot more realistic, after all they were recorded from real instruments! The sounds, however, took a lot more storage, since the compression was poor. Games that used WAV music sounded a lot better than AdLib, but also took a lot more storage.

Lost in all this is the incredibly rich and largely forgotten history of the Roland MIDI hardware, like the MT-32:

(c) Wikipedia

These modules used the MIDI protocol (Musical Instrument Digital Interface) to receive information about what notes to play, along with "hints" (like sustain, echo, and so on). Like in the case of the AdLib, the music was encoded very efficiently, so it could easily fit on a floppy. The reason this music sounded so incredible is that the Roland hardware had inside it hundreds of rich and detailed "samples" of the instruments it was asked to play. When the game instructed the Roland to play middle-C on a piano, the Roland could render this note by modulating its internal sample of a high-quality piano.

The downside of Roland's hardware is that it was expensive. While an AdLib or Sound Blaster card could be had for around $50, Roland's modules cost over $500 -- a small fortune at the time, so few could afford them. Ironically, the musicians at LucasArts and Sierra composed most of the game music on Roland hardware, and included it in the games, but then also shipped "downgraded" versions of it for the AdLib cards.

Creative Labs tried to compete with Roland by means of a technology called "wavetable synthesis". This was similar to the way the MIDI protocol works: the game would tell the card to play certain notes, and the card would have pre-recorded sounds for various instruments. An example of this is the Sound Blaster AWE32 card, which prominently featured the wavetable technology. The problem with this approach is that memory was still expensive, so in order to stay affordable, these cards shipped with limited storage for these sounds, and therefore had fewer instruments sampled at lower quality. Furthermore, the AWE32 protocol was not fully compatible with MIDI, so it was harder and somewhat more awkward to program.

The advent of cheap CDs, at around the same time, eventually rendered the whole issue moot: games could now afford to ship with an entire CD of pre-rendered, high-quality music, so there was no need for expensive MIDI hardware. Roland came out the loser in all of this: it stopped producing PC hardware entirely and today focuses exclusively on other segments of the market.

One nice side-effect of this struggle, that exists to this day is the advent of General MIDI or GM. This was a protocol that standardized the meaning of instruments so that when a program told the MIDI hardware to play instrument #4, everyone agreed that this would mean a saxophone, say. Most modern hardware today (including PCs and Macs) support General MIDI and have sets of instrument samples licensed from Roland for this exact purpose.

Another development related to this, is the advent of sound fonts. These are basically richer sets of instrument samples that can be used instead of the standard ones that come with General MIDI, and therefore can make the same MIDI hardware sound a lot better when playing the same tune.

Let's take a brief interlude and listen to the history of PC audio as told by Monkey Island:


Today you can still enjoy these adventure classics on emulators, such as the excellent ScummVM engine. In terms of sound emulation, here are your options in ScummVM:

• PC speaker
  
• AdLib (FM synthesis)
  
• SoundBlaster (in FM synthesis mode, or to play WAV samples, if any)
  
• MIDI
  

For the MIDI category, there are the following sub-options:

• Use the General MIDI hardware in your PC, with the standard instruments.
  
• Use MIDI with a sound-font, to significantly improve the quality of the music.
  
• Use a Roland (or equivalent) MIDI hardware attached to your PC.
  

For the purist, a Roland MT-32 can be obtained fairly easily on eBay for around $50 (you will need an external MIDI interface in your PC, however). If you enjoyed these games in the past, you owe it to yourself to try them again with a sound-font. I recommend Shan's excellent (and free!) sound font which is reasonably close to what you'd get out of a Roland. Here's how to use a sound-font in ScummVM:

• First, you need a reasonably fast PC (at least 1GhZ; ScummVM uses software emulation for sound-fonts.)
  
• Click on the "Options" button.
  
• Select the "Audio" tab, and set the "Music Driver" to "FluidSynth".
  
• Select the "MIDI" tab, and set the "SoundFont" path to your sound-font file.
  
• Launch the game, and be amazed.
  

In closing, here are two more beautiful pieces from Loom and Space Quest 1, as rendered on the Roland:



References:

• The History of PC Game Midi
• ScummVM Forums: Question about MT-32
• ScummVM Forums: How to tune your ScummVM sound

Photo library data mining

I've been very happy with my DSLR, and I really enjoy the photos it produces when compared to my previous point-and-shoots. I've been thinking of getting another fast prime lens for low-light indoor shots and for walking around. I have a 50mm prime, and although it takes wonderful portraits, it acts as a telephoto lens on my crop sensor, so I often back into walls or have trouble capturing the entire scene.

To decide which lens to get, I decided to look at what I could learn from the EXIF data from the approximately 9,000 photos in my library over the past 8 years or so. In particular, is there a particular zoom I tend to prefer? Of course, the zoom is intimately tied to the camera itself and its capabilities -- a camera that's limited to a particular zoom range means that no matter what I'd be taking photos within that zoom range (and correcting by walking back and forth to frame the shot -- which is naturally impossible to capture in the EXIF). Nonetheless, I thought this would be a fun and interesting exercise, so let's go mining.

To start, I have to extract the EXIF data from all my photos and set it in a format where I could explore it. I used the excellent exiftool command line utility which displays the full EXIF information. I am only interested in a few EXIF fields, specifically:

• Camera name/model
• Zoom range supported by the camera
• Zoom at which the photo was taken

I would like this data to be displayed all on one line, per photo, so I used the following BASH script:

#!/bin/bash

exiftool "$1" | awk -vfname="$1" -F " : " '

BEGIN {

  camera = "";

  zoom = "";

  lens = ""

}

{

  gsub(/ *$/, "", $1);

  gsub(/^ */, "", $2);

  if ($1 == "Camera Model Name") {

    camera = $2;

  }

  if ($1 == "Focal Length") { 

    zoom = $2;

  }

  if ($1 == "Lens") {

    lens = $2;

  }

}

END {

  printf("%s, %s, %s, %s\n", fname, camera, zoom, lens);

}'

The script basically pulls out the EXIF lines that start with "Camera Model Name", "Focal Length", and "Lens" and displays their corresponding values, along with the name of the file, on a single line.

I then ran the script over my photo library, which all resides under the same directory, with the following commands:

$ find . -iname "*.jpg" -print0 | xargs -0 -I % ./exinfo.sh "%" > data.txt

Note that I ran these commands on Windows (in Cygwin), so I used "-iname" to search over both *.jpg and *.JPG files.

The script took a few hours to run (exiftool, for all its awesome qualities, is not particularly fast), and in the end I got all the data I needed in data.txt

The first interesting part was the focal length of each photo, which is reported in the following format by exiftool:

4.6 mm (35 mm equivalent: 27.2 mm)

I actually want the 35 mm equivalent value (in this case, 27.2 mm, rounded to the nearest integer), which I extracted in Excel using the following formula (assuming the value above is in cell B2):

=INT(MID(B2, FIND(": ",B2) + 2,FIND(" mm)", B2) - FIND(": ", B2) - 2))

Lastly, I produced a histogram of the 35mm equivalent data, which looks like this:

The histogram seems to indicate that:

• 17% of my photos were taken at 27mm (in 35mm equivalent)
• 34% of my photos were taken in a range of (35 - 40) mm (in 35mm equivalent)
• This was the largest coherent cluster in the entire library

Because the D90 has a 1.5 crop sensor, the 27 mm equivalent comes from an 18mm focus; the (35 - 40) mm range comes from a (23 - 27) mm range. Intuitively, these numbers make sense to me -- I tend to strongly prefer wide-angles over zooms in my photos in general.

Nikon makes a number of non-fisheye prime lenses in this range, but many of them are either too wide for my taste (10 or 14mm), or very expensive, or have pretty poor quality. The lens which seems to have good quality, reasonable price, and fits nicely in the range above is the 24mm prime. I'll want to refine this further by weighing the zoom range by the capabilities of the camera, but as a first cut I'm pretty happy with the data and it agrees with my intuition.

Another nice side-effect of this analysis is that it reminded me of the all cameras I've used over the years:

• 10% of photos with Fuji FinePix 2600 Zoom 
• 35mm equivalent: 38-114
• 25% of photos with Canon PowerShot A70
• 35mm equivalent: 34-104
• 19% of photos with Canon PowerShot A75
• 35mm equivalent: 34-104
• 27% of photos with Canon PowerShot SD800 IS
• 35mm equivalent: 27-102
• 19% of photos with Nikon D90
• 18-55mm zoom
• 50mm prime

I really liked my PowerShot A70 (and only "upgraded" to A75 because my A70 broke and there was no way to replace it since Canon had discontinued the A70). The SD800 was a quantum leap in that it gave me a wider angle (27 vs. 34) and IS (so I could take photos without flash in low-light). Nowadays, I only use my Nikon.

My first month with a Mac

About a month ago I upgraded my company laptop from a Thinkpad to a MacBook Pro. Although I've used Apple computers in the past, it's never been my "main" computer or even close to it. I've always used Thinkpads as my work laptop and felt they were and still are exceptional laptops. I'm pretty proficient with the Mac now; I still miss the Thinkpad every so often, but overall I'm happy. Here are some of my impressions thus far from the transition.

My most important requirement for a computer is that it lets me be "productive". I define productivity as "doing my job as fast as I know how using the given tool". If the machine gets in the way (by being unreliable, slow, lacking software, etc.) then that's a deal breaker and I'm not interested. I don't have particular allegiances to certain companies or hardware manufacturers, so long as I can be productive. In that respect, the Mac got off to a surprisingly good start, and I can now safely say that I'm about 95% as productive on it as I used to be on my Thinkpad. I honestly did not expect this, so it was a rather pleasant surprise.

In terms of software, the applications I run most on my Mac are:

• Chrome = duh
• iTerm = SSH client
• TextMate = general purpose programming text editor
• OpenOffice = word processing, spreadsheet, presentations
• Eclipse/Android = work
• iTunes = streaming radio
• Picasa = photos
• VLC/QuickTime/Flip4Mac = media
  
• Skype = video conferencing
• Solitaire etc. = blow off steam

All these programs work almost flawlessly. I don't detect almost any difference from the Thinkpad, they run fast, they're reliable, they're as bug-free as they are on the PC.

The only application that took some work was iTerm -- I run Emacs in screen, and getting the keybindings to work was a bit painful. In particular, by force of habit I'd often hit Command+W to copy in Emacs, which would close the current tab. Fortunately, it's easy to re-bind the keys for any menu to some other key combination, so that's what I did. This is actually a remarkable feature of OSX, and one that I'm not sure exists on Windows -- you can specify any app and any menu entry in that app (by name), and then give it another key combination. Cool!

The main drag was getting used to the new key combinations. Command+{C, X, Z, TAB} all work the same and fairly intuitive for someone used to PCs. But that's where the similarities end:

• I used Home/PgUp/PgDn/End/Del a lot on the PC, and the Mac is inadequate here. Having to hit Fn+{Left, Right, Up, Down} just isn't the same. Not to mention that different apps interpret the key combinations differently, sometimes it recognizes Fn+, other times Command+.
• I still find it confusing to have to deal with Control, Alt, and Command. I know how they work, but I still have to think about it. Why do we need 3 keys to do what a PC does with just Control and Alt?
• Switching between applications is done via Command+TAB, switching between multiple windows of the same application is done via Command+~ Is this really necessary? Command+TAB should be enough IMO, why the extra aggravation?

Another minor drag is connecting an external monitor. I sometimes use an external digital monitor (DVI), other times an external analog monitor (VGA). Apple decided that they couldn't use a single connector for this, you need two dongles: one for DVI (DVI-D to be precise), and another for VGA. This is crazy. The mini-port can clearly put out both digital and analog signal, and DVI connectors are perfectly capable of relaying said signals. I suspect the reason Apple insisted on a DVI-D dongle is to speed the demise of VGA monitors. This is part of the general company culture (we know what's best for you, trust us), which I personally find patronizing.

Now that we got the bad out of the way, there's plenty of stuff to like:

• The touchpad is fabulous. Scrolling with 2 fingers, clearing the desktop with 4 fingers -- it's just beautiful. Furthermore, I simply could not use the touchpad on my Thinkpad, I always used the little red knob instead; the PC touchpad is so sensitive it's useless. The Mac touchpad simply works, and works well. If you've never tried to use one, I do recommend you try it.
• The laptop feels extremely sturdy and well built. I used to think that Thinkpads were the best built laptops out there, but the unibody design wins hands down. It's not even a comparison.
• The keyboard feels good, the screen looks beautiful. I'd say on par with the Thinkpad.
• The battery life is amazing. I get 4+ hours easily, which was never possible with any Thinkpad I've ever used. The magnetic power connector is really neat and has saved wire accidents a few times already.
• The wireless configuration just works. As soon as the laptop comes out of standby, it's connected to the wireless, which is fantastic (the Thinkpad used to take a good 15-20 seconds, which gets old fast). The laptop also knows to stay connected to the current SSID, unlike the Thinkpad which would switch between SSIDs seemingly randomly if more than one "preferred" network was in range (I guess they thought this was a feature?)
• Last but not least, going in and out of standby is remarkably fast and reliable -- it's almost instant ON/instant OFF. The Thinkpad would sometimes take 1 minute (yes, 60 seconds) to do this, and lock-up in the process. I've experienced this with every Thinkpad I've used.

All in all, I think the Mac has come a long way in the area I care the most about -- productivity. I would go so far as to say that if I had some Windows-only apps I needed to run, I'd run them in VMWare (which is very fast and nice to use on the Mac).


The Mac is a remarkable beautiful piece of engineering. The rumours are true, there's no laptop quite like it. It's also expensive, almost double the price of an equivalent Thinkpad. This is the reason I'd have trouble justifying the price if it came out of my own pocket. But I do love using it, and I can understand why some people never settle for less.

Shuffling tricks

I recently read this interesting article about an error in the EU Microsoft Browser Ballot. Briefly:

• In the EU, Microsoft must give users a choice of browsers in Windows.
• This is done via a message box with 5 top-choices (Firefox, Opera, Safari, MSIE, and Chrome), and a bunch of other intermediate-choices.
• The box should display the 5 top-choices in random order.

As it turns out, the order is not exactly random! For instance, MSIE is more likely than not to appear towards the right-end of the list, and Chrome towards the left-end.

The reason for this is programmer error, as described in the article. The programmer implemented the shuffle algorithm incorrectly by assuming that JavaScript uses QuickSort, when in reality it uses some other algorithm.

It turns out that shuffling is notoriously hard to get right.


Consider the following two simple shuffling algorithms:

1. For each entry Ex in the array: swap Ex with any other entry in the array.
2. For each entry Ex in the array: swap Ex only with entries after it.

Intuitively, it seems like either approach should produce a good shuffle. However, your intuition would be wrong: the first approach produces a non-uniform shuffle, while the second approach produces a uniform shuffle.

Better writers than I have explained how and why this happens. One good explanation of this is at Coding Horror: The Dangers of Naivete. The crux of the proof is:

• The total number of entries in a shuffle of n cards is n! (n factorial).
• The total number of arrangements of cards where you swap each card with any other card is n^n (n to the power n).
• n! does not divide evenly into n^n, so the shuffle cannot be uniform.

Prove to yourself that the 2nd algorithm above does not suffer from the problem described above. For extra credit, also prove that it is uniform.

In thinking about this problem, I was struck by one non-obvious fact: how can it be that, in the first (non-uniform) algorithm some configurations occur more than others? I mean, think about it:

• Start with an array in some order.
• For each element, randomly swap it with some other element.

Somehow, this makes it so that some configurations occur more often than others, and reliably so! Although the algorithm is random, somehow it favors certain configurations. This is quite surprising indeed. Really, spend a minute to let this sink in.

I haven't found a good "intuitive" explanation for why this favoritism happens. This explanation, from stackoverflow.com, comes close: "And that's the "intuitive" explanation: in your first algorithm, earlier items are much more likely to be swapped out of place than later items, so the permutations you get are skewed towards patterns in which the early items are not in their original places."

Magician-turned-statistics-professor Persi Diaconis has done a lot of very interesting work in randomness, nature, and our perception thereof. One interesting question he answered was: how many times do you have to shuffle a deck of cards so that it's "random"? Another is: how random is a coin flip, really? Both have surprisingly non-obvious and interesting answers.

Repeater on the cheap

The wireless signal in our house has difficulty reaching a few spots. On top of that, the PowerBook's metal shell interferes with the (already weak) signal, and makes it impossible to connect.

One solution is to use a repeater to boost the signal. LinkSys already makes a Range Expander, which seems to do the right thing, but at $80, it costs more than the router itself!

DD-WRT, which I already use for QoS, can also turn a $50 router into a repeater (and more!)

The generic name for this is Linking Routers. At a high-level, there are 4 ways to link two (or more) routers together:

• Client -- Router 1 is on the WAN, Router 2 extends the range of Router 1. The catch is that all clients can only connect to Router 2 via a wired connection. Additionally, Router 1 and Router 2 sit on different networks, so it is not possible to do things like "broadcast" between them.
• Client Bridged -- same as Client, but the two routers sit on the same network.
• Repeater -- same as Client, but the clients can connect to Router 2 via wired and wireless. The two routers sit on different networks.
• Repeater Bridged -- same as Repeater, but the two routers sit on the same network.

The easiest and most convenient configuration is the Repeater. Basically, Router 2 acts as just another computer that connects to Router 1, but creates a separate network (with a separate SSID) in its area of influence.

One drawback (to all these configurations) is that the wireless bandwidth is cut in half, roughly, because of collissions within the wireless broadcast between the two routers. This is apparently unavoidable. In our case, the 802.11g protocol is already fast enough for what we use it, that this does not create noticeable slowdowns (and SpeedTest confirms it).

The Client functionality has existed for a while (it even exists in my ancient v23 on my original router), but the Repeater functionality is new, only in v24, and (confusingly) only works right in some versions but not others. Read the documentation very carefully!

While I was researching repeaters, I also briefly looked into whether an 802.11n router might help -- they are supposed to be "faster" and have increased range. Unfortunately, the Linksys own N-products get mixed to bad reviews -- the range is not much increased, neither is the speed, and the installation process leaves much to be desired. Add to that the fact that, to use it, I'd have to get compatible 802.11n PCMCIA cards, and I'm not all that interested or excited about this.

DD-WRT continues to be an exceptional product, especially when combined with a solid piece of hardware like the WRT54GL.

IPv6: here already?

With the phenomenal recent growth of the internet, the 32-bit IP address space (IPv4) is starting to run out. There are efforts to address this, like subdividing or repossessing large blocks already in use, or the ubiquitous use of NATs, but they're only slowing the inevitable. The IPv6 standard has been around since circa 1996, and it has recently started to get more visible traction: all major operating systems (Windows XP/Vista, Linux, and OSX) support IPv6 natively, there exist IPv6 networks that you can access via dedicated tunnels ("tunnel brokers"), and there are rumors that some companies use IPv6 internally. Some ISPs (like Sonic.net) even provide such dedicated IPv6 tunnels for free, in an effort to encourage experimentation. Although no major ISP has yet migrated to IPv6, I believe it's only a matter of time.

Discussing the in-depth technical aspects of IPv6 is much beyond the scope of this blog post, though I do encourage the adventurous reader to take a look at some of the resources linked above. Instead, this is more of a high-level overview for how to get IPv6 up and running on your home LAN. I recommend using the excellent DD-WRT firmware's IPv6 support, upon which this post is roughly based.

IPv4 addresses are 32 bit in size, which gives us a total of 4 billion different addresses. The effective number of machines that can be accessed in practice is smaller (since many addresses were given out in "blocks" to institutions that don't use the entire block), but it still gives an idea about the upper-bound. In contrast, IPv6 addresses are 128 bit in size. This is an enormously large number, and it's hard to provide a good metaphor for how large it really is. IPv6 would allocate a trillion IPv6 addresses for each square centimeter of our planet, or over 1 trillion trillion addresses for each of the 6.5 billion people currently on the planet. Hopefully, this should be enough for the foreseeable future.

The internet at large is still IPv4, but you can configure your internal LAN (behind your NAT'ed router) to run over IPv6. All the machines on the LAN would talk amongst themselves using IPv6. However, the router is still connected to an IPv4 network (your ISP). Therefore, the LAN traffic must have a way to travel across IPv4 to its destination, regardless if the destination is an IPv4 or an IPv6 network. This is done in one of two major ways:

1. Use 6to4

This is a protocol which assigns each IPv4 address a special IPv6 equivalent (which, by convention, always starts with the prefix "2002:"). Your router would package all IPv6 packets, regardless of destination, into IPv4 equivalents, and send them to a relay gateway. This relay gateway is a machine that sits at the edge between an IPv4 and an IPv6 network, and looks at the destination address of incoming traffic:

• If the address is an IPv6 address on the relay's IPv6 network, it routes it using IPv6 routing.
• If the address is an IPv6 address on another IPv6 network, it packages it into an IPv4 packet, sends it over IPv4 to the relay gateway of the destination IPv6 network, which unpacks it, and routes it.
• If the address is a 6to4 IPv6 address (corresponding to an IPv4 address), it unpacks it, sends the traffic over the IPv4 network, collects the response, and sends it back to your router.
  

Confused? Here's a diagram:

One risk with the 6to4 protocol is that the nearest relay gateway may be far away, which negatively impacts performance. If you're careful, you can select a 6to4 relay that's reasonably close to you, and you should be fine.

2. Use a tunnel broker

A tunnel broker is a dedicated link from your router's gateway to an IPv6 gateway. This makes life a lot simpler for your router: all it has to do is take IPv6 packets from the LAN, package them as IPv4, and push them into the tunnel. Provided that the tunnel isn't too slow (that is, too far away), the tunnel 's end-point will do all the hard work of routing IPv6 and IPv4 packets correctly. Here is a diagram that describes the process visually (note that the Tunnel Server can either directly route over IPv6, or ask the Tunnel Broker to route over IPv4; your LAN uses only a single IPv6 tunnel to connect to the Tunnel Server).

Some tunnels require that your router have a static address, while others can handle dynamic addresses (via the AICCU utility). The two big IPv6 tunnel providers in the US are SixXS and Hurricane Electric.

If you succeed in setting up IPv6 on your LAN, you can connect to this site, and watch the turtle dance!

"All this work to see a silly turtle dance on the screen?" I hear you ask?

Although the IPv6 internet is significantly smaller than the IPv4 at this point, this might be a good exercise for you to learn how IPv6 works. All indications are that we will migrate to IPv6 before too long, so consider this a way to get your feet wet. There is no downside to running your LAN over IPv6, your operating system, as well as most modern browsers (Firefox, IE, etc.) and network applications (Thunderbird, etc.) natively support IPv6. The performance penalty for running using tunnels should not be prohibitive, and most DNS servers know to serve both IPv4 and IPv6 addresses for multi-homed servers, to make the accessible from both networks. IPv6 is, in some ways, much simpler and more elegant than IPv4, so for those curious to learn more, this is a great opportunity to look under the covers.

More Ubuntu

I played some more with Ubuntu, and I continue to be very impressed with it. It really does "just work", I find it intuitive, fast, and efficient. If I hadn't been so entrenched into running Windows, I would very likely switch over to Ubuntu. Out of the box, Ubuntu does the following with minimal or no configuration:

• Browse the web (Firefox), read and write e-mail (Thunderbird)
• Read and write office documents (OpenOffice) and PS/PDF (Evince)
• Instant Messaging (Gaim)
• Download and browse digital images (gThumb), edit images (Gimp), scan images (XSane)
  
• Burn CD/DVD (Gnomebaker)
• P2p (Bit Torrent)
• Detect and automount external hardware (printer, flash drive, digital camera, iPod)
  

Notice anything missing? Here's a hint: digital media. Ubuntu has its roots in Debian, so as a matter of philosophy it wants to support only free formats (such as Ogg). That being said, it is possible to install support for other formats (most notably MP3 and others), but the process is a mess.

First, there is a staggering number of digital media players out there for Linux: totem (gstreamer and xine flavors), xine, ogle, mplayer, rhythmbox, sound-juicer, kaffeine, amarok, and many more. Each one has its own idiosyncracies, and they only share codecs to a limited extent. The user interfaces (especially for DVD playback) are a huge hack and look like they were written by a 5 year old with a spoon. Some players (notably xine) natively support the important formats (WMV, MOV, etc.), while others use elaborate wrappers around Win32 DLLs (yes, you read that right) to decode the more stubborn formats. DRM formats (e.g. iTunes) are decidely out. DVD playback requires the controversial DeCSS library, and even then only works right about half the time (the gstreamer flavor is very slow, while the xine flavor works). Integration in Firefox only works with MPlayer, and even then crashes about a third of the time. In the end I got it to work acceptably well, but it was not easy, and does not do everything that Windows does.

The one lesson I took away from this is that the entire digital media software world is a gigantic mess. There are huge interests (and probably lots of $$$) in keeping these formats proprietary and opaque, and I wonder if Linux (or, more specifically, the free open source world) will always be one step behind in that regard. The fact that you have to run through so many hoops to play a DVD on a Linux installation (and even then with problems), is simply crazy in this day and age. I would have hoped that Apple would provide a Linux version of iTunes, given that it already runs on a flavor of Unix (OSX), but I wonder if that will ever actually happen.

It will be interesting to see what the future brings.

OpenSource rant

Someone gave a technical talk today about the code quality in OpenOffice. He's an engineer working on improving various aspects of OpenOffice, and the talk was somewhat of a retrospective and personal opinions on the codebase.

Before I begin, I should mention that I like OpenOffice. I find it useful, stable, and generally a worthy competitor to Microsoft Office. In particular, its importers do a very good job of translating from Microsoft's formats to open formats.

I have heard people complain about the code quality in OpenOffice, in particular the fact that it's severely threatened by bit rot. This is in part because the codebase is enormous (over 5 million lines), and in part because there is no overseeing authority.

The first vignette was about a feature (or lack thereof) in Impress (the equivalent of PowerPoint). If you don't move the mouse for about 3 seconds during a presentation, the cursor should disappear (it's distracting). Apparently, this feature was implemented in Impress in 2002 by 3 different people, but it was never approved into the codebase because the Sun usability folks (which still control OpenOffice) could not agree on what is a reasonable time interval (should it be 1.5 seconds or 2 seconds or ... ?)

The second vignette was about the first bug this engineer fixed in the codebase, namely:

#define protected public // eine kleine Schweinerei

If you know anything about Object Oriented Programming, this is about as gross and blatant violation of OOP principles as you can imagine. Apparently, someone was in a rush, and instead of refactoring a protected method into public, he decided to make all protected methods public. There is also a "rule" on the team that you never modify code that has German comments.

I always felt that Open Source, as a philosophy, works remarkably well despite the fact that it seems it shouldn't work at all. However, I also feel that the Closed Source philosophy is just as viable, since financial pressures are very effective and in fact drive just as much innovation (consider the fact that Microsoft, although much maligned, does produce some of most usable interfaces around, and it's no accident that lots of open source projects copy them).

In my personal experience working on Microsoft Office, the attention to detail and code quality, was extremely impressive. Never once would anyone have dreamt of implementing the kind of hacks I heard of in this talk about OpenOffice. I do believe that open source projects suffer from code quality issues and bit rot, and I have heard lots of similar stories about MySQL and other high-profile successful projects.

I suppose one lesson is that Open Source zealotry (as well as zealotry in general) is really out of place and should be regarded with due skepticism.

Ubuntu

I used to run Linux in college, and I liked it a lot. It did take time to tweak and setup and fight with stubborn hardware and all that, but it was also fun. And, more importantly, I learned a lot from it, which has consistently come in handy recently.

Since then, I've been running Windows. It takes less time to setup and there's more software available for it. I feel that my time is better spent and I am more productive and faster on Windows. Occasionally, I try out a version of Linux on my spare partition just to keep up, but I never end up using it for any significant period of time.

I recently tried out Ubuntu, and I have to say I'm very impressed with it. It feels very polished, in ways that no other Linux felt before. You boot off the CD, provide a few simple configuration options, and after about 30 minutes, you have a fully functional, polished, and usable Linux running on your machine. Everything from the careful selection of default packages, to the icons, and the default settings seems to have been thought out with usability and people in mind. All my hardware was supported right off the bat (printer, scanner, iPod, network, etc.) I feel that they really do live up to their creed: "Linux for Human Beings".

Interestingly, the most annoying part in getting Ubuntu to do what I want, was installing the various software and codecs for non-free media (DVD, MP3, etc.) Ubuntu has its roots in Debian, which is all about free software, so it does not come with any packages that are not free (in license or in spirit). You can install these codecs relatively easily, but it takes some time to get them to work just right (DVD playback in particular). If nothing else, it made me appreciate the whole controversy (and silliness, really) over DeCSS in a different light.

At the end of the day, however, Ubuntu feels right in ways that no Linux before has felt. I will probably keep this one around for a while, and I will probably post on it again.