Shredder Chess

I recently started to play chess again. I've always been a casual player, I enjoy the game, but never really read books or got serious about it.

Some of the most enjoyable games I've played have been against players of similar skill, who probably have a similar chess background. Unfortunately, I'm not always able to find people or time to play chess in person, so I sometimes play against a chess program. Some of the chess programs I've used are:

• Colossus Chess and Cyrus Chess, on the ZX Spectrum
• ChessMaster, Battle Chess, and GNU Chess, on the PC
• Chess Free, on the Android

My general impression of chess programs is that they're imbalanced: they either play very strong or very weak chess. At the strong end of the spectrum, many chess engines play quite well, and I feel that I lose the game without really understanding why I lost or what I could have done differently. At the weak end of the spectrum, the chess engine tends to make really obvious mistakes, like nonsensical trades, that appear almost random, and it's no fun to win a game like that. If the chess engine has a "level" setting, to make it more or less strong, the levels appear discontinuous, where changing the setting even by a little makes the engine strength change by a lot.

The most recent chess program I played, Chess Free, uses a "probabilistic search engine" called TreeBeard. The game can be tuned by level, from 1 to 10. When I tune it anywhere from levels 1-4, I can win handily, as the game makes really silly mistakes; level 5 however, is far too strong, I think I've only won one or two games against it.

I spent a bit of time searching for a chess engine that doesn't suffer from these problems, and I think I found it: it's called Shredder Chess. I've played a dozen games against it at various levels, and it's amazing: not only can it match my level extremely well, but it feels like I'm playing against a human! It doesn't make obviously bad mistakes, and its level of tactical play can be very closely matched to mine: whenever I lose, the engine dials down its strength by some amount, whenever I win, it goes in the opposite direction, eventually settling at a level close to mine. The various engine strength levels are subtly different, without discontinuous jumps one way or another.

It seems that modern chess engines, like Fritz, Rybka, or Shredder are now arguably able to handily beat even the strongest human players, especially if they can run on parallel hardware. So recent chess engine research appears to be increasingly focused on making chess playing enjoyable for human players, rather than focusing on pure strength

Shredder Chess is available on all major platforms, including Android. The main difference between these platforms is the GUI and its ability to show you intermediate moves, the current positional "score" of the board, what the computer is thinking, and so on. The Android version also has the ability to also show chess puzzles, which can be turned into actual games if you wish. The price for the classic engine is modest and, in my opinion, well-worth it.

If you enjoy chess, I highly recommend Shredder Chess!

Musical archaeology

I got my first PC (a 286) in the early 90s, and one of my favorite activities at the time was playing games. In particular, I loved playing the LucasArts and Sierra graphical adventures (like Secret of Monkey Island and the Space Quest series). At the time, PC hardware was expensive, and in particular a sound card was considered a luxury. Most of these games could play rudimentary music through the PC speaker, like this:


It turns out that these games were also able to play much, much better music if you had the right hardware in your PC. Here is the same game on a Roland MT-32:


Remember that this is the same exact game, that came on 3 floppy disks, running on the same 286! There was no CD with hundreds of megabytes of orchestral scores. And yet, the music is all there, amazingly rich, deep, and polyphonic. To understand how this all worked, a little bit of musical archaeology is necessary.

Throughout the 80s and early 90s, the next best thing to the PC speaker was an AdLib sound card:

(c) Wikipedia

This sound card was reasonably affordable, and generated music by a process called FM synthesis: mix certain frequencies, produced by independent generators, to produce music. If you mixed the frequencies in the right way, you could simulate various instruments. The music still sounded pretty synthetic, but it was a big step up from the PC speaker. One advantage of the AdLib card is that the music could encoded very efficiently -- the game would simply store the frequencies, durations, and volume for the notes that make up the music, and that's it. This is how a game like Secret of Monkey Island could have reasonably rich AdLib music and still fit on 3 floppies.

At around the same time, Creative Labs started to develop the Sound Blaster series:

(c) Wikipedia

These cards could also do FM synthesis (so they were "AdLib compatible"), but they also had a digital sound chip that could play WAV files, or recorded sounds. These sound a lot more realistic, after all they were recorded from real instruments! The sounds, however, took a lot more storage, since the compression was poor. Games that used WAV music sounded a lot better than AdLib, but also took a lot more storage.

Lost in all this is the incredibly rich and largely forgotten history of the Roland MIDI hardware, like the MT-32:

(c) Wikipedia

These modules used the MIDI protocol (Musical Instrument Digital Interface) to receive information about what notes to play, along with "hints" (like sustain, echo, and so on). Like in the case of the AdLib, the music was encoded very efficiently, so it could easily fit on a floppy. The reason this music sounded so incredible is that the Roland hardware had inside it hundreds of rich and detailed "samples" of the instruments it was asked to play. When the game instructed the Roland to play middle-C on a piano, the Roland could render this note by modulating its internal sample of a high-quality piano.

The downside of Roland's hardware is that it was expensive. While an AdLib or Sound Blaster card could be had for around $50, Roland's modules cost over $500 -- a small fortune at the time, so few could afford them. Ironically, the musicians at LucasArts and Sierra composed most of the game music on Roland hardware, and included it in the games, but then also shipped "downgraded" versions of it for the AdLib cards.

Creative Labs tried to compete with Roland by means of a technology called "wavetable synthesis". This was similar to the way the MIDI protocol works: the game would tell the card to play certain notes, and the card would have pre-recorded sounds for various instruments. An example of this is the Sound Blaster AWE32 card, which prominently featured the wavetable technology. The problem with this approach is that memory was still expensive, so in order to stay affordable, these cards shipped with limited storage for these sounds, and therefore had fewer instruments sampled at lower quality. Furthermore, the AWE32 protocol was not fully compatible with MIDI, so it was harder and somewhat more awkward to program.

The advent of cheap CDs, at around the same time, eventually rendered the whole issue moot: games could now afford to ship with an entire CD of pre-rendered, high-quality music, so there was no need for expensive MIDI hardware. Roland came out the loser in all of this: it stopped producing PC hardware entirely and today focuses exclusively on other segments of the market.

One nice side-effect of this struggle, that exists to this day is the advent of General MIDI or GM. This was a protocol that standardized the meaning of instruments so that when a program told the MIDI hardware to play instrument #4, everyone agreed that this would mean a saxophone, say. Most modern hardware today (including PCs and Macs) support General MIDI and have sets of instrument samples licensed from Roland for this exact purpose.

Another development related to this, is the advent of sound fonts. These are basically richer sets of instrument samples that can be used instead of the standard ones that come with General MIDI, and therefore can make the same MIDI hardware sound a lot better when playing the same tune.

Let's take a brief interlude and listen to the history of PC audio as told by Monkey Island:


Today you can still enjoy these adventure classics on emulators, such as the excellent ScummVM engine. In terms of sound emulation, here are your options in ScummVM:

• PC speaker
  
• AdLib (FM synthesis)
  
• SoundBlaster (in FM synthesis mode, or to play WAV samples, if any)
  
• MIDI
  

For the MIDI category, there are the following sub-options:

• Use the General MIDI hardware in your PC, with the standard instruments.
  
• Use MIDI with a sound-font, to significantly improve the quality of the music.
  
• Use a Roland (or equivalent) MIDI hardware attached to your PC.
  

For the purist, a Roland MT-32 can be obtained fairly easily on eBay for around $50 (you will need an external MIDI interface in your PC, however). If you enjoyed these games in the past, you owe it to yourself to try them again with a sound-font. I recommend Shan's excellent (and free!) sound font which is reasonably close to what you'd get out of a Roland. Here's how to use a sound-font in ScummVM:

• First, you need a reasonably fast PC (at least 1GhZ; ScummVM uses software emulation for sound-fonts.)
  
• Click on the "Options" button.
  
• Select the "Audio" tab, and set the "Music Driver" to "FluidSynth".
  
• Select the "MIDI" tab, and set the "SoundFont" path to your sound-font file.
  
• Launch the game, and be amazed.
  

In closing, here are two more beautiful pieces from Loom and Space Quest 1, as rendered on the Roland:



References:

• The History of PC Game Midi
• ScummVM Forums: Question about MT-32
• ScummVM Forums: How to tune your ScummVM sound

Photo library data mining

I've been very happy with my DSLR, and I really enjoy the photos it produces when compared to my previous point-and-shoots. I've been thinking of getting another fast prime lens for low-light indoor shots and for walking around. I have a 50mm prime, and although it takes wonderful portraits, it acts as a telephoto lens on my crop sensor, so I often back into walls or have trouble capturing the entire scene.

To decide which lens to get, I decided to look at what I could learn from the EXIF data from the approximately 9,000 photos in my library over the past 8 years or so. In particular, is there a particular zoom I tend to prefer? Of course, the zoom is intimately tied to the camera itself and its capabilities -- a camera that's limited to a particular zoom range means that no matter what I'd be taking photos within that zoom range (and correcting by walking back and forth to frame the shot -- which is naturally impossible to capture in the EXIF). Nonetheless, I thought this would be a fun and interesting exercise, so let's go mining.

To start, I have to extract the EXIF data from all my photos and set it in a format where I could explore it. I used the excellent exiftool command line utility which displays the full EXIF information. I am only interested in a few EXIF fields, specifically:

• Camera name/model
• Zoom range supported by the camera
• Zoom at which the photo was taken

I would like this data to be displayed all on one line, per photo, so I used the following BASH script:

#!/bin/bash

exiftool "$1" | awk -vfname="$1" -F " : " '

BEGIN {

  camera = "";

  zoom = "";

  lens = ""

}

{

  gsub(/ *$/, "", $1);

  gsub(/^ */, "", $2);

  if ($1 == "Camera Model Name") {

    camera = $2;

  }

  if ($1 == "Focal Length") { 

    zoom = $2;

  }

  if ($1 == "Lens") {

    lens = $2;

  }

}

END {

  printf("%s, %s, %s, %s\n", fname, camera, zoom, lens);

}'

The script basically pulls out the EXIF lines that start with "Camera Model Name", "Focal Length", and "Lens" and displays their corresponding values, along with the name of the file, on a single line.

I then ran the script over my photo library, which all resides under the same directory, with the following commands:

$ find . -iname "*.jpg" -print0 | xargs -0 -I % ./exinfo.sh "%" > data.txt

Note that I ran these commands on Windows (in Cygwin), so I used "-iname" to search over both *.jpg and *.JPG files.

The script took a few hours to run (exiftool, for all its awesome qualities, is not particularly fast), and in the end I got all the data I needed in data.txt

The first interesting part was the focal length of each photo, which is reported in the following format by exiftool:

4.6 mm (35 mm equivalent: 27.2 mm)

I actually want the 35 mm equivalent value (in this case, 27.2 mm, rounded to the nearest integer), which I extracted in Excel using the following formula (assuming the value above is in cell B2):

=INT(MID(B2, FIND(": ",B2) + 2,FIND(" mm)", B2) - FIND(": ", B2) - 2))

Lastly, I produced a histogram of the 35mm equivalent data, which looks like this:

The histogram seems to indicate that:

• 17% of my photos were taken at 27mm (in 35mm equivalent)
• 34% of my photos were taken in a range of (35 - 40) mm (in 35mm equivalent)
• This was the largest coherent cluster in the entire library

Because the D90 has a 1.5 crop sensor, the 27 mm equivalent comes from an 18mm focus; the (35 - 40) mm range comes from a (23 - 27) mm range. Intuitively, these numbers make sense to me -- I tend to strongly prefer wide-angles over zooms in my photos in general.

Nikon makes a number of non-fisheye prime lenses in this range, but many of them are either too wide for my taste (10 or 14mm), or very expensive, or have pretty poor quality. The lens which seems to have good quality, reasonable price, and fits nicely in the range above is the 24mm prime. I'll want to refine this further by weighing the zoom range by the capabilities of the camera, but as a first cut I'm pretty happy with the data and it agrees with my intuition.

Another nice side-effect of this analysis is that it reminded me of the all cameras I've used over the years:

• 10% of photos with Fuji FinePix 2600 Zoom 
• 35mm equivalent: 38-114
• 25% of photos with Canon PowerShot A70
• 35mm equivalent: 34-104
• 19% of photos with Canon PowerShot A75
• 35mm equivalent: 34-104
• 27% of photos with Canon PowerShot SD800 IS
• 35mm equivalent: 27-102
• 19% of photos with Nikon D90
• 18-55mm zoom
• 50mm prime

I really liked my PowerShot A70 (and only "upgraded" to A75 because my A70 broke and there was no way to replace it since Canon had discontinued the A70). The SD800 was a quantum leap in that it gave me a wider angle (27 vs. 34) and IS (so I could take photos without flash in low-light). Nowadays, I only use my Nikon.

Cellphone radiation

The recent iPhone 4 antenna problems have made a lot of news. As an Android user, I have no experience or vested interest in the matter, but I did read one excellent article about cell phone antennas in general and what causes the problems in the case of the iPhone 4:

"Just about every cell phone in current production has the antenna located at the bottom.  This insures that the radiating portion of the antenna is furthest from the head.  Apple was not the first to locate the antenna on the bottom, and certainly won't be the last.  The problem is that humans have their hands below their ears, so the most natural position for the hand is covering the antenna.  This can't be a good design decision, can it?  How can we be stuck with this conundrum?  It's the FCC's fault."

The article got me curious about how much radiation does my Android emit. To determine, I had to dig out the "FCC ID" for my Android -- this requires taking out the back panel and the battery, the FCC ID is printed underneath: NM8PB99100.

I went to the FCC website, and typed the ID into the form. Incidentally, why on earth does the FCC website require the user to split the ID in this way? Wouldn't it be easier to just have one text input box and have the code do the splitting for you? Bizarre.

The resulting page is a jumble of information, poorly organized and hard to dig through. When I click on "Details", I'm greeted with the following lovely warning:

To get the actual radiation report, I need to figure out which of the following documents actually contains this information:

The attachments are all listed as PDF files, but when I click on them, they download as HTML, so I can't open any of them.

Are you kidding me? I'm not exactly inclined to trust the FCC much based on what I've seen so far.

After some digging, it looks like the numbers I'm after are in "EN-2009-90001_OET65_Test report(Page.1-100)", on pages 4/5:

These are pretty standard numbers as far as smartphones go -- I think the FCC requires no more than 1.5W/kg SAR, and the Android is under that. The iPhone is in the same range. Interestingly, the radiation is lower next to the head but higher when the phone is body-worn.

The PDF itself is actually pretty interesting to read, in particular the part about how they do the testing. As the original AntennaSys article above points out, the FCC requires testing with the phone next to the head but does not specify how the phone is held next to the head. All kinds of interesting.

DIY Oscilloscope

For one of my projects I need to measure the frequency of an oscillator. I don't have an oscilloscope, and buying one is prohibitively expensive. I looked for alternatives, and I was intrigued to find a surprisingly full-featured and very inexpensive DIY oscilloscope kit made by JYETech; I got mine at NKC Electronics. The oscilloscope is built around an ATMega64 CPU running at 16Mhz. The oscilloscope can sample AC or DC signals up to 5M times/second, and has a variety of other nice features which cover a broad range of hobbyist uses.

This blog posts details my experience assembling and testing this oscilloscope kit, at revision 062.

This is an average complexity project, you'll need to be pretty comfortable with soldering in pretty tight spaces (around 1mm between leads). You'll need a good soldering iron, or even better a temperature controlled soldering station -- I use the XYTronic 379, which is excellent and sells for around $50 (click on any image below to see a larger version):

You'll also need the following tools:

• Rosin core solder
• Helping hands -- very helpful for holding the board while you solder some small components
• Flush cutter -- essential for trimming leads underneath the LCD to make it fit
• Long nose pliers
• Multimeter -- essential for testing intermediate voltages during assembly

The oscilloscope kit is tiny -- a little bigger than your palm:

In it you'll find the following (in clockwise order):

• Instruction sheet -- this is one piece of paper with URLs to more information online
• LCD screen
• Bottom- and Top- cover panels
• Mainboard
• Components

The mainboard comes with all surface mount components pre-soldered, which is really nice: some of these components are extremely tiny (a couple of millimeters), so soldering them would have been a nightmare.

Important note: the mainboard the revision number printed in the upper-left corner -- if you are following these instructions to assemble your own oscilloscope, make sure that you have the same exact version otherwise the steps may not match and you can damage the board!

The components can be grouped into categories as follows. First, we have the actual electronic components

• L7805CV voltage regulator (3-pins, with a screw attachment at the top)
• Heat-sink for L7805CV
• 1x 470uF capacitor
• 5x 100uF capacitors
• 1x 0.1uF capacitor (orange)
• 1x 100mH inductor (2 pins, round, no other markings on it)
• 1x 1N4001 diode (smallest component, white stripe at one side)

Then we have the switches and external connectors:

• 9x push-button switches with 9x white caps
• 3x slider switches (3 positions)
• 1x RCA input jack
• 1x power jack
• 1x RS-232 male connector
• 3x male connectors

The screws and pins that will hold everything together:

And, last but not least, the wires and connectors for the rudimentary probe you'll be using with your oscilloscope:

As I mentioned above, the documentation that comes with the oscilloscope is sparse, so you'll also want to open the following URLs in your browser, or print them out and have them nearby:

• Detailed parts list: this lists all the parts above along with the names referenced on the schematic.
• Assembly notes: this shows the final placement of the electronic components on the mainboard, referencing the parts by the names in the detailed parts list above.
• Voltage reference chart: this shows the expected voltages you should see on the board after you assemble it; very useful for testing your work before you finalize it, as well as visuall verifying the placement of parts.

There is also a helpful discussion board where you can search for answers or ask questions.

First, some definitions:

• The back of the board is the side that has all the chips and surface-mount components
• The front of the board is the side that will hold the switches and LCD later on

Start by installing components on the back of the board. The first component is the power adaptor. The leads are large, so you'll want to use a higher temperature on the iron and a fair bit of solder on the other side:

Take the voltage regulator and bend its legs using the long nose pliers at the joint:

Align the voltage regulator with the heat-sink, place the legs through the 3 holes at the top of the board, run the screw through the hole in the heat-sink (it is threaded, so you'll probably want to use a screw-driver), and put a washer on the other side. Solder it, then use the flush cutter to cut it as close as possible to the mainboard.

Take the diode and bend its legs using the long nose pliers as shown:

Place the diode on the mainboard in the D3 slot, with the white stripe towards the right (this is essential: the diode only works in one direction). Solder it:

Insert the 470uF capacitor as shown. This is the largest capacitor by size, so you can't miss it. The capacitor is electrolytic, so it's essential to get + and - correct: the capacitor has a grey stripe on it that indicates the - (negative) lead; the other lead is the positive lead. Insert the positive lead in the hole on the mainboard that has a square shape, and the negative lead in the hole that has a round shape. In this case, the grey stripe will be pointing down, towards the power plug:

Insert the other 5x 100uF capacitors as shown on the schematic. They are all the same size, so order doesn't matter, but placement, just like above, does matter. The capacitors are denoted on the schematic as C10, C14, C15, C16, and C32, along with the position of the + lead.

Important note: the C14 position is wrong on the schematic, you will want to put it in reverse, that is to say with the + lead towards the right, and the grey stripe (- lead) towards the left:

Bend the leads on the ceramic capacitor (orange) as shown:

Then place it on the right side of the board:

Install the probe connector opposite the power connector. Just like the power connector, it has large leads, so it will need more solder than usual:

Optional: install the RS-232 connector in the middle of the board. This connector can be used to hook-up the oscilloscope to a PC and take screenshots of its LCD. If you'll never need this feature, you can skip installing this, it will also make your life easier later when you install the LCD:

Short the leads across D4 (green wire, circled in the diagram below).

Important note: this step is missing from the official documentation:

It's now time to power-up the oscilloscope and do a preliminary test. You'll need a 9V DC power supply, where the jack is center-positive and outer-negative. Plug it in, and, using your multi-meter, measure the DC voltage at the "+5V" hole indicated with a red arrow in the image above. The multi-meter should read 5V DC. If it doesn't, something is wrong, the most likely causes are:

• You got the polarity wrong on one of the capacitors.
• You got the polarity wrong on the diode.
• One of your solder joints is loose.

Go back and carefully check your assembly, cross-reference with the schematic, and keep testing the voltage at the "+5V" hole above until it's right.

Once you get the voltage right, install another lead across the "JP1" connector, as shown below:

Check the "+5V" voltage hole again -- it should still read 5V DC. If it doesn't go back and check the capacitors and solder joints.

Now turn the board on its front and install the 9 push-button switches: their leads are springy, so they should snap into place, after which you can solder them. The orientation of the switches across their 4 holes doesn't matter:

Install the 3 slide-switches. Their leads are asymmetric, so they only fit in one direction:

You are now done with the mainboard for the time being. Pick up the LCD assembly and look on its back: you'll see a row of holes with labels:

Place the long strip of leads into these labeled holes, by inserting the shorter end into the holes (the longer end will go into the mainboard.) Solder them onto the LCD assembly, taking great care to keep the leads perpendicular to the LCD board, otherwise the LCD assembly won't fit right on the mainboard.

Important note: be sure to insert the strip of leads into the labeled holes, and not the other ones!

Important note: this is the hardest, most delicate part of the whole assembly. These holes are about 1mm apart, and the LCD board's holes are not pre-soldered. You need a steady hand and patience. The helping hands tool should come in handy here. A technique I found works reasonably well is to get one of the leads soldered correctly, holding the rest at the right position and angle; once you get that done, soldering the rest should be easier:

Solder the two other 2x lead strips at opposite ends of the row of holes opposite the one you just soldered. These leads are not electrical contacts, they just hold the LCD in place on the other side. Pay close attention to solder these in perpendicularly as well:

Using the flush cutter tool, cut one of the leads as shown in the image. This is because the hole into which it goes on the mainboard is obstructed by the heat-sink as shown:

Now solder the LCD assembly on the front side of the board. The LCD assembly can only go in one direction, don't force it.

Important note: you will need to use the flush cutter to cut all leads of components sticking out through the board behind the LCD very flush to the board, otherwise you risk shorting the LCD!

Screw together the two halves of the metal posts that will hold the panels above and below the mainboard. The shorter metal post goes on the front of the mainboard, the longer metal post on the back:

The final assembly should look like this:

Now comes the moment of truth: plug it in, and, if all is well, it will go through the boot sequence like this:

Place the white-caps on the push-buttons, and, using the 4 remaining screws, assemble the top and bottom panels like this. If you find that the push-buttons are sticky after installing the panels, either trim the white-caps with a knife, or loosen the screws a little:

You're almost done! The last step is to assemble the probe:

Strip the ends of the red and black wires -- these will connect to the alligator clips. Be careful to slide the shrink-wrap tube and alligator clip cover on each wire before finalizing the assembly:

Do the same with the larger black coaxial wire, and connect it to the RCA plug. Just like above be careful to slide the shrink wrap tube and RCA cover over the wire before finalizing the assembly. Connect the red wire to the inner coaxial lead (leading to the center jack), and the black wire to the outer coaxial lead (leading to the outer shield):

After everything is assembled, slide the shrink-wrap tubes over the connections and use a hair-dryer on high to shrink them into place.

Let's test that the oscilloscope can actual measure frequencies! Turn it on, and plug the probe into the RCA jack. Set the oscilloscope as follows:

• Using the SEC/DIV button, set it to "1ms"
• Using the LEVEL button, set it to "i"
• Using the V.POS button, and the +/- buttons, slide the Y level down towards the bottom of the screen

Now connect just the red alligator clip to the upper-left corner hole on the mainboard marked "500Hz/5V pp" (this hole is also marked on the top panel with the same indicator):

The oscilloscope should show you a 500Hz square wave:

You might consider getting a better probe, I like this one.

Enjoy!