This experiment is a touch tedious: the balance, if made well, is rather sensitive, and thus you have to be pretty careful while working with it. There were a couple of times when I accidentally whacked it, or dropped my ruler on it, and screwed up everything. I got really ticked off when this happened, and I imagine you will too ... so, my overall recommendation is be VERY gentle with the balance once you've got it made. The final measurements can turn out surprisingly well if everything is done carefully, which was pretty satisfying; one false move, though, and you can wipe out a lot of sensitive adjustments.
Here are some specific recommendations:
1. In Step 6 of the experiment, Zap tells you to strip insulation from the ends of the magnet wire. This is because Step 6 comes up right before you run current through the coils and start measuring the magnetic forces. However, Step 6 occurs well after you've built the sensitive balance. Stripping the wire at that point runs a severe risk of whacking the balance and screwing everything up: I recommend you strip the ends of the wire before doing anything else. According to Zap, you can burn the insulation off with the soldering iron. Zap lies - I couldn't burn that crap off if my life depended on it. (It is possible that this is because my iron doesn't get hot enough; the flat, broad irons might do a better job than mine did.) The insulation is easily scraped off with one of the razor blades, though. Test whether the insulation has been successfully removed with your ohmmeter: if you succeed in stripping the insulation off, the resistance of the 50 ft coil is really small; otherwise, it will be infinite.
2. Everything that needs to be done prior to winding the coils (building the brass rod balance and mounting it on the wood screws - ie, Step 1) is pretty easy. It's a good idea to make the balance so that it is as nearly balanced as possible before the coils are mounted. It doesn't have to be perfect, though: since, in the end, we want to measure the difference between the torque with no current and the torque with current, any mismatch due to an imperfect balance cancels out.
3. When you wind the coils, try to make them so that they have the same number of turns. This is easy to screw up; I estimated that I could get 70 turns out of the full coil. After making one coil with 35 turns, my second coil had only 28. D'oh. This has important ramifications: the magnetic force between the coils is proportional to the product of the turns in the two coils. You should be able to convince yourself that, for fixed total number of turns, this product is maximal when the turns in coil 1 equals the turns in coil 2.
4. The nuts you use as the counterbalance are the stainless steel ones; I used 3 big ones and 2 small ones in a big clump, and then slid the 3rd small one farther out to complete the balance. According to David Beckman (who measured a huge bunch of them), the masses are (3.16 +/- 0.05) gm for the big nuts, (1.34 +/- 0.03) gm for the small ones. Don't forget to measure their thicknesses as well: when they are on the balance arm, they act like a mass at their center point. In other words, a nut that is L cm away from the fulcrum has a lever arm of (L + t/2) cm (where t cm is the thickness of the nut).
5. When you hook up the LVPS, you get the most useful data when the current through the coils is around 1 amp; try a bunch of values in the range from 0.5 amp to 1.5 amp or so. (You actually get a rather strong force with these currents - I was surprised how much the balance shifted). However, when the LVPS delivers this much current, it is of course also generating a pretty large amount of power. The LM317 on your LVPS will get pretty hot at this power level, and will probably overheat. If this happens, the LM317 automatically cuts the current by a large amount. This will be very obvious, since the balance will shift and the lightbulb will turn off. If this happens to you, just disconnect one of your clip leads and let the power supply cool off for a few minutes. I found I could deliver 1 amp of current for only about 3 minutes before the overheating caused the current to cut off; I then had to wait 3 - 5 minutes before using it again. This made adjusting the nuts to put things in balance a bit of a pain. Once you've got the nuts in place, you don't need to keep running current to measure their positions - you can disconnect the power supply and take your measurements without any danger of the power supply overheating.
Feel free to email me if you have any questions.