I went for it and bought an image stabilized, L-series telephoto zoom: Canon’s 70-200mm f/4. I briefly considered the f/2.8, but I decided it was just too massive to carry around (it weighs about 1.5 kilos). I also considered the non-IS version, but decided that it made more sense to buy the best possible lens once than buy a lesser one and regret it a few years later.
On the basis of my brief experience, the image stabilizing ability of the lens is not exaggerated. Being able to hand hold a shot at 320mm with a 1/45th of a second shutter speed is quite impressive.
Next, I will probably aim to buy a wide angle zoom. Then, at some distant future point, a body with a full frame sensor, like the 5D.