Does the fog light/ dimmer switch have any lighting to it? I can't recall if my car ever had it working when it was newer, im just assuming it does but the bulb is broken.
I believe on the '95-'99 Js it did not light up (Sunfires didn't at least), but it did on the '00+ Js. Feel free to correct me if I'm wrong.
So, since you have an '01 Cavalier Z24, you're right the bulb is just burnt out. You will have to order the entire little part that houses the bulb because it's a teeny little bulb that you would never find at any auto parts store.
Fortunately it's only $11.87 (before shipping) on GM Parts Direct. Call them about the shipping price though because it is some times wrong on the site.
i have a dimmer light on my 95 and 97 sunfire, plus the cavalier switch still has the socket that accepts the light bulb but the panel is lacking the cutout for the light to shine thru the overlay.