The Symptoms & Solution:
The oil level sensors failed on both of my '96 Cavalier 2.4Ls. Even when my oil level was full, the yellow oil-level light (not the red oil-pressure light) would come on when I'd first start the car. This problem persisted for years before I bothered to replace the oil-level sensors. It may be worth noting that if I turned off the engine and restarted it shortly thereafter, the oil light would go off. Replacing the oil level sensor (~$29) solved the problem for me.

Legal Disclaimer: I wrote the following instructions for myself after changing my oil-level sensor. I am posting these instructions, just in case it helps others. Injury to your person or damage to your car could result from doing this job yourself. If you're not comfortable or experienced doing this type of work, please have a trained mechanic perform this job. I do not guarantee that these instructions are correct. If you choose to follow these instructions, you do so at your own risk.

Overview About Buying & Replacing the Oil Level Sensor:
Whether or not you decide to do this job yourself, it's convenient to replace the sensor at your next scheduled oil change because the oil needs to be drained in order to replace the sensor. For me, this was a VERY easy job that only added about 5 minutes to my oil-change time. It's one of those rare jobs that was easier than I expected. My Chilton's guide only has instructions for replacing the sensor in the 2.2L, not the 2.4L.

The oil level sensor (AC Delco D8057 or similar, ~$29 on Amazon) is located about 3 inches to the right (passenger side) of the oil drain plug. It inserts vertically into the oil pan--as does its (8mm) retaining bolt. I did not have to raise the car in order to do this job. I could access everything by laying on the floor under the front bumper, as I would for changing the oil.

I had hoped that the sensor could be cleaned and replaced, but there was really no grime on it and its (magnet-holding) bobber moved very freely, so I replaced it with a new sensor. (See important notes at bottom about how the sensor works and how it may have ejected its magnet inside your oil pan.)

Tools & parts required:

New oil level sensor.
13mm wrench or socket wrench.
8mm socket wrench.
Oil drain pan.
Oil filter (optional, if needed).
Oil filter wrench (optional, if changing filter).
4 quarts of oil.
Paper towels for cleanup.
Latex or vinyl gloves (optional, if you want to keep your hands clean).

Instructions for changing the oil-level sensor (and doing an oil change).

1) Park the car on a level surface. Put it in gear or in "Park". And apply the parking brake for safety.
2) Lube the new oil-level-sensor's O-ring with some clean motor oil.
3) Using a 13mm box wrench (or socket wrench) to remove the oil drain plug bolt and drain the oil into a container.
4) Change the oil filter, if needed.
5) Clean the area and replace the 13mm oil pan bolt (so that it stops dripping while you change the sensor).
6) Remove the electrical plug from the sensor. Gently pull down its tab with your thumb and then pull-off the plug towards the rear of the car. It's easy. No tool required.
7) Use an 8mm socket wrench to remove the 8mm retaining-bolt that holds the sensor in place. (For me this removed very easily.)
8) Gently pull the sensor downward to remove it, noting the orientation. (For me this removed very easily.) Per the note at the bottom, observe carefully as you remove the sensor to see if the sensor's magnet falls out onto the floor. After removing the sensor, I recommend checking to see if the sensor's bobber's magnet is still in the bobber, or on the floor). If not, you may want to see if you can locate and remove it from the oil pan using another strong magnet.
9) Clean the area with a paper towel.
10) Insert the new oil-level-sensor.
11) Reinstall the 8mm sensor-retaining bolt-- at first by hand and then with a socket wrench. (Don't over-tighten.)
12) Reattach the sensor's electrical plug. (It has a notch so that you can only connect it with the proper orientation.)
13) Fill the engine appropriately with motor oil. (That should be 4 quarts if it was drained properly.)
14) Check the oil level. If it's correct, then start the car.
15) If the old sensor was the problem, then the yellow oil-level light should not come on.

An important note about the sensor and its magnet which may have come loose inside your oil pan.
The oil level sensor has a cylindrical (foam-rubber) bobber/float on an axle, and the bobber has a small magnet inserted into it (parallel to the bobber's axle). The magnet is a small steel rod (about 3/8" long and 1/16" in diameter). You can just barely see the bottom of the steel magnet if you look at the bottom of the bobber. If the oil-level is low, then the bobber sinks down near the sensor's body and the magnet is detected by the sensor, thereby indicating that the oil level is low.

After I removed my sensor, I found the magnet laying on the floor. I had no idea what it was until I figured it out later. I also don't know if the magnet had become detached from the bobber before or after I removed the sensor. I suspect that it was beforehand, because I gently removed the sensor and laid it down, so it seems VERY unlikely that I jarred it loose. After I replaced my sensor, I cleaned off the old one to see if I could understand how it works and maybe fix the sensor for use on my other 1996 Cavalier. I then noticed a small hole in the bottom of the bobber that was a perfect fit for the magnet that I found on the floor. (When I found the magnet on the floor, I didn't know that it was a magnet or that it was even part of the sensor. I thought it might just have been some garbage that was already on my floor before I started the job. But when I saw that it was a perfect fit, I put the magnet against some steel to determine that it is indeed magnetic.) The only thing that holds the magnet in place is friction (which is a bad design). However, the magnet is just SLIGHTLY longer than the maximum gap between the bobber and the sensor's body, therefore the magnet SHOULDN'T be able to completely come out. (But obviously it did.)

Did the sensor fail because the magnet had come out? I don't know. Maybe. But if so, I don't understand why (on both of my cars) the "Check Oil" light turned off if I briefly turned off and restarted the car.

It's easy to reinsert the magnet into the filter (though I'm not sure if the polarity is correct or if it matters). But I figured that if it fell out once, it would likely fall out again. I suppose that I could have jammed a toothpick or something alongside the magnet to provide more friction, but I didn't want to take the chance that the toothpick or magnet would come loose and harm the engine. So I think it's best to buy a new sensor rather than trying to repair the old one.