Correcting for Atmospheric Refraction
TL;DR: Because Earth’s atmosphere acts like a giant lens, stars appear slightly higher in the sky than they actually are; you must subtract this "refraction" error from your sextant reading to find the star's true position.
The Atmosphere as a Trickster
You have already mastered the , which gives you the theoretical "where" of the stars. But when you look through your sextant, you are not looking at the star in a vacuum. You are looking through a thick, swirling blanket of gases. Think of the atmosphere like a pool of water. If you have ever stood at the edge of a swimming pool and looked at a coin resting on the bottom, you know it looks like it is floating higher than it really is. This happens because light rays bend—or —as they pass from the air into the water.
Our atmosphere does the exact same thing to starlight. As light from a distant star enters the top of our atmosphere, it moves from the thin, cold vacuum of space into the increasingly dense layers of air near the surface. This change in density acts like a prism, bending the light downward toward the horizon. Because your eye traces that light ray back in a straight line, your brain interprets the star as being slightly higher in the sky than it truly is. If you trust your raw measurement without adjustment, you will be off by a few precious arcminutes, which can translate to miles of error on the open ocean.
Quantifying the Bend
How much does the light actually bend? It depends entirely on the angle of the star. If you are looking at a star directly overhead—at the —the light is hitting the atmosphere straight on. In this position, there is very little bending. However, the closer a star gets to the horizon, the more "atmosphere" that light must travel through to reach your eye. The light has to slice through a thicker, denser wedge of air, which forces it to bend more aggressively.
To correct this, we use a standard mathematical correction factor. For most navigation, we rely on a simple formula where the refraction () in arcminutes is roughly related to the cotangent of the altitude ():
In practice, you do not need to do this complex trigonometry on the deck of a rolling ship. Navigators use pre-calculated tables that list the "dip" and "refraction" based on the height of your eye and the altitude of the star. You simply find your measured altitude on the chart, look up the corresponding refraction value, and subtract it from your reading.
The Precision of the Correction
It is tempting to think that such a small shift—often only a few arcminutes—doesn't matter. But remember our goal: pinpointing a location on a vast, featureless ocean. One arcminute of error in your calculation creates a one-nautical-mile error in your position. If you are trying to find a small island or avoid a reef, that one-mile gap is the difference between safety and disaster.
This is why we treat the atmosphere not as empty space, but as a physical tool that requires calibration. Once you have corrected for the atmospheric bend, you have moved from merely "guessing" where you are based on a visual estimate to "calculating" your position with scientific rigor. You are essentially stripping away the distortion of the planet itself to see the universe as it truly sits in the sky.
Atmospheric refraction forces light to bend as it enters the dense lower air, making stars appear higher than they are, requiring you to subtract a correction factor from your sextant reading to ensure your position is accurate.
Now that you have learned to account for the tricks the atmosphere plays on your eyes, you are ready to tackle the most important star in the navigator's toolkit: Polaris. In the next station, we will use your corrected altitude to lock in your latitude, providing the final piece of the puzzle to determine exactly where you sit on the globe.