You may have experienced it yourself and weren't sure why it happened. Or maybe you blamed it on something else. But rest assured, if you've used really small apertures in your shots and noticed them getting softer and softer, don't blame yourself. Blame physics.

Why does this happen? The core physical phenomenon is light diffraction. Basically, light is a wave. And similar to other waves (like waves on the surface of water), when it encounters a barrier with a small hole, the waves don't travel straight through, they expand and fan out on the other side. Here's a basic animation of the effect.

Credit Wikipedia and Lookang

If you want even more technical background on this effect, check out the article on Wikipedia. But really what matters is how does this actually change your photos?

Diffraction At Work

When there's a lot of diffraction (as is the case in tiny apertures), the light coming through the aperture spreads out (see the animation above). This results in a loss of sharpness. 

Here is a real world test I did to demonstrate the effect. These macro shots were all taken with a Canon 50D and a Sigma 105mm prime lens. The only thing that was varied was the shutter speed and aperture. The focus point was in the exact center of the circle and remained the same for each photo.

This first shot was taken at f/2.8. Very shallow DoF. Notice how the center of the dandelion is fairly clear but the structures radiating out from the center have many blurry elements. 

Taken at f/2.8

This next shot was taken at f/5. There's a definite difference. Not only are the radiating sections starting to become clearer due to the DoF, but the center itself (the focal point) is also becoming clearer. 

At f/5 we notice an immediate difference in clarity

The next shot is right in this lens's sweet spot, at f/10. Note how clear all the central elements are, and we've also increased the DoF a great deal.

Taken at f/10, the clarity has improved considerably

Now watch what happens if we keep pushing the aperture smaller and smaller (higher f-stop). This next shot is at f/18 and the shot is already starting to slightly soften.

Taken at f/18, there's starting to be a little softness

These last two shots, taken at f/29 and f/45 respectively, solidly demonstrate how the greater DoF of the small aperture is completely counter balanced by a loss of clarity.

At f/29 the shot is distinctly soft

At f/45 (a very small aperture) the shot is now totally blurry.

Here are the extremes shows side-by-side for comparison:

f/2.8

f/10

f/45

There's a tremendous difference between the extremes and the example clearly demonstrates there is a "sweet" spot. I've had this particular lens for years and even without a controlled test like this I knew from experience the sweet spot exists in the f/8-13 range. Different lenses perform differently so don't take this test as a hard and fast rule for every situation. This test is just a demonstration of the effect that diffraction has on the tiny apertures some lenses have.

So the next time you want to crank down that aperture to get a super depth of field or to cut down on available light, be aware you may be sacrificing clarity along the way. This is why it's good to be aware of the hyperfocal distance of a shot so you don't overdo the aperture unnecessarily.