To be honest, 4 of these words are interchangeable in most cases. A "peninsula" is a general term for land projections into water that pretty much every native speaker would know. "Foreland" and "headland" are not terms that are used where I grew up, but seem to be synonymous. According to Wikipedia, a "Cape" is a large projection that marks a significant change in the shape of the coast. That's obviously a very subjective definition, and I suspect a lot of capes are called capes because that's just what someone decided to call it. "Promontory" is a word I had never heard before. This seems to be a technical term in the field of geography for a relatively steep transition between high land and low land. It doesn't necessarily need to be on the ocean. They can be inland as well. I would suggest sticking to peninsula for most cases, and use cape when cape is in the name of the geographic feature you're talking about.

"Cod" is a kind of fish. "Cape Cod" is named that because a) it is a cape, and b) at the time it was named, the waters around it were rich in codfish.

I think I have it! I suddenly woke up with this idea. It's just a theory. I think it is the difference between land and nautical terminology. It's whether you're looking at the shape of the land or the shape of the ocean. To someone living on land, who wants to live on it, or add territory to a country, it is a "peninsula." To someone sailing a ship, who wants to know the shape of a coast, it is a "cape." You live on a "peninsula," you navigate around a "cape." If I am right, "capes" and "bays" go together and both represent features _of the coastline._ Thus, for example, Cape Cod separates Buzzard's Bay and Cape Code Bay. Of course, when a feature gets named, the name sticks. Most of these features were first seen and name by navigators. So many things are named "cape" (Cape Cod, Cape Hatteras, Cape Horn) and very, very few have "peninsula" as part of their place name. Again, this is just my theory.