"One degree Celsius." (Or "Centigrade," although I think that term is deprecated).
As a computer programmer, this is an issue I had to cope with frequently when writing code to express numbers in words. For any unit, if the quantity is exactly one, you use the singular: "one unit." If it is anything else, including zero, you use the plural, "units." If it is expressed with a decimal point, you use the plural even if all of the digits after the decimal are zeroes. You use the singular for both positive and negative one.
Thus:
39.37 inches: "Thirty nine point thirty seven inches."
1 g: "One gram."
2 cm: "Two centimeters."
-1 ºC: "Minus one degree Celsius."
1.00 km: "One point zero zero kilometers." (Or, alternatively, "exactly one kilometers.")
P.S. For programmers: if you can't be bothered to write the code to use the singular form, an easy way out is to use the phrase "number of;" for example, "number of words: 1". Because of the logical structure of the phrase, no matter how many words there are, "number of words" is always correct no matter what the number of words is.