"Can you (as cats do) begin with a C" is an actual joke that can be explained. Trained female opera singers--sopranos--are capable of reaching a "high C," the note on a piano that is an octave above middle C. Good singers can sing it beautifully, but it is considered the high end of the range and it is a challenge. And a singer would not begin with a high C, they would work up to it--it would be at the climax of a song, near the end.
The word "cats" begins with a C.
So "can you (as cats do) begin with a C" is a pun. So they can "begin with a C," sort of.
The song is combines many different impressions and feelings. It suggests that cats are magical, mystical, have special powers, are connected with myths and fairy tales. At the same time, this solemn content is expressed in verse that sounds humorous, like the books of Dr. Seuss. There is too much clever internal rhyme ("tense when you sense," "blind when you find.") These carry the idea that the singer is a clever trickster.
And also a highly educated one.
A few more until I run out of space. "Can you ride on a broomstick to places far distant?" is a reference to the idea of witches riding broomsticks. Part of the legends of witches is that they have pet cats, their "familiars," who share in their witchcraft.
The "Heaviside layer" is not really a cultural reference, exactly. The Heaviside layer is a layer in the atmosphere, in the ionosphere, which reflects radio waves. Before satellites, shortwave radio worked by bounding waves off the Heaviside layer. It means the very highest level of the atmosphere.