A crow is a black bird and the expression refers to it flying in a direct line from A to B without the detours we humans have to take,when following a road.It is an odd choice though ,because crows in their migration don't necessarily fly straight and many times fly in wheeling arcs in their quest for food.
So the expression indicates the most direct route that could be taken from A to B.
When using it ,it indicates an approximate estimation of the distance way lesser than the real distance crossed,due to encumbrances of roads and landscape features that man is hampered by.
The earliest known citation of the phrase, which explicitly defines its meaning, comes in The London Review Of English And Foreign Liturature:
" The Spaniaad , if on foot, always travels as the crow flies, which the openness and dryness of the country permits; neither rivers nor the steepest mountains stop his course, he swims over the one and scales the other."