In most contexts they are identical.
You can also say, 'dole out', 'portion out', 'mete out', and 'toss out', although these expressions have additional meanings.
"Handout" is also a noun, meaning the thing which is distributed.
An action of 'handing out' normally implies it's a thing that's meant to be distributed, such as:
"The teacher handed out the tests"
"The application forms were handed out"
An action of giving out, on the other hand, often implies a donation.
"They gave out clothes to the needy"
"They gave out advice for the uninitiated"
"Give out these umbrellas to those soaking wet tourists"
"Distribute" would be a synonym which would also work in both cases.