In the distance means that something or someone is physically located in a distant location from you. For example:
"As I was driving through the tunnel, I began to see light in the distance."
At a distance doesn't necessarily imply a location, but rather you know something or someone from far away or not in real life. For example:
"I knew her for years at a distance, but it was the first time I actually met her in real life."
PS: In the second definition, you can say either "at" or "from" a distance. They are used interchangeably.