when using BY, it means the action or the event will start before that time and be done by that time.
I will be home by 5 pm - within 5 pm, I will reach home.(of course implying I will start for home before 5)
When using AT, it just speak of the instance, the time mentioned.
Let us meet at 6 pm = we will meet when the time is 6 pm (and the meeting may go on, no end time is mentioned)
Hope this helps :)