It means to make a person angry and frustrated, usually in a situation where they can't do something about it and often some constant situation. Imagine the feeling you would have if you swallowed something that stuck in your throat (craw).
"The way that he orders people around really sticks in my craw."
"The messes she leaves all over the house, never picking up after herself, is beginning to stick in my craw.