I find both of those sentences odd. Yes, I think it does need to be negative. With 'let alone' there should also be an implicit 'value' to the things - the second, 'let alone' one should be clearly more difficult. Remembering a birthday would normally be easier than remembering an anniversary, that may be why that example doesn't really work.

I agree with Gary. The sentence does need to include a negation, and the point after "let alone" should be more difficult, or less obvious or necessary, than the first point. "You didn't even remember my birthday this year, let our anniversary." You should have remembered all of these things, but you didn't even remember the most important one (the birthday). "You didn't even look at your own daughter's paintings, let alone anybody else's paintings." It would have been best if you'd looked at many people's paintings, but in fact, you didn't even look at the ones that should have been most important to you, your daughter's.

Thank you for your helpful and detailed comment, Legz:)

Most commonly, "let alone" is used to add extra emphasis to a statement. By itself it doesn't feel particularly positive or negative. It depends on the context. I second Gary's notion that whatever follows "let alone" must be more difficult to obtain than what is mentioned before. EX: I don't have a bicycle, let alone a car. Think of the situation. You're trying to get somewhere, but have no way to get there. Public transport isn't available so you try to arrange transportation by another means. The person on the phone isn't particularly helpful and can't understand why you can't meet them somewhere far away. In a moment of frustration, to emphasize exactly how "without transport" you are, you say "I don't have a bicycle, let alone a car!" This tells the person that you really need their help because you don't even have a bicycle to get you from place to place. Transport via car is much more common than transport via bicycle, and usually if people don't have a car, they at least have a bicycle. Someone without a bicycle or public transport is stuck walking from place to place without help. !!Bonus!! In the southern states to "let (someone) alone", it means to leave someone alone. Ex: "Let her alone!" It's completely incorrect grammatically, but used quite often in the deep south.