I didn't like this beret, but now it IS GROWING ON ME because I look very cute when I wear it.
Another option: "but it has grown on me"
The closest opposite I can think of is "to go off" something (but this is pretty specific to the UK). e.g. I started to go off this beret when I noticed another kid in school wearing the same one.
Another good option is "to be over," e.g. I'm completely over this beret now - the color is completely faded.
You could also use "to lose interest in," or "to get bored of", but they aren't quite as specific.
Hope that helps! :)