@Luke, Thank you so much for giving me your opinion based on the experience. It's been really helpful! :D

I think so. I'm not Japanese of course and I speak English, but I studied Japanese before  Korean. Learning postpositions (조사) and the subject-object-verb order was really hard for me when I first studied Japanese and I struggled a lot but eventually got the hang of it. When started studying Korean, though, my Japanese knowledge was really helpful and those parts of Korean were no problem at all. As a result, I picked up Korean pretty quickly for an English speaker.

Also, when I studied Korean at 언어학당, the Japanese students were so much better than me and their Korean seemed effortlessly perfect. 

I think if you study Japanese, you'll find it much easier than English. You still gotta learn the annoying Chinese characters (한자) though ㅠㅠ