The two nouns 'passivity and passiveness' are synonyms and both have the same definitions, 'the state or condition of being passive.'
1 the trait of remaining inactive;
2 a lack of initiative / submission to others or to outside influences;
3 the trait of being willing to yield to the will of another person or a superior force.
The only difference between the two (IMHO), if there a difference, might be in their usage.
I would use 'passivity' in a way similar to a mass/collective noun, describing the passive nature or state of a large group, and 'passiveness' being used to describe a person's state of being passive.
For example:
The nation is failing because of the passivity of its population.
He/She is failing because of his/her passiveness.
Please note that my interpretation of the usage difference is just my opinion and is not based on anything I've read or been taught.
**********************************
submissiveness (noun): the trait of being willing to yield to the will of another person or a superior force.
e.g., The employees exhibited their submissiveness towards their demanding boss.
submission (noun): the act of submitting; usually surrendering power to another
e.g., The boss demanded his employee's submission.