小声一点! (Xiǎoshēng yīdiǎn!) is certainly one way to do it. It is more like "Not so loud!", or, more literally, "A little quieter!"
The most direct to say it is 闭嘴!(Bì zuǐ!) and it means "Shut up!" in English.
Or you can simply say 嘘!(Xū!) which is like "Shhh!"
A very common word for quiet is 安静 (ānjìng). You can say e.g.
能不能安静一点?(Néng bù néng ānjìng yīdiǎn?) Can you quiet down a bit?
or
大家安静一下!(Dàjiā ānjìng yīxià!) Everyone, quiet down!