Strictly, experiment is what a scientist does as part of his research. Experimentation is a noun for the activity of performing experiments.
More generally, "experiment" can be applied to trying something new to oneself in any part of life: "She had never been to the opera before, and thought of her attendance tonight as an experiment/as a form of experimentation."
"Experience" is either an activity one has done, or the knowledge gained from that activity: "That was my first experience cooking for forty people. Fortunately, there was another cook there who had twenty years of experience doing that kind of cooking."