Perfect. You can place "thoughtfully" anywhere in the sentence:
"Thoughtfully, he was looking at the picture" ("thoughtfully" modifies "he")
"He, thoughtfully, was looking at the picture" ("thoughtfully" modifies "he")
"He was thoughtfully looking at the picture" ("thoughtfully" modifies "looking")
"He was looking thoughtfully at the picture" ("thoughtfully" modifies "looking")
He was looking at the picture thoughtfully. ("thoughtfully" modifies "looking")
He was looking at the picture, thoughtfully. ("thoughtfully" modifies either word)
The decision where best to place "thoughtfully" depends upon which word you wish it to modify. Do you want it to modify "looking" (the way he looked)? Do you want it to modify "he" (the thoughtful person)? You should place it closest to the word you want it to modify.