The only way you can use "worry about that" is if you already established what you are worried about.
For example, if two people were talking and they said:
"I'm worried that I'll be late to class today."
"Yeah. I'm worried about that, too."
- - -
Other than that, it makes more sense to say sentences like "He worried that the outsiders might have opinions about him."
Hope this helped :)