My dog growls at me sometimes when I am petting him or talking to him?

He is a 3 year old german shepherd. He is very sweet annd affecionate to me and my family as well as strangers and other dogs. Sometimes when I am petting him and talking to him he does a low growl and seems scared If I play with him roughly he never bited me hard and is very gentle. I do not like the growling and want it to stop. He does it when he has a bone too. If I try to scold him or hit him to make him stop he just growls louder. I have taken his toys away when he growls at me and he never bites me but Ido not know what to do. I have tried pining him on his back to show him I am the dominate one but he just keeps growling. What can I do?