It helps if you hold your rifle securely.
Also it sounds like you are using scopes with no padding. For those scopes dont put your eye on the actual scope, keep it a half inch back. Adjust the scope and you can see it perfectly a distance back bacause of where the image is focused.
Some scopes do have padding/sun shields… Get one of those if your scope cant be focused…