How does he know it's ammunition? You could have ordered some brick and tile samples.
I would, at your next interaction where he says stupid shit like this, insinuate that he broke federal law and opened your packaging because he doesn't agree with your 2nd.
If your ammo guy shipped it in a non-standard box that said AMMO FOR BABYKILLER ANON he fucked up. Your carrier should never and I do mean NEVER be able to tell if you're receiving ammo if it's sent through a private party and not the goddamn postal service.
Also get a porchlight camera - they're absurdly cheap and most people think they're a decorative part of the light if installed correctly.