How to Grant Anonymous Access to Webpage that has unique permissions?

I have anonymous access configured on our sharepoint server and it's working fine.  The issue is that if I try granting design access to a specific user, the anonymous access to that page goes away when I agree to stop inheriting permissions.  This only seems to happen with web pages and not other items such as lists for example.