Anonymous access won't work

I have anonymous access set up on one of my web applications in SharePoint 2010. I've been using it without a problem for several weeks. However, recently I temporarily disabled the anonymous access at the app level from CA: Application Management > Manage web application > app name > Authentication Providers > Default > Unchecked "Enable anonymous access". When I did this, it started prompting me for users' credentials and it won't accept any of them. It just kept prompting. So I went back to CA and enabled the anonymouse access back. But it did NOT fix the problem! That was the only thing I changed! I even restarted the web app in IIS, that didn't help. Any ideas?