You don't have to accomodate everyone, but you should do your best to make you site useable by most people. You wouldn't think many blind people use the Internet, but they really are begining to make an impact in some areas. When running a business your target audience is everyone willing to pay for what you are selling or at least that has always been my philosophy. Sure you direct the site mainly at certian groups but you need to make sure other groups can use it as well, or you could be losing money.

We have a girl in our e-Commerce program that is bascially completly blind. She has everyone use her laptop every now and then with their eyes closed just to see what it is like. All I can say is that most sites are impossible to use if you can't see, but some are pretty smooth.