You may or may not be surprised to learn that not everybody visiting your site has perfect eyesight. In fact, over a million people in the UK have been identified as having sight problems. The number of people unidentified, or choosing to surf with larger fonts for another reason, is far higher. If you aren't using resizable text, you are putting those visitors off. And probably their friends as well.
That said, there is another group within that group for whom setting fonts so they can be resized will have no effect. Those users, the blind, are unable to see just how pretty your site looks. They use screen readers, software that reads the content of a page to them (or outputs it through a braille device).
This brings with it a whole host of problems, most of which are reasonable simple to deal with.
For example, did you know that a screen reader will try to pronounce "CSS" as a word, rather than spelling it out letter by letter? The way to work around that is to write it like so:
<abbr title="Cascading Style Sheets">CSS</abbr>
(Please note that the <acronym> tag is due to be deprecated in XHTML 2.0. With that in mind, it is better to just use the <abbr> tag from now).
In the above example, a screen reader will read out the value of the title attribute, "Cascading Style Sheets", instead of trying to pronounce what it now knows to be an abbreviation.
One of the most frequent problems blind users face, aside from image tags missing their alt attributes (imagine having to hear "spacer dot gif" twenty times every time you view a page on a site - would you return?), is that every page view means having to hear the title and navigation of the page read out before the content. This, especially on large sites, is extremely frustrating. I can vouch for this - I've spent time using screen readers specifically to find out what problems are faced.
There are a few techniques around for enabling the user to skip past the navigation and get to the content. One method involves placing the navigation after the content. I don't personally see a great deal of advantage to this method, as you still need to add a "skip to navigation" link to make the page accessible.
The method I prefer to use is to hide a link at the top of the page, before any other content. That link links to an anchor within the page marking the beginning of the actual content.
The link is hidden from visual browsers. Some people leave it as a visible link, and it can be a useful navigation aid. Personally, I prefer to hide it, but the choice is yours. The markup for this is usually done line so:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"><html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" ><head><title>Page Title</title></head><body><div id="skipnavigation"><a href="#content" accesskey="2" title="AccessKey 2: Skip to Content">Skip Navigation</a></div><div id="header">Page Title</div><div id="navigation">Menu<br /><a href=".">Home</a><br /><a href=".">About Us</a><br /><a href=".">Products</a><br /><a href=".">Services</a><br /><a href=".">Contact Us</a></div><div id="content">Page Content</div></body></html>
Now, without the skip navigation link, a user visiting the above would need to sit through a screen reader reading out "Page Title. Menu. Home. About Us. Products. Services. Contact Us." before actually getting to the content of the page (and that's a very slim version of what most pages actually contain and what screen readers may read out before the content). By adding a link to the beginning, we give them a means to skip past all the navigation and get straight to the bit of the page they are interested in. A "Back to Top" link is also very useful after content - without that, a user can't get back to the navigation, which isn't a lot of help if they want to move on from the page, because without that they are going to have trouble returning to the navigation.
Now we've added that link, we know that those without a traditional visual browser can jump to the points of the page they are used to. The addition of an Access Key means that most people will be able to skip to the content on each page without needing to wait for that link to be read out again (2 is usually chosen as the access key for the skip navigation link ... more on access keys is available if you would like to know more).
I usually also hide this link from visual browsers. There are plenty of ways to do this, however some are better than others. Be careful when hiding a link that you do not hide it from the screen readers it is there to help! Using "display: none;" in your CSS will hide the link from many user agents. I usually use the below, which simply places the link out of the visible area of the page (but it will still be perfectly usable for screen readers and text-based browsers):
#skipnavigation {position: absolute;left: -1000px;}
AddedBytes.com is the online playground of 
Dooyoo know of any sites like that?