Yes, you should absolutely care. There have been more than a handful of directory traversal exploits out there over the years, many of which gave you access to the shadow PW file as well as many other "protected" areas of the file system. In fact, just a few weeks back, I found one in a very well respected product.

Do yourself a favor, especially if you're going to expose that host to the internet, follow security best practices. Good passwords are the foundation to good security.

--TH13