correct me if I am wrong but....

Make sure you only block that port coming inbound from the network. dont block connections from localhost to localhost, or you wont be able to run xwindows.