Obsessed: If your rudder shaft moves back & forth/side to side in the port, most likely the seal is bad; it should fit pretty snug. As mentioned, sometimes greasing the shaft will help seal it better and prevent water from leaking in. You should grease the port/shaft annually anyways.
I believe your rudder port was made by OJ Props...I would send your pic to Eric and get his opinion...extremely helpful, super customer service! I just replaced my 2005 rudder port because of seal wear; the new one works great/no more water in the bilge! mac