We need to define some things here because I think we're calling some of the parts by different names. The flywheel is on the back of the motor, so I think you were referring to the crank's harmonic balancer, on the front of the crankshaft and has the pulley mounted to it. For the sake of our conversation, let's call the part of the raw water pump that has the bolt holes the "mounting flange". The pulley is the black part that has the belt going around it and has grooves.
The pulley that the belt goes around may move when the bolts are loosened, is this what you meant when you said "the flywheel moves side to side a bit."?
The pulley should be true when it rotates and there should be almost no perceptible "out of roundness" to it. The crank pulley should look very stable when the motor runs. If that wobbles, it may have loosened and/or may have been removed and re-installed badly. I can't say for sure because I haven't seen it run, which leads me to ask, is there's any way you can e-mail a video of the pulley and raw water pump in action? If this is possible, that would be great and it doesn't need to be a long video, but it does need to be fairly well lit.