Not sure what your boat weighs, but I pull my 2200lb 83 S&S with a 4.0L Cherokee XJ. I don't have any problems pulling it up the ramp, and it pulls straight without any sway. My trailer doesn't have brakes either, and the Jeep handles it fine. I should also note that I'm lifted about 5.5" on 32" tires with no regearing. I wouldn't pull across the country with it, but to and from the lake or from my house to storage unit (20 miles) it does just fine. There's a picture of it a couple pages back I think. You said you're 80lbs under the limit... I'd be careful with that. Be sure to add in the weight of fuel, and all your gear that's loaded in the boat.
Were they still putting the 4.0L in the Wrangers in 05?