Mine is a diesel too and yes, you notice that there is a Star behind you but it is still ok to tow/drive. Especially with the 8 gear transmission which mine has. I could imagine that 'only' having half of the weight behind you (Prostar) you won't really notice.
My decision was to go with the expensive X5 because I need the car to travel as well and I did not want to have two cars, one for towing and one for the normal usage. It uses about 9 Liter per 100 km without and 10.2 with the boat behind so not much of a difference there.