There is a dance here. Too small an engine will be working its *** off with a heavy load. The dance part is how often you waltz and how often you Tango.
If you tow a trailer mainly only for 2 weeks for summer vacation, go with the smaller engine. If hauling a work trailer all week, the larger engine will not work as hard, may get a bit better economy and will last longer.