Yebbut, a top cyclist riding a Raleigh Shopper would beat me off-road on my mtb and on-road on my 'dale but I still don't want to ride a Raleigh Shopper (and nor do they really, I imagine) ;-)
You just more technical terrain. They won't beat you if they must carry the shopper while you still ride.