Shadow ministers “haven’t run a piss-up in a brewery”, says Tory star