If this is the case for Brexit, I worry for eurosceptics