Ruling elite in ‘state of panic’ over Brexit – Galloway