Prince Harry’s invitation to stay at Buckingham Palace during his trip was rescinded days before he arrived.