Rare X-shaped radio galaxies shaped by feeding supermassive black holes