Fed Chairman Kevin Warsh was scheduled to speak at the IMF on a Friday