Joseph Schwartz attended a White House Hanukkah party in the month following Donald Trump granting Schwartz a presidential pardon