RL training can cause a significant distribution shift away from the human text on which models were pre-trained