It was a popular saying during his presidency, used to justify his decision to run for a third term in 1940 during World War II. The idea was that it was dangerous and unwise to change leadership in the middle of a crisis.
While FDR didn't invent the saying, he was certainly associated with it. It became a popular political slogan used to discourage switching political parties or presidents during difficult times.