To explore the physics behind it in a more quantitative manner, suppose that a rider is sitting on the back of a horse, and that both are initially stationary. The coefficient of friction between the horse's back and the rider's pants is $\mu_s$. Suddenly, the horse accelerates forwards with acceleration $a$.
Newton's second law for the rider has the following form:
$$m_r a_r = \mu_s N_n,$$
where \(a_r\) and \(m_r\) are the rider's acceleration and mass respectively, and \(N_n\) is the normal force exerted by the horse's back.
At the moment right after the horse starts accelerating, \(a_r = 0\), so we get:
$$0 = \mu_s N_n.$$
This means that the frictional force between the rider and the horse is initially zero, and the rider remains stationary (Newton's first law). The relative velocity between the rider and the horse starts to increase (since the horse is accelerating forwards, while the rider lags behind). Only after some time has passed, the friction will slow down the rider and accelerate him in the direction of the horse's motion.