The impact of fishbone oscillations onto a confined fast ion population is simulated for a JET plasma and benchmarked against experiment with the help of neutron rate measurements. The transient drops in volume integrated neutron emission are found to be mainly caused by a redistribution of the confined ion population rather than fast ion loss. The simulations yield a quadratic dependence of the neutron drop on the fishbone amplitude (rather than the linear dependence expected from mode particle pumping theory), which is indicative of fast ion stochastic orbit diffusion dominating over convective transport processes. It is found that the simulations are able to correctly reproduce the magnitude of the experimentally observed drop in volume integrated neutron emission to within a factor 2.