The problem is caused by the large number of refit orders, especially refit to any available cargo.
This interacts poorly with cargodist, in that it needs to estimate the link capacity for all 64 cargoes.
In particular this involves querying the GRF for the refitted capacity for each cargo for each vehicle at each refit order.
Refreshing the link capacities and checking for stale links generates a large number of such operations, and as GRF operations are very slow, this creates a noticeable pause.
I can probably optimise this a bit, but in general using both cargodist and refit to any available cargo is problematic.