Dynamic pickup and delivery problem which involves random orders, transshipment locations selection and last-in-first-out (LIFO) constraints, brings more challenges for distribution optimization and routing planning. Therefore, we first construct a multi-objective mathematical model for dynamic pickup and delivery problem with transshipments and LIFO constraints (DPDPTL) aiming at the shortest driving distance and the highest order satisfaction. Especially to get a feasible scheme at transshipment locations, the synchronization time of vehicles arriving is considered to obtain the overlapping time windows of order transshipment. Furthermore, in terms of LIFO constraint which easily causes the obtained scheme infeasible and the solution falling into local optimum, an improved heuristic algorithm is proposed to improve the solution quality, where the initial solution generated by Clarke-Wright (CW) saving algorithm as the input is incorporated into adaptive large neighborhood search (ALNS), and Q-learning is adjusted the operator weights to improve solving efficiency. Finally, the experiments are operated to verify the validity of model and the superiority of algorithm. The results show that there is a dilemma between driving distance and order satisfaction, longer driving distance often leads to higher satisfaction, while shorter driving distance leads to lower satisfaction. DPDPTL proposed in this paper is widespread in large-scale distribution, by combining or splitting similar orders, transshipment can improve the delivery efficiency and get better solutions.