Bearing misalignment may be one of the causes of cage fracture fault. However, the research of bearing misalignment on vibration characteristics of the rotating machinery system has not been paid enough attention. In this paper, the effects of axial, radial and tilt misalignments of the rotor support bearing on the dynamic characteristics of the planetary gear set-rotor system are analyzed. Firstly, based on the Hertz contact theory, a nonlinear restoring force model of the ball bearing is proposed under axial, radial, and inner/outer ring tilt misalignments. Secondly, taking the planetary gear transmission system of a tracked vehicle as the research object. The dynamic model of the planetary gear set is established through the lumped parameter method. The rotor is established by the Timoshenko beam element. Finally, the dynamic model of the planetary gear set-ball bearing-rotor system is obtained by coupling these three parts. Based on the proposed model, the dynamic responses are analyzed considering bearing misalignment. The results showed that the three types of misalignments reduce the vibration amplitude due to the strengthened constraint of the bearing on the system, but increase the load-bearing area and contact load. Radial and tilt misalignments increase the varying compliance (VC) frequency amplitude of the bearing. On the contrary, axial misalignment reduces the VC frequency amplitude. The sensitive evaluation indexes of three misalignment states are given. The research results can provide a theoretical basis for fault diagnosis and identification of bearing misalignment in complex rotating machinery system.