Aerodynamic shape optimization is usually a loop of an optimization model, an optimizer and an evaluation workflow. A new optimizer is proposed and tested for a typical aerodynamic shape optimization of missile control surfaces with computational fluid dynamics (CFD). The new optimizer emphasizes the use of machine learning techniques , reinforcement learning and transfer learning , to improve performance and efficiency. Reinforcement learning is applied to extract the optimization experience from the semi-empirical method DATCOM using deep neural networks . Transfer learning is implemented to reuse the experience as priori knowledge in the CFD-based optimization by sharing neural network parameters. For the considered aerodynamic shape optimization problem of missile control surfaces, a remarkable reduction in the computational time has been accomplished. The new approach significantly decreases the required CFD calls by over 62.5%. Meanwhile, the time spent in the experience extraction and parameter transfer process is negligible.