This paper proposes an optimized tracking control approach using neural network (NN) based reinforcement learning (RL) for a class of nonlinear dynamic systems, which requires both tracking and optimizing to be performed simultaneously. Generally, for obtaining optimal control solution, Hamilton-Jacobi-Bellman equation is expected to be solvable, but, owing to strong nonlinearity, the equation is solved difficultly or even impossibly by analytical methods. Therefore, adaptive NN approximation based RL is usually considered. In the optimized control design, for driving output state following to the desired trajectory, an error term is split from optimal performance index function, and then both actor and critic NNs are built to perform RL algorithm. Actor NN aims to execute control behaviors, and critic NN aims to appraise control performance and make feedback to actor. The proof of stability concludes that the desired control performances are obtained. A numerical simulation is designed and implemented, and the desired results are shown.