Task scheduling is one of the essential techniques in the cloud computing environment. It is required for allocating tasks to the proper resources and optimizing the overall system performance. Particle swarm optimization (PSO) algorithm is one of the most popular scheduling algorithms, which is used to maximize resource utilization. However, the performance of the PSO scheduling algorithm decreases when the number of tasks is significant. In this paper, the improved PSO (IPSO) algorithm is proposed to provide the optimal allocation for a large number of tasks. This is achieved by splitting the submitted tasks into batches in a dynamic way. The resources utilization state is considered in each creation of batches. After getting a sub-optimal solution for each batch, the algorithm appends all the sub-optimal solutions for batches into a final allocation map. Finally, IPSO tries to balance the loads over the final allocation map. The proposed algorithm is compared with different scheduling algorithms, namely, honey bee, ant colony, and round-robin algorithms. The results of experiments show the efficiency of the proposed algorithm in terms of makespan, standard deviation of load, and degree of imbalance.