Flow shop scheduling

Engineering

Statistics

Markov process

Two-level scheduling

Scheduling (production processes)

Operating system

Computer science

Mathematical optimization

Markov decision process

Distributed computing

Preemption

Operations research

Reinforcement learning

Rate-monotonic scheduling

Mathematics

Dynamic priority scheduling

Industrial engineering

Artificial intelligence

Schedule

Fixed-priority pre-emptive scheduling

Job shop

Fair-share scheduling

Job shop scheduling

Robotics and Computer-Integrated Manufacturing

In smart manufacturing, robots gradually replace traditional machines as new processing units, which have significantly liberated laborers and reduced manufacturing expenditure. However, manufacturing resources are usually limited so that the preemption relationship exists among robots. Under this circumstance, job scheduling puts forward higher requirements on accuracy and generalization. To this end, this paper proposes a scheduling algorithm to solve job scheduling problems in a resource preemption environment with multi-agent reinforcement learning. The resource preemption environment is modeled as a decentralized partially observable Markov decision process, where each job is regarded as an intelligent agent that chooses an available robot according to its current partial observation. Based on this modeling, a multi-agent scheduling architecture is constructed to handle the high-dimension action space issue caused by multi-task simultaneous scheduling. Besides, multi-agent reinforcement learning is employed to learn both the decision-making policy of each agent and the cooperation between job agents. This paper is novel in addressing the scheduling problem in a resource preemption environment and solving the job shop scheduling problem with multi-agent reinforcement learning. The experiments of the case study indicate that our proposed method outperforms the traditional rule-based methods and the distributed-agent reinforcement learning method in total makespan, training stability, and model generalization.

Solving job scheduling problems in a resource preemption environment with multi-agent reinforcement learning