Computation offloading

Statistics

Markov process

Virtualization

Server

Operating system

Telecommunications

Computer science

Cloud computing

Markov decision process

Distributed computing

Computer network

Reinforcement learning

Mathematics

Enhanced Data Rates for GSM Evolution

Mobile edge computing

Artificial intelligence

Wireless

Edge computing

Wireless network

IEEE Transactions on Vehicular Technology

The rapid development of maritime activities has led to the emergence of more and more computation-intensive applications. In order to meet the huge demand for wireless communications in maritime environment, mobile edge computing (MEC) is considered as an effective solution to provide powerful computing capabilities for maritime terminals of resource scarcity or latency sensitive. A basic technology to implement MEC is virtual machine (VM) multiplexing, through which multi-task parallel computing on a server is realized. In this paper, a two-layer unmanned aerial vehicles (UAVs) maritime communication network with a centralized top-UAV (T-UAV) and a group of distributed bottom-UAVs (B-UAVs) is established and MEC is used on T-UAV. We aim to solve the latency minimization problem for both communication and computation in this maritime UAV swarm mobile edge computing network. We reformulate this problem into a Markov decision process (MDP), since it is a non-convex and multiply constrained but has the characteristics of MDP. Based on this MDP model, we take deep reinforcement learning (DRL) as our tool to propose a deep Q-network (DQN) and a deep deterministic policy gradient (DDPG) algorithms to optimize the trajectory of T-UAV and configuration of virtual machines (VMs). Using these two proposed algorithms, we can minimize the system latency. Simulation results show that the given solutions are valid and effective.

Deep Reinforcement Learning Based Latency Minimization for Mobile Edge Computing With Virtualization in Maritime UAV Communication Network