Mechanism (biology)

Quantum mechanics

Gyroscope

Wireless sensor network

Philosophy

Operating system

Computer science

Computer network

Accelerometer

Physics

Pattern recognition (psychology)

Mathematics

Activity recognition

Epistemology

Domain (mathematical analysis)

Mathematical analysis

Artificial intelligence

Artificial neural network

Convolutional neural network

Recurrent neural network

Machine learning

Sensor-based human activity recognition is a fundamental research problem in ubiquitous computing, which uses the rich sensing data from multimodal embedded sensors such as accelerometer and gyroscope to infer human activities. The existing activity recognition approaches either rely on domain knowledge or fail to address the spatial-temporal dependencies of the sensing signals. In this paper, we propose a novel attention-based multimodal neural network model called AttnSense for multimodal human activity recognition. AttnSense introduce the framework of combining attention mechanism with a convolutional neural network (CNN) and a Gated Recurrent Units (GRU) network to capture the dependencies of sensing signals in both spatial and temporal domains, which shows advantages in prioritized sensor selection and improves the comprehensibility. Extensive experiments based on three public datasets show that AttnSense achieves a competitive performance in activity recognition compared with several state-of-the-art methods.

AttnSense: Multi-level Attention Mechanism For Multimodal Human Activity Recognition