Question answering

Context (archaeology)

Linguistics

Engineering

Philosophy

Aerospace engineering

Biology

Latent variable

Variable (mathematics)

Range (aeronautics)

Reciprocal

Computer science

Language model

Paleontology

Natural language processing

Mathematics

Leverage (statistics)

Mathematical analysis

Artificial intelligence

Human–computer interaction

Machine learning

Latent variable model

Pre-training models have been proved effective for a wide range of natural language processing tasks. Inspired by this, we propose a novel dialogue generation pre-training framework to support various kinds of conversations, including chit-chat, knowledge grounded dialogues, and conversational question answering. In this framework, we adopt flexible attention mechanisms to fully leverage the bi-directional context and the uni-directional characteristic of language generation. We also introduce discrete latent variables to tackle the inherent one-to-many mapping problem in response generation. Two reciprocal tasks of response generation and latent act recognition are designed and carried out simultaneously within a shared network. Comprehensive experiments on three publicly available datasets verify the effectiveness and superiority of the proposed framework.

PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable