Perplexity

Similarity (geometry)

Image (mathematics)

Language model

Artificial intelligence

Semantics (computer science)

Programming language

Natural language processing

Sentence

Generative grammar

Computer science

Transactions of the Association for Computational Linguistics

We propose a new generative language model for sentences that first samples a prototype sentence from the training corpus and then edits it into a new sentence. Compared to traditional language models that generate from scratch either left-to-right or by first sampling a latent sentence vector, our prototype-then-edit model improves perplexity on language modeling and generates higher quality outputs according to human evaluation. Furthermore, the model gives rise to a latent edit vector that captures interpretable semantics such as sentence similarity and sentence-level analogies.

Generating Sentences by Editing Prototypes