Deep learning

Key (lock)

Natural language understanding

Biology

Semantic similarity

Lexical item

Philosophy

Paleontology

Artificial intelligence

Natural language

Semantics (computer science)

Programming language

Natural language processing

Distributional semantics

Context (archaeology)

Computer science

Linguistics

Question answering

Semantic network

Lexical semantics

Computer security

Artificial neural network

Word (group theory)

Computing

The representational basis for downstream natural language processing tasks is word embeddings, which capture lexical semantics in numerical form to handle the abstract semantic concept of words. Recently, the word embeddings approaches, represented by deep learning, has attracted extensive attention and widely used in many tasks, such as text classification, knowledge mining, question-answering, smart Internet of Things systems and so on. These neural networks-based models are based on the distributed hypothesis while the semantic association between words can be efficiently calculated in low-dimensional space. However, the expressed semantics of most models are constrained by the context distribution of each word in the corpus while the logic and common knowledge are not better utilized. Therefore, how to use the massive multi-source data to better represent natural language and world knowledge still need to be explored. In this paper, we introduce the recent advances of neural networks-based word embeddings with their technical features, summarizing the key challenges and existing solutions, and further give a future outlook on the research and application.

A survey of word embeddings based on deep learning