Artificial Neural Network Models

Published on Jan 1, 2015
· DOI :10.1007/978-3-662-43505-2_27
Peter Tino25
Estimated H-index: 25
(University of Birmingham),
Lubica Benuskova19
Estimated H-index: 19
(University of Otago),
Alessandro Sperduti34
Estimated H-index: 34
(UNIPD: University of Padua)
We outline the main models and developments in the broad field of artificial neural networks (ANN). A brief introduction to biological neurons motivates the initial formal neuron model – the perceptron. We then study how such formal neurons can be generalized and connected in network structures. Starting with the biologically motivated layered structure of ANN (feed-forward ANN), the networks are then generalized to include feedback loops (recurrent ANN) and even more abstract generalized forms of feedback connections (recursive neuronal networks) enabling processing of structured data, such as sequences, trees, and graphs. We also introduce ANN models capable of forming topographic lower-dimensional maps of data (self-organizing maps). For each ANN type we outline the basic principles of training the corresponding ANN models on an appropriate data collection.
📖 Papers frequently viewed together
5 Citations
4 Citations
#1Dan Ciresan (USI: University of Lugano)H-Index: 22
#2Ueli Meier (USI: University of Lugano)H-Index: 21
Last. Jürgen Schmidhuber (USI: University of Lugano)H-Index: 102
view all 4 authors...
Good old online backpropagation for plain multilayer perceptrons yields a very low 0.35% error rate on the MNIST handwritten digits benchmark. All we need to achieve this best result so far are many hidden layers, many neurons per layer, numerous deformed training images to avoid overfitting, and graphics cards to greatly speed up learning.
636 CitationsSource
#1Dan CiresanH-Index: 22
#2Ueli MeierH-Index: 21
Last. Jürgen SchmidhuberH-Index: 102
view all 4 authors...
Good old on-line back-propagation for plain multi-layer perceptrons yields a very low 0.35% error rate on the famous MNIST handwritten digits benchmark. All we need to achieve this best result so far are many hidden layers, many neurons per layer, numerous deformed training images, and graphics cards to greatly speed up learning.
511 CitationsSource
#1Alex Graves (Information Technology University)H-Index: 63
#2Marcus LiwickiH-Index: 27
Last. Jürgen Schmidhuber (Information Technology University)H-Index: 102
view all 6 authors...
Recognizing lines of unconstrained handwritten text is a challenging task. The difficulty of segmenting cursive or overlapping characters, combined with the need to exploit surrounding context, has led to low recognition rates for even the best current recognizers. Most recent progress in the field has been made either through improved preprocessing or through advances in language modeling. Relatively little work has been done on the basic recognition algorithms. Indeed, most systems rely on the...
1,286 CitationsSource
#1Markus Hagenbuchner (UOW: University of Wollongong)H-Index: 20
#2Alessandro Sperduti (UNIPD: University of Padua)H-Index: 34
Last. Ah Chung Tsoi (Hong Kong Baptist University)H-Index: 20
view all 3 authors...
Self-organizing maps capable of processing graph structured information are a relatively new concept. This paper describes a novel concept on the processing of graph structured information using the self-organizing map framework which allows the processing of much more general types of graphs, e.g. cyclic graphs, directed graphs. Previous approaches to this problem were limited to the processing of bounded graphs, their computational complexity can grow rapidly with the level of connectivity of ...
24 CitationsSource
This paper presents a new approach for learning in structured domains (SDs) using a constructive neural network for graphs (NN4G). The new model allows the extension of the input domain for supervised neural networks to a general class of graphs including both acyclic/cyclic, directed/undirected labeled graphs. In particular, the model can realize adaptive contextual transductions, learning the mapping from graphs for both classification and regression tasks. In contrast to previous neural netwo...
200 CitationsSource
This paper addresses the problem of recovering both the intrinsic and extrinsic parameters of a camera from the silhouettes of an object in a turntable sequence. Previous silhouette-based approaches have exploited correspondences induced by epipolar tangents to estimate the image invariants under turntable motion and achieved a weak calibration of the cameras. It is known that the fundamental matrix relating any two views in a turntable sequence can be expressed explicitly in terms of the image ...
17 CitationsSource
#2Herbert JaegerH-Index: 23
Last. Herbert JaegerH-Index: 21
view all 2 authors...
Echo State Networks (ESNs) and Liquid State Machines (LSMs) introduced a simple new paradigm in artificial recurrent neural network (RNN) training, where an RNN (the reservoir) is generated randomly and only a readout is trained. The paradigm, becoming known as reservoir computing, made RNNs accessible for practical applications as never before and outperformed classical fully trained RNNs in many tasks. The latter, however, does not imply that random reservoirs are optimal, but rather that adeq...
25 Citations
#1Sepp Hochreiter (Johannes Kepler University of Linz)H-Index: 39
#2Martin Heusel (Johannes Kepler University of Linz)H-Index: 8
Last. Klaus Obermayer (Johannes Kepler University of Linz)H-Index: 46
view all 3 authors...
Motivation: As more genomes are sequenced, the demand for fast gene classification techniques is increasing. To analyze a newly sequenced genome, first the genes are identified and translated into amino acid sequences which are then classified into structural or functional classes. The best-performing protein classification methods are based on protein homology detection using sequence alignment methods. Alignment methods have recently been enhanced by discriminative methods like support vector ...
95 CitationsSource
#1Matthew H. Tong (UCSD: University of California, San Diego)H-Index: 10
#2Adam D. Bickett (UCSD: University of California, San Diego)H-Index: 1
Last. Garrison W. Cottrell (UCSD: University of California, San Diego)H-Index: 54
view all 4 authors...
Echo State Networks (ESNs) have been shown to be effective for a number of tasks, including motor control, dynamic time series prediction, and memorizing musical sequences. However, their performance on natural language tasks has been largely unexplored until now. Simple Recurrent Networks (SRNs) have a long history in language modeling and show a striking similarity in architecture to ESNs. A comparison of SRNs and ESNs on a natural language task is therefore a natural choice for experimentatio...
137 CitationsSource
Dec 4, 2006 in NeurIPS (Neural Information Processing Systems)
#1Yoshua Bengio (UdeM: Université de Montréal)H-Index: 192
#2Pascal Lamblin (UdeM: Université de Montréal)H-Index: 13
Last. Hugo Larochelle (UdeM: Université de Montréal)H-Index: 59
view all 4 authors...
Complexity theory of circuits strongly suggests that deep architectures can be much more efficient (sometimes exponentially) than shallow architectures, in terms of computational elements required to represent some functions. Deep multi-layer neural networks have many levels of non-linearities allowing them to compactly represent highly non-linear and highly-varying functions. However, until recently it was not clear how to train such deep networks, since gradient-based optimization starting fro...
3,128 Citations
Cited By25
Last. Yan K
view all 5 authors...
#1Yogiraj Sargam (Iowa State University)H-Index: 5
#2Kejin Wang (Iowa State University)H-Index: 35
Last. In Ho Cho (Iowa State University)H-Index: 8
view all 3 authors...
Abstract Thermal conductivity, k, is an important property of concrete, and it influences the design and energy-efficiency of many concrete-based structures. Due to the requirement of sophisticated test procedures, experimental measurement of k of concrete for every such structure is impractical, and therefore, a model for prediction of k is demanded. For this purpose, a data-driven machine learning (ML) model was developed in this study. The dataset for model training was developed from the pub...
4 CitationsSource
#1Mohammed Ali Jallal (Cadi Ayyad University)H-Index: 4
#2Abdessalam El Yassini (Cadi Ayyad University)H-Index: 2
Last. Saida Ibnyaich (Cadi Ayyad University)H-Index: 7
view all 5 authors...
Trustworthy acquaintance and accurate solar radiation measurements are a condition for designing and managing solar energy systems. Frequently, there are substantial spatial and temporal lacks of measures so that predictive approaches become of interest. In the present paper, an ensemble learning approach is proposed based on the deep neural network technique. The suggested approach is applied to forecast the hourly time series of global solar radiation related to Marrakech, Morocco. For that pu...
#1Sameer I. Ali Al-Janabi (University of Anbar)H-Index: 1
#2Sufyan Al-Janabi (University of Anbar)H-Index: 4
Last. Belal Al-Khateeb (University of Anbar)H-Index: 7
view all 3 authors...
Image Retrieval (IR) has become one of the main problems facing computer society recently. To increase computing similarities between images, hashing approaches have become the focus of many programmers. Indeed, in the past few years, Deep Learning (DL) has been considered as a backbone for image analysis using Convolutional Neural Networks (CNNs). This paper aims to design and implement a high-performance image classifier that can be used in several applications such as intelligent vehicles, fa...
#1Alice C. Schwarze (UW: University of Washington)H-Index: 1
#2Mason A. Porter (UCLA: University of California, Los Angeles)H-Index: 68
The study of motifs in networks can help researchers uncover links between structure and function of networks in biology, the sociology, economics, and many other areas. Empirical studies of networks have identified feedback loops, feedforward loops, and several other small structures as "motifs" that occur frequently in real-world networks and may contribute by various mechanisms to important functions these systems. However, the mechanisms are unknown for many of these mechanisms. We propose t...
3 Citations
#1Eunhye Baek (TUD: Dresden University of Technology)H-Index: 5
#2Nikhil Ranjan Das (CU: University of Calcutta)H-Index: 10
Last. Gianaurelio Cuniberti (TUD: Dresden University of Technology)H-Index: 69
view all 14 authors...
Neuromorphic architectures merge learning and memory functions within a single unit cell and in a neuron-like fashion. Research in the field has been mainly focused on the plasticity of artificial synapses. However, the intrinsic plasticity of the neuronal membrane is also important in the implementation of neuromorphic information processing. Here we report a neurotransistor made from a silicon nanowire transistor coated by an ion-doped sol–gel silicate film that can emulate the intrinsic plast...
7 CitationsSource
#1Thai Duong NguyenH-Index: 1
#2Trong Duc NguyenH-Index: 2
During ship manoeuvring and course alteration, the trajectory of ship or the time of ship's turning which remarkably affect safety and effectiveness of navigation depends on ship's factor, meteorological factor and control factor. Simulation commonly is applied for methods used for calculating the ship's trajectory. However, these computed results which are only applied for simulated cases or for certain external affected conditions are not reliable enough for navigation officers to predict the ...
1 CitationsSource
#1Olga Krestinskaya (NU: Nazarbayev University)H-Index: 13
#2Alex Pappachen James (NU: Nazarbayev University)H-Index: 16
Last. Leon O. Chua (University of California, Berkeley)H-Index: 134
view all 3 authors...
The volume, veracity, variability, and velocity of data produced from the ever increasing network of sensors connected to Internet pose challenges for power management, scalability, and sustainability of cloud computing infrastructure. Increasing the data processing capability of edge computing devices at lower power requirements can reduce several overheads for cloud computing solutions. This paper provides the review of neuromorphic CMOS-memristive architectures that can be integrated into edg...
82 CitationsSource
#1Solmaz Rasoulzadeh Gharibdousti (OSU: Oklahoma State University–Stillwater)H-Index: 2
#2Gehendra Kharel (OSU: Oklahoma State University–Stillwater)H-Index: 9
Last. Arthur L. Stoecker (OSU: Oklahoma State University–Stillwater)H-Index: 5
view all 3 authors...
: Best management practices (BMPs) are commonly used to reduce sediment loadings. In this study, we modeled the Fort Cobb Reservoir watershed located in southwestern Oklahoma, USA using the Soil and Water Assessment Tool (SWAT) and evaluated the impacts of five agricultural BMP scenarios on surface runoff, sediment yield, and crop yield. The hydrological model, with 43 sub-basins and 15,217 hydrological response units, was calibrated (1991-2000) and validated (2001-2010) against the monthly obse...
3 CitationsSource
#1P. Sahithya (Rajalakshmi Engineering College)
#2M. Arulmozhi (Rajalakshmi Engineering College)
Last. Nandini Praveen (Rajalakshmi Engineering College)
view all 3 authors...
Artificial Neural Network (ANN) are significantly used for fast and highly accurate computation in various fields. This paper addresses digital design for two Machine learning Algorithms, Radial Basis Function Neural Network (RBFNN) and the Long Short Term Memory Recurrent Neural Network (LSTM-RNN). The stochastic gradient descent (SGD) method is used as a learning algorithm for the former and Simultaneous Perturbation Stochastic Approximation (SPSA) method is used for the latter. The design are...