Introduction to
computational and biological inspired intelligent circuits (ICs)

ICs classification according to: input-output
characteristic, learning paradigms and rules, the aggregation and
topology. ICs applications: static data processing, dynamic data
processing.

Fundamentals of Linear Adaptive Systems

Normal equations in Wiener-Hopf and Yule-Walker
notations. Least-Squares (LS) method and variants: weighted LS,
regularized LS. Adaptive first and second order algorithms:
steepest-descent (SDA) and least mean square (LMS) algorithms,
recursive LS (RLS) and variants, affine projection (APA) and
exponentiated gradient algorithms. General law of adaptation.
Multi-channel extension. Convergence property and stochastic
performance evaluations, for the first and the second order
algorithms.

Human brain and biologically inspired intelligent circuits

The biological brain. McCulloch and Pitts’s
neuron model. Neuron models and classification problem. Activation
functions. Nonlinear classification functions. Definition of
artificial neural networks (ANN). ANN topologies, learning paradigms
and rules. Mathematical properties of ANN.

Supervised learning by error correction

Multi layers perceptron (MLP) and
back-propagation (BP) learning algorithm. Learning in dynamic neural
networks. Signal-flow-graph approach for gradient calculation of
adaptive dynamic systems.
Autoencoders.

Regularized networks,
Extreme Learning Machine and Reservoir
Computing

Functions Approximation and Interpolation.
Tikhonov Regularization Theory and Radial Basis Function networks.
RBF Learning. Self Regularizing Networks
Extreme Learning
Machine and Reservoir Computing
.

Statistical learning theory of Vapnik and Support Vector Machine

Loss function and risk functional. The
VC-dimension and rate convergence bounds of the learning processes.
Bias vs variance dilemma. Support vector machine (SVM) definition
and learning algorithms.

Dynamic stochastic neural networks

Hopfield networks, Boltzmann machine,
competitive learning, Kohonen’s self
organizatoin feature maps, adaptive resonance theory.

Deep neural
networks

Gradient diffusion and vanish gradient
problem.
Autoencoder and Stacked Autoencoder.
Restricted Boltzmann machine,
Convolutional
neural networks.
Neural networks for *big-data* problems.

Information theory based unsupervised learning

Principal component analysis (PCA). Independent
component analysis (ICA). Learning algorithms for PCA and ICA.
Natural gradient learning algorithms. Blind Signal Separation.
Infomax and Amari algorithms.

Soft computing and data mining

Optimization problems with noisy or unknown
performance surface. Global optimization algorithms – heuristic
approach: simulated annealing and tabu search; Darwinian approach:
genetic and evolutionary algorithms; swarm intelligence based
algorithms: ant colony optimization, particles swarm optimization.
Introduction to fuzzy logic, fuzzy set and fuzzy set operations.
Symbolic vs. linguistic knowledge representation

Applications

Static and dynamic pattern
recognition, complex dynamic system modelling, Deep Learning on Big-Data, time-series filtering and
forecasting, multi sensors data-fusion.