Introduction to
computational and biological inspired intelligent circuits (ICs)
ICs classification according to: input-output
characteristic, learning paradigms and rules, the aggregation and
topology. ICs applications: static data processing, dynamic data
processing.
The linear adaptive filter theory
Normal equations in Wiener-Hopf and Yule-Walker
notations. Least-Squares (LS) method and variants: weighted LS,
regularized LS. Adaptive first and second order algorithms:
steepest-descent (SDA) and least mean square (LMS) algorithms,
recursive LS (RLS) and variants, affine projection (APA) and
exponentiated gradient algorithms. General law of adaptation.
Multi-channel extension. Convergence property and stochastic
performance evaluations, for the first and the second order
algorithms.
Human brain and biologically inspired intelligent circuits
The biological brain. McCulloch and Pitts’s
neuron model. Neuron models and classification problem. Activation
functions. Nonlinear classification functions. Definition of
artificial neural networks (ANN). ANN topologies, learning paradigms
and rules. Mathematical properties of ANN.
Supervised learning by error correction
Multi layers perceptron (MLP) and
back-propagation (BP) learning algorithm. Learning in dynamic neural
networks. Signal-flow-graph approach for gradient calculation of
adaptive dynamic systems.
Regularized networks
Functions Approximation and Interpolation.
Tikhonov Regularization Theory and Radial Basis Function networks.
RBF Learning. Self Regularizing Networks.
Statistical learning theory of Vapnik and support vector machine
Loss function and risk functional. The
VC-dimension and rate convergence bounds of the learning processes.
Bias vs variance dilemma. Support vector machine (SVM) definition
and learning algorithms.
Dynamic stochastic neural networks
Hopfield networks, Boltzmann machine,
competitive learning, Kohonen’s self
organizatoin feature maps, adaptive resonance theory.
Information theory based unsupervised learning
Principal component analysis (PCA). Independent
component analysis (ICA). Learning algorithms for PCA and ICA.
Natural gradient learning algorithms. Blind Signal Separation.
Infomax and Amari algorithms.
Soft computing and data mining
Optimization problems with noisy or unknown
performance surface. Global optimization algorithms – heuristic
approach: simulated annealing and tabu search; Darwinian approach:
genetic and evolutionary algorithms; swarm intelligence based
algorithms: ant colony optimization, particles swarm optimization.
Introduction to fuzzy logic, fuzzy set and fuzzy set operations.
Symbolic vs. linguistic knowledge representation
Applications
Static and dynamic pattern
recognition, complex dynamic system modelling, time-series filtering and
forecasting, multi sensors data-fusion.
Text book
Further
books
· S. Haykin,
“Neural Networks”, MacMillan College Publishing Company, NY, 2009.
· Thomas Weise,
“Global Optimization Algorithms Theory and Applications”, University of
Kassel, http://www.it-weise.de/
· R.O. Duda e P.E.
Hart, “Pattern Classification and Scene Analysis”, J. Wiley & Sons, 1973
(MAT 68-1973-03IN, ING2 EL.0069).
· J.-S.R. Jang,
C.-T. Sun, E. Mizutani, “Neuro-Fuzzy and Soft Computing”, Prentice Hall,
1997.