computational and biological inspired intelligent circuits (ICs)
ICs classification according to: input-output
characteristic, learning paradigms and rules, the aggregation and
topology. ICs applications: static data processing, dynamic data
Fundamentals of Linear Adaptive Systems
Normal equations in Wiener-Hopf and Yule-Walker
notations. Least-Squares (LS) method and variants: weighted LS,
regularized LS. Adaptive first and second order algorithms:
steepest-descent (SDA) and least mean square (LMS) algorithms,
recursive LS (RLS) and variants, affine projection (APA) and
exponentiated gradient algorithms. General law of adaptation.
Multi-channel extension. Convergence property and stochastic
performance evaluations, for the first and the second order
Human brain and biologically inspired intelligent circuits
The biological brain. McCulloch and Pitts’s
neuron model. Neuron models and classification problem. Activation
functions. Nonlinear classification functions. Definition of
artificial neural networks (ANN). ANN topologies, learning paradigms
and rules. Mathematical properties of ANN.
Supervised learning by error correction
Multi layers perceptron (MLP) and
back-propagation (BP) learning algorithm. Learning in dynamic neural
networks. Signal-flow-graph approach for gradient calculation of
adaptive dynamic systems.
Extreme Learning Machine and Reservoir
Functions Approximation and Interpolation.
Tikhonov Regularization Theory and Radial Basis Function networks.
RBF Learning. Self Regularizing Networks
Machine and Reservoir Computing
Statistical learning theory of Vapnik and Support Vector Machine
Loss function and risk functional. The
VC-dimension and rate convergence bounds of the learning processes.
Bias vs variance dilemma. Support vector machine (SVM) definition
and learning algorithms.
Dynamic stochastic neural networks
Hopfield networks, Boltzmann machine,
competitive learning, Kohonen’s self
organizatoin feature maps, adaptive resonance theory.
Gradient diffusion and vanish gradient
Autoencoder and Stacked Autoencoder.
Restricted Boltzmann machine,
Neural networks for big-data problems.
Information theory based unsupervised learning
Principal component analysis (PCA). Independent
component analysis (ICA). Learning algorithms for PCA and ICA.
Natural gradient learning algorithms. Blind Signal Separation.
Infomax and Amari algorithms.
Soft computing and data mining
Optimization problems with noisy or unknown
performance surface. Global optimization algorithms – heuristic
approach: simulated annealing and tabu search; Darwinian approach:
genetic and evolutionary algorithms; swarm intelligence based
algorithms: ant colony optimization, particles swarm optimization.
Introduction to fuzzy logic, fuzzy set and fuzzy set operations.
Symbolic vs. linguistic knowledge representation
Static and dynamic pattern
recognition, complex dynamic system modelling, Deep Learning on Big-Data, time-series filtering and
forecasting, multi sensors data-fusion.