Sapienza - University of  Rome

DIET -  Department of Information Engineering, Electronics and Telecommunications  

 

Course Info: 

Deep Learning (old Tecniche Audiovisive)

Prof. Aurelio Uncini   

 

Information

 

Course beginning:

Febrary  - last week,  2023
Duration 13 weeks.

 

Lesson timetable

TBD .

 

 


SW Tools

 

 

 

 

Motivation

This course introduces the Deep Neural Networks (DNN) architectures, the relative Deep Learning Algorithms (DLAs), from now denoted to as Deep Learning (DL) and their (main) applications

 

Objectives of the Course

The educational objectives include the acquisition of the following skills: 1) knowledge and understanding of the problems related to the use of DL; 2) the ability to apply knowledge on DL in the most common problems described in the course (knowledge and know-how), 3) development of independent judgment regarding the possible optimal solution with DL of a given problem, 4) the development of communication skills on the topics covered in the course, 5) the ability to autonomous learning on specialized texts.  

 

Final examination

The exam consists of a discussion of an assigned  project or home-work (max 24pt) and some theoretical questions (max 6pt). 

 

The home-work is assigned to the student in the last week of the course, and typically the student can choose the project from a list of possible topics.

 

The project  can also be done by a group of maximum 3 students. In this case the task of each individual student must be well specified.


The final discussion of the project is done at the teacher's office by appointment (at any period of the year, and usually Tuesday afternoons). For the final exam the student must: 1) present a report (in the form of a short scientific paper) on the project carried out; 2) give a short presentation  in which the results and the acquired skills are highlighted.

 


Course syllabus (Tentative)

 

1.         Introduction and Prerequisites:

a.         Rewiev of Supervised and Unsupervised Learning

b.         Stochastic optimization algorithms

c.         Regression and Classification Performance Metric

d.         Lab session: Python and NumPy

 

2.         Deep Neural Networks

a.         End-to-End Problems and DNNs’ Architectures

b.         Backpropagation, BP-variants and Automatic Differentiation

c.         Mechanisms of regularization

d.         Lab session: introduction to TensorFlow

 

3.         Deep Recurrent Neural Networks

a.         Elman Models

b.         Convolutional NNs

c.         Long Short-Time Memory and Gated NNs

d.         Lab session: TensorFlow prediction and filtering

 

4.         Generative models

a.         Autoencoder (AE), variational AE.

b.         Generative Adversarial Networks (GAN), Cycle GAN.

c.         Deep Convolutional GANs.

d.         Lab session: generative models.

 

5.         Graph Neural Networks

a.         Definition of GNNs

b.         Deep Learning for GNNs

c.         Graph Convolutional NNs

d.         Lab session: GNNs.

 

6.         DL's Main Applications

a.         Computer vision.

b.         Sequence and text analysis.

c.         Audio and Speech Signal

d.         Environment, Energy and Smart Grid

e.         ......

f.         Lab sessions.

 

 



References

 

Text books and papers

  • A. Uncini, Introduction to Neural Networks and Deep Learning, Lecture notes + slides - ed. 2022.

  • A. Uncini, Mathematical Elements for Machine Learning, Lecture notes + slides - ed.  2022.

  • Aston Zhang, Zack C. Lipton, Mu Li, Alex J. Smola, “Dive into Deep Learning,” CoRR abs/2106.11342 (2021)

     

 Other other recommended text and papers

 

Further reading books

  • S. Haykin, “Neural Networks”, MacMillan College Publishing Company, NY, 2009.

  • Thomas Weise, “Global Optimization Algorithms Theory and Applications”, University of Kassel, http://www.it-weise.de/

  • R.O. Duda e P.E. Hart, “Pattern Classification and Scene Analysis”, J. Wiley & Sons, 1973 (MAT 68-1973-03IN, ING2 EL.0069).

  • J.-S.R. Jang, C.-T. Sun, E. Mizutani, “Neuro-Fuzzy and Soft Computing”, Prentice Hall, 1997.

  • A. Uncini, Fundamentals of Adaptive Signal Processing - Springer, Febrary 2015.