Introduction to Machine Learning & Deep Learning

Fall 2019, Class: Friday & Saturday from 8:30 AM to 12:30 PM

Description:

The term machine learning or statistical learning refers to the science of automated detection of patterns in data. It has been widely used in tasks that require information extraction from large data sets. Examples of tasks include SPAM detection, fraudulent credit card transaction detection, face recognition by digital cameras, and voice commands recognition by personal assistance on smart-phones. Machine learning is also widely used in scientific domains such as Bioinformatics, medicine, and astronomy. One characteristic of all these applications is that a human developer cannot provide an explicit and detailed specification of how these tasks should be executed, due to the complexity of the patterns that need to be detected.

Learning Objectives:

This is a undergraduate-level course. It provides a thorough grounding in the methods, techniques, and algorithms of machine learning. In the end of this course, the students should be able to describe the main concepts underlying machine learning, including for instance:

what is learning
how can a machine learning
what kind of problems can be solved by using machine learning approaches
how to formalize them as a machine learning problem, and
how to compare and evaluate the performance of different machine learning
apply machine learning methods into different use cases, using Python, Pandas, scikit-learn, among others

Format

The course is a combination of theoretical and practical sessions. The theory part discusses the fundamentals of the topics required to understand and design machine learning algorithms.

This practical one implements the discussed machine learning methods using Python 3 as the programming languages and Jupyter notebook as the computing environment.

All the Jupyter notebooks are available in the GitHub repository of the course.

Grading scheme

A practical session that will realized in groups of two students.
One individual project used for the personal evaluation at the end of the course. It must be an IoT use case.

Textbooks

Hastie, T., Tibshirani, R., and Friedman, J. (2016). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, 2nd edition
Daumé III, H. (2017). A Course in Machine Learning. Self-published, 2nd edition
Murphy, K. P. (2012). Machine learning: a probabilistic perspective. MIT press.
Bishop, C. M. (2006). Pattern recognition and machine learning. Springer.
James, G., Witten, D., Hastie, T., and Tibshirani, R. (2013). An introduction to statistical learning. Springer
Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep learning. MIT press.
Valiant, L. (2013). Probably Approximately Correct: Nature’s Algorithms for Learning and Prospering in a Complex World. Basic Books, Inc.

Timeline

Date	Lecture	Homeworks	Notes
Week 1 Friday, October 4th	Lecture Course introduction, problem definitions, applications	[HW01]	Reading Pedro Domingos. A Few Useful Things to Know About Machine Learning. In: Communication of the ACM 55.10 (2012), pp. 78–87. D. Sculley et al. Hidden Technical Debt in Machine Learning Systems. In: 28th International Conference on Neural Information Processing Systems. 2015, pp. 2503–2511 Machine Learning for Everyone Amershi, Saleema, Andrew Begel, Christian Bird, Robert DeLine, Harald Gall, Ece Kamar, Nachiappan Nagappan, Besmira Nushi, and Thomas Zimmermann. Software engineering for machine learning: a case study. In 41st International Conference on Software Engineering: Software Engineering in Practice, pp. 291-300. 2019.
Week 2 Friday, October 11th	Lecture A Crash Course in Python	[HW02]
Week 3 Friday, October 18th	Lecture Scientific Computing with NumPy, Pandas, and Matplotlib	[HW03]	Reading Reproducible model training: deep dive Olorisade, Babatunde K., Pearl Brereton, and Peter Andras. Reproducibility in machine Learning-Based studies: An example of text mining. 2017.
Week 4 Friday, October 25th	Lecture Regression: predicting house prices Introduction Regression, gradient descent Assessing performance, error types, and bias/variance trade-off Exploratory data analysis (EDA)	[HW03]
Week 5 Friday, November 8th	Lecture Regression Overfitting, regularized regression, ridge regression, lasso Cross-validation Classification Introduction Logistic regression
Week 5 Saturday, November 9th	Lecture Tree-based methods decision trees overfitting in decision trees Precision, recall, and ROC curve Ensemble methods Boosting Bagging Random Forests		Reading Quinlan, J. Ross. Induction of decision trees]. Machine Learning 1, no. 1 (1986): 81-106 Breiman, Leo. Random forests. Machine Learning 45, no. 1. pp. 5-32, 2001 Schapire, Robert E. The strength of weak learnability. Machine learning 5, no. 2 pp. 197-227, 1990
Week 6 Saturday, November 16th	Lecture Dimensionality reduction Feature selection and extraction Principal component analysis (PCA) t-SNE		Reading Kuhn M., Johnson K. An Introduction to Feature Selection. In: Applied Predictive Modeling. Springer, New York, NY, 2013. Guyon, Isabelle, and André Elisseeff. An introduction to variable and feature selection. Journal of machine learning research 3, no. Mar (2003): 1157-1182. Shlens, Jonathon. A tutorial on principal component analysis. arXiv preprint arXiv:1404.1100, 2014. Linear Dimensionality Reduction Visualizing data using t-SNE How to Use t-SNE Effectively sklearn.decomposition.PCA
Week 7 Friday, November 22th	Lecture Clustering and similarity Introduction to clustering Nearest neighbors methods k-means clustering
Week 7 Saturday, November 23th	Reading Application of machine learning to estimate energy performance of residential buildings. Tsanas A, Xifara A. Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools. Energy and Buildings. 2012 Jun 1;49:560-7. Johannesen, Nils Jakob, Kolhe, Mohan, et Goodwin, Morten. Relative evaluation of regression tools for urban area electrical energy demand forecasting. Journal of cleaner production, 2019, vol. 218, p. 555-564.		Due[CW04] (Deadline at November 23th, 12:30 PM GMT+1) Submit your analysis (i.e., Jupyter notebook) through the following form. Submission form
Week 8 Saturday, November 30th	Lecture Neural networks Perceptron Multilayer Perceptron Support Vector Machines (SVM)		Reading Samira Pouyanfar, Saad Sadiq, Yilin Yan, Haiman Tian, Yudong Tao, Maria Presa Reyes, Mei-Ling Shyu, Shu-Ching Chen, and S. S. Iyengar. A Survey on Deep Learning: Algorithms, Techniques, and Applications. ACM Computing Survey 51, 5. 2018 Rieck, Konrad, Sören Sonnenburg, Sebastian Mika, Christin Schäfer, Pavel Laskov, David Tax, and Klaus-Robert Müller. Support vector machines. In Handbook of computational statistics, pp. 883-926. Springer, Berlin, Heidelberg. 2012
Week 9 Saturday, December 14th	Lecture Much of the knowledge required to successfully develop machine learning applications is not readily available in books. Thus, knowing the pitfalls to avoid and the important issues to focus on is necessary when conducting a machine learning project. Question & answers about the project Pedro Domingos.A Few Useful Things to Know About Machine Learning. In: Communication of the ACM 55.10 (2012), pp. 78–87.		Due[CW05] (Deadline at December 14th, 12:30 PM GMT+1) Submit your summary of the text through the following submission form. Submission form
Week 10 Tuesday, December 17th	Due Presentation of the projects Structure and template LaTeX of the report [PDF] Everyone will have ten minutes to present the work Please submit your project here Deadline: December 16th, 2019 2:00 PM Paris Time (GMT +1)

Introduction to Machine Learning & Deep Learning

Fall 2019, Class: Friday & Saturday from 8:30 AM to 12:30 PM

Description:

Learning Objectives:

Format

Grading scheme

Textbooks

Timeline

Resources

Dataset repositories

Cheat Sheets

Documentations

Online Jupyter environment

Newsletters

Podcasts

Contact