Tutorials for MATH 4432 Statistical Machine Learning, HKUST, Fall 2022

WANG Zhiwei

Last update: Dec 1, 2022

Overview

Tutorials for MATH 4432 Statistical Machine Learning

Course information

MATH 4432 Statistical Machine Learning

Instructor: Prof. YANG Can

Teaching assistant: WANG Zhiwei ([email protected])

This course is open to senior undergraduates in applied mathematics, statistics, and engineering who are interested in learning from data. It covers hot topics in statistical learning, also known as machine learning, featured with various applications.

Tutorial files

The source files of the slides are .Rmd files. If you are interested in how to create slides through R Markdown, you can have a look at them.

To get a full view of the slides, I recommend you open the .html files (e.g., Introduction.html) with your browser after downloading the entire repository. Typically this works best in Chrome.

I also provide the PDF version via John Paul Helveston and Garrick Aden-Buie's R package renderthis.

renderthis::to_pdf(from = "filename.Rmd", complex_slides = TRUE, partial_slides = FALSE)

However, the “complex” slides containing panelsets or other HTML widgets / advanced features might not render well as a PDF.

Reference

An Introduction to Statistical Learning: With Applications in R. Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani.

Acknowledgments

Slides created via Yihui Xie's R package xaringan.
Theme customized via Garrick Aden-Buie's R package xaringanthemer.
Tabbed panels created via Garrick Aden-Buie's R package xaringanExtra.

Comments

Can you plz explain why curse of dimensionality holds for KNN, but not for standard linear regression?
Slide 53 of LinearModel.pdf says that as p (# of params) increases, linear regression's MSE increases which is obvious (more noise). But I don't get the following:

Why KNN increases much more that Linear regression? I presume it means MSE of KNN increases exponentially as p increases, but why? I didn't quite understand the explanation at https://en.wikipedia.org/wiki/Curse_of_dimensionality#Nearest_neighbor_search. I found this page https://stats.stackexchange.com/questions/159070/curse-of-dimensionality-knn-classifier that sort of explains the curse of dimensionality with an example saying that you need to look further to look for closest K points & those points will be further away from your "locus". How does that make MSE ~ O(c^p)?

Why it is not the case for linear models? I would appreciate a more rigorous approach (or an intuitive one & a link for a related paper). Thank you

good first issue
opened by aorazalin 3
Is MSE estimation in assignment 1 valid?

I was a bit confused at the code snippet when doing homework 1. Estimation of the linear | KNN models' bias & variance involved putting noise on top of the response $y$ a bunch of times, training the model on that "sample" & saving the prediction to later evaluate and average out.

But from lecture notes & the wiki page it seems like both bias and the variance are expectations of some term given X (predictors) - being random. Thus, I expected X to be sampled nrep times instead of putting noise on y nrep times. I don't get where am I wrong here. Thank you.

opened by aorazalin 2

Tutorials for MATH 4432 Statistical Machine Learning, HKUST, Fall 2022

Related tags

Overview

Tutorials for MATH 4432 Statistical Machine Learning

Course information

Tutorial files

Reference

Acknowledgments

You might also like...

Math magicians is a website for all fans of mathematics. It is a Single Page App (SPA) that allows users to make simple calculations and read a random math-related quote. Build with React.js

It is a simple Math Sprint Game in which the user must agree or disagree with the answers to math problems in the quickest time possible.

"Math magicians" is a website for all fans of mathematics. It is a Single Page App (SPA) that allows users to: Make simple calculations. Read a random math-related quote.

Math magicians is a website for all fans of mathematics. It is a Single Page App (SPA) that allows users to: Make simple calculations. Read a random math-related quote.

"Math magicians" is a website for all fans of mathematics. It is a Single Page App (SPA) that allows users to make simple calculations and read random math-related quotes. Its built using react

Module 03 project: Math magicians is a website for all fans of mathematics. It is a Single Page App (SPA) that allows users to Make simple calculations and Read a random math-related quote.

Math magicians" is a website for all fans of mathematics. It is a Single Page App (SPA) that allows users to: Make simple calculations and Read a random math-related quote.

"Math magicians" is a website for all fans of mathematics. It is a Single Page App (SPA) that allows users to: Make simple calculations. Read a random math-related quote.

"Math magicians" is a website for all fans of mathematics. It is a Single Page App (SPA) that allows users to make simple calculations and read a random math-related quote.

Comments

Can you plz explain why curse of dimensionality holds for KNN, but not for standard linear regression?

Is MSE estimation in assignment 1 valid?

Owner

WANG Zhiwei

HackMIT 2022. 2nd Place in Blockchain for Society sponsored by Jump Crypto. A revolutionary web application that leverages machine learning and blockchain technology to improve the crowdsourcing experience!

An ongoing curated list of frameworks, books, articles, talks, screencasts, recordings, libraries, learning tutorials and shiny resources about Javascript Development.

Homework Assignments for Visualization for Data Science, Fall 2022, University of Utah

This is a project that is in partial fulfillment of our CSCI 318 - Programming Language Concepts class of the Fall 2022 semester in New York Institute of Technology

Examples of using various CSS-in-JS libs in Astro (repo for withastro/astro#4432)

Grupprojekt för kurserna 'Javascript med Ramverk' och 'Agil Utveckling'

Hemsida för personer i Sverige som kan och vill erbjuda boende till människor på flykt

Kurs-repo för kursen Webbserver och Databaser

"Math magicians" is a website for all fans of mathematics. It is a Single Page App (SPA) that allows users to: Make simple calculations. Read a random math-related quote.

Math magicians" is a website for all fans of mathematics. It is a Single Page App (SPA) that allows users to: Make simple calculations. Read a random math-related quote.