lesson-support-vector-machines

Support Vector Machines

---

Materials We Provide

Topic	Description	Link
Lesson	Support Vector Machines Code-Along	Link

Dataset Description: MNIST Handwritten Digits Dataset

Learning Objectives

After this lesson, students will be able to:

1. Describe linear separability.
2. Differentiate between maximal margin classifiers, support vector classifiers, and support vector machines.
3. Implement SVMs in scikit-learn.
4. Describe the effects of C and kernels on SVMs.

---

Student Requirements

Before this lesson(s), students should already be able to:

1. Describe the difference between regression and classification.
2. Evaluate the performance of regression and classification models.
3. Understand the bias-variance tradeoff.

---

Lesson Outline

Total Time: 90 minutes

Agenda

I. Support Vector Machines (60 minutes total)

• Intuition
• Maximal Margin Classifiers
• Support Vector Classifiers
• Support Vector Machines
• Kernel Trick

II. Coding (30 minutes total)

OPTIONAL: Resources for Practice and Learning

For supplemental reading material on this topic, check out the following resources:

• For an excellent resource with visuals, check out these slides from the University of Oxford.
• For a more academic resource that walks through identifying the separating hyperplane and corresponding margin, check out these Stanford notes.
• SVM documentation on SKLearn
• Visualization of different kernels with iris data SKLearn
• Hyperplane walkthrough on SKLearn
• A comprehensive user guide to support vector machines.
• A blog post tutorial of understanding the linear algebra behind SVM hyperplanes. Check part 3 of this blog on finding the optimal hyperplane
• This Quora discussion includes a high-level overview, plus a 20 minute video walking through the core “need-to-knows.”
• A slideshow introduction to the optimization considerations of SVM
• A second slideshow overview from UCF on the high notes of SVM.
• Andrew Ng’s notes on support vector machines from Stanford’s CS 229 course.
• A full lecture (1+ hour) on SVMs from Dr. Yasser. He does a follow-up on kernel tricks too.
• A full lecture (50 minutes) on SVMs from MIT Open Courseware.
• An infamous paper (cited 7000+ times!) on why SVM is a great text classifier.
• An advanced discussion of SVMs as probabilistic models.
• A quick overview of SVMs (h/t Mukul Ram, DSI-DC-5).

• YouTube Playlist on Machine Learning from Sentdex. The support vector machine videos are marked 20 - 33.