In machine learning, **Support Vector Machines** are supervised learning models used for classification and regression analysis. The basic SVM takes a set of input data and predicts, for each given input, which of two possible classes forms the output, making it a non-probabilistic binary linear classifier. To classify examples, we choose the hyperplane so that the distance from it to the nearest data point on each side is maximized. If such a hyperplane exists, it is known as the maximum-margin hyperplane and the linear classifier it defines is known as a maximum margin classifier.

This figure was drawn for maximum-margin hyperplane and margins for an SVM trained with samples from two classes.

\documentclass[10pt,letterpaper]{article} \usepackage{tikz} \usetikzlibrary{arrows} \usepackage[active,tightpage,pdftex]{preview} \PreviewEnvironment{tikzpicture} \begin{document} \begin{tikzpicture}[>=stealth'] % Draw axes \draw [<->,thick] (0,5) node (yaxis) [above] {$y$} |- (5,0) node (xaxis) [right] {$x$}; % draw line \draw (0,-1) -- (5,4); % y=x-1 \draw[dashed] (-1,0) -- (4,5); % y=x+1 \draw[dashed] (2,-1) -- (6,3); % y=x-3 % \draw labels \draw (3.5,3) node[rotate=45,font=\small] {$\mathbf{w}\cdot \mathbf{x} + b = 0$}; \draw (2.5,4) node[rotate=45,font=\small] {$\mathbf{w}\cdot \mathbf{x} + b = 1$}; \draw (4.5,2) node[rotate=45,font=\small] {$\mathbf{w}\cdot \mathbf{x} + b = -1$}; % draw distance \draw[dotted] (4,5) -- (6,3); \draw (5.25,4.25) node[rotate=-45] {$\frac{2}{\Vert \mathbf{w} \Vert}$}; \draw[dotted] (0,0) -- (0.5,-0.5); \draw (0,-0.5) node[rotate=-45] {$\frac{b}{\Vert \mathbf{w} \Vert}$}; \draw[->] (2,1) -- (1.5,1.5); \draw (1.85,1.35) node[rotate=-45] {$\mathbf{w}$}; % draw negative dots \fill[red] (0.5,1.5) circle (3pt); \fill[red] (1.5,2.5) circle (3pt); \fill[black] (1,2.5) circle (3pt); \fill[black] (0.75,2) circle (3pt); \fill[black] (0.6,1.9) circle (3pt); \fill[black] (0.77, 2.5) circle (3pt); \fill[black] (1.5,3) circle (3pt); \fill[black] (1.3,3.3) circle (3pt); \fill[black] (0.6,3.2) circle (3pt); % draw positive dots \draw[red,thick] (4,1) circle (3pt); \draw[red,thick] (3.3,.3) circle (3pt); \draw[black] (4.5,1.2) circle (3pt); \draw[black] (4.5,.5) circle (3pt); \draw[black] (3.9,.7) circle (3pt); \draw[black] (5,1) circle (3pt); \draw[black] (3.5,.2) circle (3pt); \draw[black] (4,.3) circle (3pt); \end{tikzpicture} \end{document}