Tikz example – SVM trained with samples from two classes

By | September 15, 2013

In machine learning, Support Vector Machines are supervised learning models used for classification and regression analysis. The basic SVM takes a set of input data and predicts, for each given input, which of two possible classes forms the output, making it a non-probabilistic binary linear classifier. To classify examples, we choose the hyperplane so that the distance from it to the nearest data point on each side is maximized. If such a hyperplane exists, it is known as the maximum-margin hyperplane and the linear classifier it defines is known as a maximum margin classifier.

This figure was drawn for maximum-margin hyperplane and margins for an SVM trained with samples from two classes.

\documentclass[10pt,letterpaper]{article}

\usepackage{tikz}
\usetikzlibrary{arrows}

\usepackage[active,tightpage,pdftex]{preview}
\PreviewEnvironment{tikzpicture}

\begin{document}
\begin{tikzpicture}[>=stealth']
% Draw axes
\draw [<->,thick] (0,5) node (yaxis) [above] {$y$}
|- (5,0) node (xaxis) [right] {$x$};
% draw line
\draw (0,-1) -- (5,4); % y=x-1
\draw[dashed] (-1,0) -- (4,5); % y=x+1
\draw[dashed] (2,-1) -- (6,3); % y=x-3
% \draw labels
\draw (3.5,3) node[rotate=45,font=\small]
{$\mathbf{w}\cdot \mathbf{x} + b = 0$};
\draw (2.5,4) node[rotate=45,font=\small]
{$\mathbf{w}\cdot \mathbf{x} + b = 1$};
\draw (4.5,2) node[rotate=45,font=\small]
{$\mathbf{w}\cdot \mathbf{x} + b = -1$};
% draw distance
\draw[dotted] (4,5) -- (6,3);
\draw (5.25,4.25) node[rotate=-45] {$\frac{2}{\Vert \mathbf{w} \Vert}$};
\draw[dotted] (0,0) -- (0.5,-0.5);
\draw (0,-0.5) node[rotate=-45] {$\frac{b}{\Vert \mathbf{w} \Vert}$};
\draw[->] (2,1) -- (1.5,1.5);
\draw (1.85,1.35) node[rotate=-45] {$\mathbf{w}$};
% draw negative dots
\fill[red] (0.5,1.5) circle (3pt);
\fill[red]   (1.5,2.5)   circle (3pt);
\fill[black] (1,2.5)     circle (3pt);
\fill[black] (0.75,2)    circle (3pt);
\fill[black] (0.6,1.9)   circle (3pt);
\fill[black] (0.77, 2.5) circle (3pt);
\fill[black] (1.5,3)     circle (3pt);
\fill[black] (1.3,3.3)   circle (3pt);
\fill[black] (0.6,3.2)   circle (3pt);
% draw positive dots
\draw[red,thick] (4,1)     circle (3pt);
\draw[red,thick] (3.3,.3)  circle (3pt);
\draw[black]     (4.5,1.2) circle (3pt);
\draw[black]     (4.5,.5)  circle (3pt);
\draw[black]     (3.9,.7)  circle (3pt);
\draw[black]     (5,1)     circle (3pt);
\draw[black]     (3.5,.2)  circle (3pt);
\draw[black]     (4,.3)    circle (3pt);
\end{tikzpicture}
\end{document}

This site uses Akismet to reduce spam. Learn how your comment data is processed.