2018-03-21 17:37:55 +01:00
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-12-05 12:27:39 +01:00
"# Workshop: Machine Learning for Beginners"
2018-03-21 17:37:55 +01:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-12-05 12:27:39 +01:00
"## What is Machine Learning"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Machine learning is the process of **extracting knowledge from data** in an automated fashion.\n",
2018-03-21 17:37:55 +01:00
"\n",
2019-12-05 12:27:39 +01:00
"The use cases usually are making predictions on new and unseen data or simply understanding a given dataset better by finding patterns.\n",
2018-03-21 17:37:55 +01:00
"\n",
2019-12-05 12:27:39 +01:00
"Central to machine learning is the idea of **automating** the **decision making** from data **without** the user specifying **explicit rules** how these decisions should be made."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-10-02 09:06:44 +02:00
"<img src=\"raw/what_is_machine_learning.png\" width=\"60%\">"
2018-03-21 17:37:55 +01:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-12-05 12:27:39 +01:00
"### Examples"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-10-02 09:06:44 +02:00
"<img src=\"raw/examples.png\" width=\"60%\">"
2018-03-21 17:37:55 +01:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-12-05 12:27:39 +01:00
"## Types of Machine Learning"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img src=\"raw/3_types_of_machine_learning.png\" width=\"60%\">"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"- **Supervised** (focus of this workshop): Each entry in the dataset comes with a \"label\". Examples are a list of emails where spam mail is already marked as such or a sample of handwritten digits. The goal is to use the historic data to make predictions.\n",
2018-03-21 17:37:55 +01:00
"\n",
"- **Unsupervised**: There is no desired output associated with a data entry. In a sense, one can think of unsupervised learning as a means of discovering labels from the data itself. A popular example is the clustering of customer data.\n",
"\n",
"- **Reinforcement**: Conceptually, this can be seen as \"learning by doing\". Some kind of \"reward function\" tells how good a predicted outcome is. For example, chess computers are typically programmed with this approach."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-12-05 12:27:39 +01:00
"### Types of Supervised Learning"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img src=\"raw/classification_vs_regression.png\" width=\"60%\">"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"- In **classification** tasks, the labels are *discrete*, such as \"spam\" or \"no spam\" for emails. Often, labels are nominal (e.g., colors of something), or ordinal (e.g., T-shirt sizes in S, M, or L).\n",
"- In **regression**, the labels are *continuous*. For example, given a person's age, education, and position, infer his/her salary."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Case Study: Iris Flower Classification"
2018-03-21 17:37:55 +01:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-10-02 09:06:44 +02:00
"<img src=\"raw/iris_data.png\" width=\"60%\">"
2018-03-21 17:37:55 +01:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-12-05 12:27:39 +01:00
"## Python for Scientific Computing: A brief Introduction"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Python itself does not come with any scientific algorithms. However, over time, many open source libraries emerged that are useful to build machine learning applications.\n",
2018-03-21 17:37:55 +01:00
"\n",
2019-12-05 12:27:39 +01:00
"Among the popular ones are [numpy](https://numpy.org/) (numerical computations, linear algebra), [pandas](https://pandas.pydata.org/) (data processing), [matplotlib](https://matplotlib.org/) (visualisations), and [scikit-learn](https://scikit-learn.org/stable/index.html) (machine learning algorithms).\n",
2018-03-21 17:37:55 +01:00
"\n",
"First, import the libraries:"
]
},
{
"cell_type": "code",
2018-03-21 17:42:59 +01:00
"execution_count": 1,
2018-03-21 17:37:55 +01:00
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import pandas as pd\n",
"import matplotlib.pyplot as plt"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The following line is needed so that this Jupyter notebook creates the visiualizations in the notebook and not in a new window. This has nothing to do with Python."
]
},
{
"cell_type": "code",
2018-03-21 17:42:59 +01:00
"execution_count": 2,
2018-03-21 17:37:55 +01:00
"metadata": {},
"outputs": [],
"source": [
2018-12-05 13:28:52 +01:00
"%matplotlib inline"
2018-03-21 17:37:55 +01:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Standard Python can do basic arithmetic operations ..."
]
},
{
"cell_type": "code",
2018-03-21 17:42:59 +01:00
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"3"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"a = 1\n",
"b = 2\n",
2019-12-05 12:27:39 +01:00
"\n",
2018-03-21 17:37:55 +01:00
"c = a + b\n",
2019-12-05 12:27:39 +01:00
"\n",
2018-03-21 17:37:55 +01:00
"c"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"... and provides some simple **data structures**, such as a list of values."
]
},
{
"cell_type": "code",
2018-03-21 17:42:59 +01:00
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[1, 2, 3, 4]"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"l = [a, b, c, 4]\n",
2019-12-05 12:27:39 +01:00
"\n",
2018-03-21 17:37:55 +01:00
"l"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-12-05 12:27:39 +01:00
"Numpy provides a data structure called an **n-dimensional array**. This may sound fancy at first but when used with only 1 or 2 dimensions, it basically represents vectors and matrices. Arrays allow for much faster computations as they are implemented in the very fast [C language](https://en.wikipedia.org/wiki/C_%28programming_language%29)."
2018-03-21 17:37:55 +01:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-12-05 12:27:39 +01:00
"To create an array, we use the [array()](https://docs.scipy.org/doc/numpy/reference/generated/numpy.array.html#numpy-array) function from the imported `np` module and provide it with a `list` of values."
2018-03-21 17:37:55 +01:00
]
},
{
"cell_type": "code",
2018-03-21 17:42:59 +01:00
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([1, 2, 3])"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"v1 = np.array([1, 2, 3])\n",
2019-12-05 12:27:39 +01:00
"\n",
2018-03-21 17:37:55 +01:00
"v1"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"A vector can be multiplied with a scalar."
]
},
{
"cell_type": "code",
2018-03-21 17:42:59 +01:00
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([3, 6, 9])"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"v2 = v1 * 3\n",
2019-12-05 12:27:39 +01:00
"\n",
2018-03-21 17:37:55 +01:00
"v2"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To create a matrix, just use a list of (row) list of values instead."
]
},
{
"cell_type": "code",
2018-03-21 17:42:59 +01:00
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[1, 2, 3],\n",
" [4, 5, 6]])"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"m1 = np.array([\n",
" [1, 2, 3],\n",
" [4, 5, 6],\n",
"])\n",
2019-12-05 12:27:39 +01:00
"\n",
2018-03-21 17:37:55 +01:00
"m1"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we can use numpy to multiply a matrix with a vector to obtain a new vector ..."
]
},
{
"cell_type": "code",
2018-03-21 17:42:59 +01:00
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([14, 32])"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"v3 = np.dot(m1, v1)\n",
2019-12-05 12:27:39 +01:00
"\n",
2018-03-21 17:37:55 +01:00
"v3"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"... or simply transpose it."
]
},
{
"cell_type": "code",
2018-03-21 17:42:59 +01:00
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[1, 4],\n",
" [2, 5],\n",
" [3, 6]])"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"m1.T"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The rules from maths still apply and it makes a difference if a vector is multiplied from the left or the right by a matrix. The following operation will fail."
]
},
{
"cell_type": "code",
2018-03-21 17:42:59 +01:00
"execution_count": 10,
"metadata": {},
"outputs": [
{
"ename": "ValueError",
"evalue": "shapes (3,) and (2,3) not aligned: 3 (dim 0) != 2 (dim 0)",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mValueError\u001b[0m Traceback (most recent call last)",
2018-12-05 13:28:52 +01:00
"\u001b[0;32m<ipython-input-10-c170f5d663e1>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mv1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mm1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
2019-10-02 09:06:44 +02:00
"\u001b[0;32m<__array_function__ internals>\u001b[0m in \u001b[0;36mdot\u001b[0;34m(*args, **kwargs)\u001b[0m\n",
2018-03-21 17:42:59 +01:00
"\u001b[0;31mValueError\u001b[0m: shapes (3,) and (2,3) not aligned: 3 (dim 0) != 2 (dim 0)"
]
}
],
2018-03-21 17:37:55 +01:00
"source": [
"np.dot(v1, m1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In order to retrieve only a slice (= subset) of an array's data, we can \"index\" into it. For example, the first row of the matrix is ..."
]
},
{
"cell_type": "code",
2018-03-21 17:42:59 +01:00
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([1, 2, 3])"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"m1[0, :]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"... while the second column is:"
]
},
{
"cell_type": "code",
2018-03-21 17:42:59 +01:00
"execution_count": 12,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([2, 5])"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"m1[:, 1]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To acces the lowest element in the right column, two indices can be used."
]
},
{
"cell_type": "code",
2018-03-21 17:42:59 +01:00
"execution_count": 13,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"6"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"m1[1, 2]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Numpy also provides various other functions and constants, such as sinus or pi. To further illustrate the concept of **vectorization**, let us calculate the sinus curve over a range of values."
]
},
{
"cell_type": "code",
2018-03-21 17:42:59 +01:00
"execution_count": 14,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([-9.42477796, -9.23437841, -9.04397885, -8.8535793 , -8.66317974,\n",
" -8.47278019, -8.28238063, -8.09198108, -7.90158152, -7.71118197,\n",
" -7.52078241, -7.33038286, -7.1399833 , -6.94958375, -6.75918419,\n",
" -6.56878464, -6.37838508, -6.18798553, -5.99758598, -5.80718642,\n",
" -5.61678687, -5.42638731, -5.23598776, -5.0455882 , -4.85518865,\n",
" -4.66478909, -4.47438954, -4.28398998, -4.09359043, -3.90319087,\n",
" -3.71279132, -3.52239176, -3.33199221, -3.14159265, -2.9511931 ,\n",
" -2.76079354, -2.57039399, -2.37999443, -2.18959488, -1.99919533,\n",
" -1.80879577, -1.61839622, -1.42799666, -1.23759711, -1.04719755,\n",
" -0.856798 , -0.66639844, -0.47599889, -0.28559933, -0.09519978,\n",
" 0.09519978, 0.28559933, 0.47599889, 0.66639844, 0.856798 ,\n",
" 1.04719755, 1.23759711, 1.42799666, 1.61839622, 1.80879577,\n",
" 1.99919533, 2.18959488, 2.37999443, 2.57039399, 2.76079354,\n",
" 2.9511931 , 3.14159265, 3.33199221, 3.52239176, 3.71279132,\n",
" 3.90319087, 4.09359043, 4.28398998, 4.47438954, 4.66478909,\n",
" 4.85518865, 5.0455882 , 5.23598776, 5.42638731, 5.61678687,\n",
" 5.80718642, 5.99758598, 6.18798553, 6.37838508, 6.56878464,\n",
" 6.75918419, 6.94958375, 7.1399833 , 7.33038286, 7.52078241,\n",
" 7.71118197, 7.90158152, 8.09198108, 8.28238063, 8.47278019,\n",
" 8.66317974, 8.8535793 , 9.04397885, 9.23437841, 9.42477796])"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"x = np.linspace(-3*np.pi, 3*np.pi, 100)\n",
2019-12-05 12:27:39 +01:00
"\n",
2018-03-21 17:37:55 +01:00
"x"
]
},
{
"cell_type": "code",
2018-03-21 17:42:59 +01:00
"execution_count": 15,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([-3.67394040e-16, -1.89251244e-01, -3.71662456e-01, -5.40640817e-01,\n",
" -6.90079011e-01, -8.14575952e-01, -9.09631995e-01, -9.71811568e-01,\n",
" -9.98867339e-01, -9.89821442e-01, -9.45000819e-01, -8.66025404e-01,\n",
" -7.55749574e-01, -6.18158986e-01, -4.58226522e-01, -2.81732557e-01,\n",
" -9.50560433e-02, 9.50560433e-02, 2.81732557e-01, 4.58226522e-01,\n",
" 6.18158986e-01, 7.55749574e-01, 8.66025404e-01, 9.45000819e-01,\n",
" 9.89821442e-01, 9.98867339e-01, 9.71811568e-01, 9.09631995e-01,\n",
" 8.14575952e-01, 6.90079011e-01, 5.40640817e-01, 3.71662456e-01,\n",
" 1.89251244e-01, -1.22464680e-16, -1.89251244e-01, -3.71662456e-01,\n",
" -5.40640817e-01, -6.90079011e-01, -8.14575952e-01, -9.09631995e-01,\n",
" -9.71811568e-01, -9.98867339e-01, -9.89821442e-01, -9.45000819e-01,\n",
" -8.66025404e-01, -7.55749574e-01, -6.18158986e-01, -4.58226522e-01,\n",
" -2.81732557e-01, -9.50560433e-02, 9.50560433e-02, 2.81732557e-01,\n",
" 4.58226522e-01, 6.18158986e-01, 7.55749574e-01, 8.66025404e-01,\n",
" 9.45000819e-01, 9.89821442e-01, 9.98867339e-01, 9.71811568e-01,\n",
" 9.09631995e-01, 8.14575952e-01, 6.90079011e-01, 5.40640817e-01,\n",
" 3.71662456e-01, 1.89251244e-01, 1.22464680e-16, -1.89251244e-01,\n",
" -3.71662456e-01, -5.40640817e-01, -6.90079011e-01, -8.14575952e-01,\n",
" -9.09631995e-01, -9.71811568e-01, -9.98867339e-01, -9.89821442e-01,\n",
" -9.45000819e-01, -8.66025404e-01, -7.55749574e-01, -6.18158986e-01,\n",
" -4.58226522e-01, -2.81732557e-01, -9.50560433e-02, 9.50560433e-02,\n",
" 2.81732557e-01, 4.58226522e-01, 6.18158986e-01, 7.55749574e-01,\n",
" 8.66025404e-01, 9.45000819e-01, 9.89821442e-01, 9.98867339e-01,\n",
" 9.71811568e-01, 9.09631995e-01, 8.14575952e-01, 6.90079011e-01,\n",
" 5.40640817e-01, 3.71662456e-01, 1.89251244e-01, 3.67394040e-16])"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"y = np.sin(x)\n",
2019-12-05 12:27:39 +01:00
"\n",
2018-03-21 17:37:55 +01:00
"y"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-12-05 12:27:39 +01:00
"With matplotlib's [plot()](https://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.plot) function we can visualize the sinus curve."
2018-03-21 17:37:55 +01:00
]
},
{
"cell_type": "code",
2018-03-21 17:42:59 +01:00
"execution_count": 16,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2019-12-05 12:27:39 +01:00
"[<matplotlib.lines.Line2D at 0x7ff8934ee2e8>]"
2018-03-21 17:42:59 +01:00
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
2019-12-05 12:27:39 +01:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAD4CAYAAAAHHSreAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO29d3hc13Xo+1sYdGDQKwk2kADYSUkURTdJlmRZtPMk90h5eZFj++o6ieMb5yaO9Pk9x5/T5OTFzkuuHVtxkx1Hkq246DqUZUmW6CJTYjELQKKxohDAoM+gY2a/P8454AgESAAzc8rM/n3ffJg5deGUvfYqe21RSqHRaDQaTTRpTgug0Wg0GvehlYNGo9ForkIrB41Go9FchVYOGo1Go7kKrRw0Go1GcxXpTguwEsrKytT69eudFkOj0Wg8xdGjR/uVUuVL2daTymH9+vUcOXLEaTE0Go3GU4jIxaVuq91KGo1Go7kKrRw0Go1GcxVaOWg0Go3mKrRy0Gg0Gs1VaOWg0Wg0mquIi3IQka+LSJ+INC6yXkTkn0WkXUROisiNUeseFJE28/NgPOTRaDQaTWzEy3L4JnDPNdbvB+rMz0PAvwKISAnwl8AtwF7gL0WkOE4yaTQajWaFxEU5KKV+DgxeY5P7gG8pg0NAkYhUA28HnldKDSqlhoDnubaS0djIpYFx/uPVS5zsHHZaFI3JxHSYl5r7eOrwJaZnI06LowGUUrT3Bfn2oYu09gadFidu2DUIbjXQEfW701y22PKrEJGHMKwO1q5dmxgpNQA89vOzPHm4g3OBMQAyfMLn3ruT99xY47BkqUtrb5C/O3CGV84OMGUqhR8d7+Zff/cmCnMyHJYuNYlEFF94oZUfHu+iY3ACgJwMH//rd27gzi2VDksXO54JSCulHlNK7VFK7SkvX9Lob80K+N8nuvnbA82U5mXy6d/ayn99/M3cvL6EP/3uCf75xTb05FD2MzEd5qPfPsrxjmEe2LuWb31oL3//3p28dn6Q93/5FbqGJ5wWMSV5/NcX+JeftbO+NI+/ftd2fvzHb2ZTRT7/7VtH+PahJQ9Edi12WQ5dwJqo3zXmsi7g9nnLX7ZJJs08ekYm+b9/2MjuNUU88d/2ke4z+g7f/P29PPyfJ/n8862MTc/yyP4tDkuaWjz67BnO9Y/xHx+5hTduKptbXlOcw3//96O8+4u/4scffzMV/mwHpUwt2vuCPPpsM3dsruBrD+5BRAB48qF9fPyJ3/D//LCRqZkwH3lLrcOSrhy7LIdngN8zs5b2ASNKqcvAc8DdIlJsBqLvNpdpbEYpxZ8/fYKp2TCf/8CuOcUAkJmexj9+YBfvu6mGr/3iPB2D4w5Kmlr8vDXA47++yIfetOF1igHgjZvKePKhfQyMTfOVg+cckjD1mAlH+MRTJ8jN9PHoe3fMKQaAvKx0vvJ/3cQdmyv4pxfaGB6fdlDS2IhXKusTwK+BBhHpFJEPi8hHReSj5iYHgHNAO/BvwB8CKKUGgb8CDpufz5rLNDbz7UMX+UVbP59651Zqy/OvWi8i/NndDaSJ8KWX2x2QMPUYHp/mz58+QV1FPp+8p2HBbbatKuTdN6zm3w9dpC84abOEqcm/vNjGqa4R/u49Oxa01tJ9afzFPZsJTc3ytV+ed0DC+BCvbKUHlFLVSqkMpVSNUuprSqkvK6W+bK5XSqk/UkptVErtUEodidr360qpTebnG/GQR7M8RsZn+LsDzdxaX87v3rJ4sL+qMJsH9q7he0c6tfVgA//fi20MhKb5wm/vJjvDt+h2H3vrJmYjSlsPNtAxOM4XXz7Le25czT3bqxfdrqHKzzt3VPONX13wrPXgmYC0JnH88HgXEzNhPvn2hteZyAvxB7dv0taDDUzOhPnPo528Y0c121cXXnPb9WV52nqwie8e6SCiFH9298KWXDQfv7PO09aDVg4pjlKKJ167xLZVBddthEBbD3bxk8YeRidnuf/mNdffGG092MFsOML3jnRyW305q4pyrru9160HrRxSnJOdIzT3BLl/79LHjmjrIfE88dol1pXmsq+2dEnba+sh8RxsDdAzOsn9Ny/9XfGy9aCVQ4rz5OFL5GT4uG/3qiXvU1WYzbtuWMUzx7uZnAknULrU5FwgxKvnB/ntm9eQlnZtN180H72tlqnZCD8+cTmB0qUuT7zWQVl+FnduqVjyPg1Vfu7YXMHTRzuJRLw1RkgrhxRmbGqWZ453886d1RRkL2+U7W/tXMXYdJiftwYSJF3q8tThDnxpwvuWOSJ9U4WfzVV+nm3UyiHe9I5O8lJLH++7qYYM3/Kazd/aWc3lkUlOeKwMjVYOKcyPT3YzNh3mgb1L82tH84aNpRTmZPCTxp4ESJa6TM9G+M9jndy5uYKKguUPatu/vZojF4foG9WupXjy9NFOwhG15BhQNHduqSTDJ557V7RySGGeeK2DTRX53Lh2+YVwM3xpvG1rJc+f6WVqVruW4sWLZ3rpD03zwDJiQNHs31GFUvBck7caIjcTiSiePHyJN9SWsr4sb9n7F+Zk8KZNZRxovOyp8jNaOaQolwbGOd4xzG/vWXPd9NXFeMeOKoKTs7zSPhBn6VKXHx3vprIgi1vrV1Y/rK4in43leRw4pZVDvPhNxzAdgxN84OaVF558x/ZqOgYnaOoejaNkiUUrhxTlYGsfAHdtXXn1yDdtKsOfla593HFiJhzhV+393LG5At8yAtHRiAjv2FHNq+cHGAhNxVnC1ORgSx9pAm9tWHogej5v21qJL004cMo774pWDinKwdYAa0tyWV+au+JjZKX7uHNLBT893ctMWM8tECvHLg4RnJrltvqVN0IA92yvIqLgp6d74yRZanOwNcDuNUUU5Wau+BjFeZm8obaUZxt7PONa0sohBZmaDfPK2QFubyhfsUvJYv+OaobHZzh0TruWYuVga4D0NOGNm5Y2tmExtlYXsK4011O9VLcyEJriZNcIt8dgNVjcs72K8/1jtHhkQiCtHFKQIxeGGJ8Oc9sK/drR3FZfTm6mj2c9lonhRg62BrhxXfGy04rnIyLs317Nr88OeHJkrpv4ZXs/ShGXd+Xt26oQwTPxIK0cUpCDrQEyfWlLHn17LbIzfLx1cwXPn+71jLnsRvqCkzR1j8alEQLYv72K2Yji5RY9DiUWDrYEKMnLZMcSSstcj3J/FjevL+F5j7j7tHJIQQ62BLh5QzF5WfGZ6+nNm8oIBKc41z8Wl+OlIr9o7Qfi00MF2L66kILsdO3ui4FIRPHztgBvqStb1kj1a/HmTWU094x6wqLTyiHFuDwyQUtvMG6NEDBngeiGaOW83Bqg3J/FtlUFcTmeL03Yu6FU35MYaOoepT80Hfd3RSl49bz7p63RyiHFsMpdxJoRE8360lwqC7I4dM79D7wbCUcUv2gLcGtd7AkC0eyrLeHCwDiXR/Qc0yvBSvd+S138lMOuNYVkpad5QmnHaya4e0SkRUTaReThBdZ/QUSOm59WERmOWheOWvdMPOTRLM7B1gBVBdnUV14929tKERH21Rq9VB13WD4nO4cZHp/htob4NUJwxaJ7VSvtFXGwNcD21QWU+7PidsysdB83rSv2REcqZuUgIj7gi8B+YCvwgIhsjd5GKfUJpdRupdRu4F+A70etnrDWKaXujVUezeLMhiP8oq2f2+rj20MFoyEKBKc4r+MOy+Zga4A0gbfMmyM6VrZUF+i4wwoZmZjh2KXhuLqULPbVlnoi7hAPy2Ev0K6UOqeUmgaeBO67xvYPAE/E4byaZdLYPUpwcpa31Me3EYLouIP7e0Ru45X2AXasLqQ4b+WDrBZCxx1WzmvnBwlHVFxdShZW3OE1l8cd4qEcVgMdUb87zWVXISLrgA3Az6IWZ4vIERE5JCLvWuwkIvKQud2RQECn562EoxeHALh5fUncj72+NJcKf5ZuiJbJ9GyEE53D7EnAPQEdd1gpxy4NkZ4m7F5TFPdjX4k7JL9yWA73A08rpaLLeK5TSu0Bfgf4JxHZuNCOSqnHlFJ7lFJ7ysvjr81TgWOXhlhdlEP
2018-03-21 17:42:59 +01:00
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
2018-12-05 13:28:52 +01:00
"metadata": {
"needs_background": "light"
},
2018-03-21 17:42:59 +01:00
"output_type": "display_data"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"plt.plot(x, y)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let us quickly generate some random data and draw a scatter plot."
]
},
{
"cell_type": "code",
2018-03-21 17:42:59 +01:00
"execution_count": 17,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2019-12-05 12:27:39 +01:00
"<matplotlib.collections.PathCollection at 0x7ff892db3240>"
2018-03-21 17:42:59 +01:00
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
2019-12-05 12:27:39 +01:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAfH0lEQVR4nO3df5Ac9Xnn8fejZW0v2PGKoBC0IIuKOXG2KdBli5DC5wJsIyzjoCPOBYpzQYxPjs+us10+xeLOFRJyVyilihPfcWei2JxxHcHEQWw4QyyrDFUYn3+tWHECgw5i41gjgkRgMZh1vJKe+2N7xOyoe7Z7uqf72z2fV9XW7vTMzjwz0/30t5/vt79t7o6IiDTXsqoDEBGRwVKiFxFpOCV6EZGGU6IXEWk4JXoRkYY7ruoA4px00km+evXqqsMQEamNXbt2PevuK+LuCzLRr169munp6arDEBGpDTP7UdJ9Kt2IiDTckonezE4zs/vN7Ptm9qiZfTRafqKZ7TSzJ6LfyxP+/+roMU+Y2dVFvwEREektTYv+EPAJd38TcB7wYTN7E7AZ+Lq7nwF8Pbq9iJmdCFwP/BpwLnB90g5BREQGY8lE7+5Pu/tD0d8vAo8BE8BlwK3Rw24FNsT8+zpgp7s/5+7PAzuBS4oIXERE0slUozez1cBa4DvAye7+dHTXPwAnx/zLBPDjjtv7omVxz73RzKbNbPrgwYNZwhIRkR5Sj7oxs9cCdwIfc/efmNnR+9zdzSzX7Gjuvg3YBjA5OamZ1qQ2pmZabN2xl/2zc6wcH2PTujVsWBvbnhGpRKoWvZmNspDkb3P37dHiZ8zslOj+U4ADMf/aAk7ruH1qtEykEaZmWly3fQ+t2TkcaM3Ocd32PUzNaDWXcKQZdWPA54HH3P3THXfdDbRH0VwN/E3Mv+8ALjaz5VEn7MXRMpFG2LpjL3Pzhxctm5s/zNYdeyuKSORYaVr05wPvAy4ys93Rz3pgC/BOM3sCeEd0GzObNLPPAbj7c8AfAd+Lfm6Ilok0wv7ZuUzLRaqwZI3e3R8ELOHut8c8fhr4QMftW4Bb+g1QJGQrx8doxST1leNjFUQjEk9nxorksGndGsZGRxYtGxsdYdO6NRVFJHKsIOe6EamL9ugajbqRkCnRi+S0Ye2EErsETaUbEZGGU6IXEWk4JXoRkYZTohcRaTglehGRhlOiFxFpOCV6EZGGU6IXEWk4JXoRkYZTohcRaTglehGRhlOiFxFpOCV6EZGGU6IXEWk4JXoRkYZTohcRabglLzxiZrcAlwIH3P0t0bI7gPa10saBWXc/J+Z/nwJeBA4Dh9x9sqC4RUQkpTRXmPoCcBPwxfYCd//t9t9m9ifACz3+/0J3f7bfAEVEJJ8lE727P2Bmq+PuMzMD/jVwUbFhiYhIUfLW6P8l8Iy7P5FwvwNfM7NdZrax1xOZ2UYzmzaz6YMHD+YMS0RE2vIm+iuB23vc/1Z3/xfAu4APm9nbkh7o7tvcfdLdJ1esWJEzLBERaes70ZvZccDlwB1Jj3H3VvT7AHAXcG6/ryciIv3J06J/B/C4u++Lu9PMTjCz17X/Bi4GHsnxeiIi0oclE72Z3Q58C1hjZvvM7NrorivoKtuY2Uozuze6eTLwoJk9DHwXuMfdv1pc6CIikkaaUTdXJiy/JmbZfmB99PcPgLNzxiciIjnpzFgRkYZTohcRaTglehGRhkszBYKI5DA102Lrjr3sn51j5fgYm9atYcPaiarDkiGiRC8yQFMzLa7bvoe5+cMAtGbnuG77HgAl+4h2hIOn0o3IAG3dsfdokm+bmz/M1h17K4ooLO0dYWt2DueVHeHUTKvq0BpFiV5kgPbPzmVaPmy0IyyHEr3IAK0cH8u0fNhoR1gOJXqRAdq0bg1joyOLlo2NjrBp3ZqE/xgu2hGWQ4leZIA2rJ3gxsvPYmJ8DAMmxse48fKz1NkY0Y6wHBp1IzJgG9ZOKLEnaH8uGnUzWEr0IlIp7QgHT6UbEZGGU6IXEWk4JXoRkYZTohcRaTglehGRhlOiFxFpuDTXjL3FzA6Y2SMdy/7AzFpmtjv6WZ/wv5eY2V4ze9LMNhcZuIiIpJOmRf8F4JKY5X/q7udEP/d232lmI8B/B94FvAm40szelCdYERHJbslE7+4PAM/18dznAk+6+w/c/efAl4DL+ngeERHJIU+N/iNm9n+j0s7ymPsngB933N4XLYtlZhvNbNrMpg8ePJgjLBER6dRvov8s8CvAOcDTwJ/kDcTdt7n7pLtPrlixIu/TiYhIpK9E7+7PuPthdz8C/AULZZpuLeC0jtunRstERKREfSV6Mzul4+a/Ah6Jedj3gDPM7HQzexVwBXB3P68nIiL9W3L2SjO7HbgAOMnM9gHXAxeY2TmAA08BH4weuxL4nLuvd/dDZvYRYAcwAtzi7o8O5F2IiEgic/eqYzjG5OSkT09PVx2GiEhtmNkud5+Mu09nxoqINJwSvYhIwynRi4g0nBK9iEjDKdGLiDScEr2ISMMp0YuINJwSvYhIwynRi4g0nBK9iEjDKdGLiDTckpOaiYiEYGqmxdYde9k/O8fK8TE2rVvDhrWJ1zKSDkr0IhK8qZkW123fw9z8YQBas3Nct30PgJJ9Ckr0Ijmppdm/tJ/d1h17jyb5trn5w2zdsVefdQpK9CI5qKXZvyyf3f7ZudjnSFoui6kzViSHXi1N6S3LZ7dyfCz2OZKWy2JK9CI5qKXZvyyf3aZ1axgbHVm0bGx0hE3r1gwktqZRohfJQS3N/mX57DasneDGy89iYnwMAybGx7jx8rNUHktJNXqRHDatW7OozgxqaaaV9bPbsHZCib1PaS4OfgtwKXDA3d8SLdsKvAf4OfB3wO+4+2zM/z4FvAgcBg4lXc9QpK7aiUejbrLTZ1eeJS8ObmZvA14CvtiR6C8G7nP3Q2b2xwDu/smY/30KmHT3Z7MEpYuDi4hkk+vi4O7+APBc17Kvufuh6Oa3gVNzRykiIgNRRI3+/cAdCfc58DUzc+DP3X1b0pOY2UZgI8CqVasKCEskDDqhSqqWK9Gb2X8CDgG3JTzkre7eMrNfAnaa2ePREcIxop3ANlgo3eSJSyQUOqFKQtD38Eozu4aFTtqrPKHQ7+6t6PcB4C7g3H5fT6SOdEKVhKCvRG9mlwC/B/yGu7+c8JgTzOx17b+Bi4FH+g1UpI50QpWEYMlEb2a3A98C1pjZPjO7FrgJeB0L5ZjdZnZz9NiVZnZv9K8nAw+a2cPAd4F73P2rA3kXIoHSCVUSgiVr9O5+Zczizyc8dj+wPvr7B8DZuaITqTmdUCUh0JmxIl2KHCWjk4IkBEr0Ih0GMUpGp+5L1TSpmUgHjZKRJlKiF+mgUTLSREr0Ih00SkaaSIlepIMucCFNpM5YkQ4aJSNNpEQv0kWjZKRpVLoREWk4tehFJJamV24OJfqAaMOSUGh65WZR6SYQ7Q2rNTuH88qGNTXTqjo0GUI6caxZ1KIPRK8NSy0oKVvdTxzT0fFiatEHou4bljRLnU8c09HxsZToA1HnDUsWksv5W+7j9M33cP6W+2qfVOp84pjKTsdS6SYQmre8PrrLAheeuYI7d7Ua1XFZ5xPHdHR8LCX6QNR5wxomcaNRbvv239N90eQm9K/U9cSxleNjtGKS+jAfHSvRB6SuG9YwiSsLdCf5tmFuQVZJR8fHSlWjN7NbzOyAmT3SsexEM9tpZk9Ev5cn/O/V0WOeMLOriwpcpApZkvcwtyCrtGHtBDdefhYT42MYMDE+xo2XnzXUjai0LfovsHBB8C92LNsMfN3dt5jZ5uj2Jzv/ycxOBK4HJllo+Owys7vd/fm8gYtUIaksYCxu2Q97C7JqOjpeLFWL3t0fAJ7rWnwZcGv0963Ahph/XQfsdPfnouS+E7ikz1iF5o3uqJuk0ShXnbcqiBak1g+Jk6dGf7K7Px39/Q/AyTGPmQB+3HF7X7TsGGa2EdgIsGrVqhxhNZdOS69eyJ3madcPnUw0fArpjHV3N7OkPqm0z7EN2AYwOTmZ67maSmfPhiHUskCa9UO
2018-03-21 17:42:59 +01:00
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
2018-12-05 13:28:52 +01:00
"metadata": {
"needs_background": "light"
},
2018-03-21 17:42:59 +01:00
"output_type": "display_data"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"x = np.random.normal(42, 3, 100)\n",
"y = np.random.gamma(7, 1, 100)\n",
2019-12-05 12:27:39 +01:00
"\n",
2018-03-21 17:37:55 +01:00
"plt.scatter(x, y)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-12-05 12:27:39 +01:00
"## Case Study (continued): Importing the Iris data"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2018-03-21 17:37:55 +01:00
"The sklearn library provides several sample datasets, among which is also the Iris dataset.\n",
"\n",
"As a table, the dataset would look like:\n",
2019-10-02 09:06:44 +02:00
"<img src=\"raw/iris.png\" width=\"50%\">\n",
2018-03-21 17:37:55 +01:00
"\n",
"However, the data object imported from sklearn is organized slightly different. In particular, the so-called **features** are seperated from the **labels**."
]
},
{
"cell_type": "code",
2018-03-21 17:42:59 +01:00
"execution_count": 18,
2018-03-21 17:37:55 +01:00
"metadata": {},
"outputs": [],
"source": [
2019-12-05 12:27:39 +01:00
"from sklearn.datasets import load_iris"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [],
"source": [
2018-03-21 17:37:55 +01:00
"iris = load_iris()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Using Python's **dir()** function we can inspect the data object, i.e. find out what **attributes** it has."
]
},
{
"cell_type": "code",
2019-12-05 12:27:39 +01:00
"execution_count": 20,
2018-03-21 17:42:59 +01:00
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2018-12-05 13:28:52 +01:00
"['DESCR', 'data', 'feature_names', 'filename', 'target', 'target_names']"
2018-03-21 17:42:59 +01:00
]
},
2019-12-05 12:27:39 +01:00
"execution_count": 20,
2018-03-21 17:42:59 +01:00
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"dir(iris)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"iris.data provides us with a Numpy array, where the first dimension equals the number of observed flowers (**instances**) and the second dimension lists the various features of a flower."
]
},
{
"cell_type": "code",
2019-12-05 12:27:39 +01:00
"execution_count": 21,
2018-03-21 17:42:59 +01:00
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[5.1, 3.5, 1.4, 0.2],\n",
" [4.9, 3. , 1.4, 0.2],\n",
" [4.7, 3.2, 1.3, 0.2],\n",
" [4.6, 3.1, 1.5, 0.2],\n",
" [5. , 3.6, 1.4, 0.2],\n",
" [5.4, 3.9, 1.7, 0.4],\n",
" [4.6, 3.4, 1.4, 0.3],\n",
" [5. , 3.4, 1.5, 0.2],\n",
" [4.4, 2.9, 1.4, 0.2],\n",
" [4.9, 3.1, 1.5, 0.1],\n",
" [5.4, 3.7, 1.5, 0.2],\n",
" [4.8, 3.4, 1.6, 0.2],\n",
" [4.8, 3. , 1.4, 0.1],\n",
" [4.3, 3. , 1.1, 0.1],\n",
" [5.8, 4. , 1.2, 0.2],\n",
" [5.7, 4.4, 1.5, 0.4],\n",
" [5.4, 3.9, 1.3, 0.4],\n",
" [5.1, 3.5, 1.4, 0.3],\n",
" [5.7, 3.8, 1.7, 0.3],\n",
" [5.1, 3.8, 1.5, 0.3],\n",
" [5.4, 3.4, 1.7, 0.2],\n",
" [5.1, 3.7, 1.5, 0.4],\n",
" [4.6, 3.6, 1. , 0.2],\n",
" [5.1, 3.3, 1.7, 0.5],\n",
" [4.8, 3.4, 1.9, 0.2],\n",
" [5. , 3. , 1.6, 0.2],\n",
" [5. , 3.4, 1.6, 0.4],\n",
" [5.2, 3.5, 1.5, 0.2],\n",
" [5.2, 3.4, 1.4, 0.2],\n",
" [4.7, 3.2, 1.6, 0.2],\n",
" [4.8, 3.1, 1.6, 0.2],\n",
" [5.4, 3.4, 1.5, 0.4],\n",
" [5.2, 4.1, 1.5, 0.1],\n",
" [5.5, 4.2, 1.4, 0.2],\n",
2018-12-05 13:28:52 +01:00
" [4.9, 3.1, 1.5, 0.2],\n",
2018-03-21 17:42:59 +01:00
" [5. , 3.2, 1.2, 0.2],\n",
" [5.5, 3.5, 1.3, 0.2],\n",
2018-12-05 13:28:52 +01:00
" [4.9, 3.6, 1.4, 0.1],\n",
2018-03-21 17:42:59 +01:00
" [4.4, 3. , 1.3, 0.2],\n",
" [5.1, 3.4, 1.5, 0.2],\n",
" [5. , 3.5, 1.3, 0.3],\n",
" [4.5, 2.3, 1.3, 0.3],\n",
" [4.4, 3.2, 1.3, 0.2],\n",
" [5. , 3.5, 1.6, 0.6],\n",
" [5.1, 3.8, 1.9, 0.4],\n",
" [4.8, 3. , 1.4, 0.3],\n",
" [5.1, 3.8, 1.6, 0.2],\n",
" [4.6, 3.2, 1.4, 0.2],\n",
" [5.3, 3.7, 1.5, 0.2],\n",
" [5. , 3.3, 1.4, 0.2],\n",
" [7. , 3.2, 4.7, 1.4],\n",
" [6.4, 3.2, 4.5, 1.5],\n",
" [6.9, 3.1, 4.9, 1.5],\n",
" [5.5, 2.3, 4. , 1.3],\n",
" [6.5, 2.8, 4.6, 1.5],\n",
" [5.7, 2.8, 4.5, 1.3],\n",
" [6.3, 3.3, 4.7, 1.6],\n",
" [4.9, 2.4, 3.3, 1. ],\n",
" [6.6, 2.9, 4.6, 1.3],\n",
" [5.2, 2.7, 3.9, 1.4],\n",
" [5. , 2. , 3.5, 1. ],\n",
" [5.9, 3. , 4.2, 1.5],\n",
" [6. , 2.2, 4. , 1. ],\n",
" [6.1, 2.9, 4.7, 1.4],\n",
" [5.6, 2.9, 3.6, 1.3],\n",
" [6.7, 3.1, 4.4, 1.4],\n",
" [5.6, 3. , 4.5, 1.5],\n",
" [5.8, 2.7, 4.1, 1. ],\n",
" [6.2, 2.2, 4.5, 1.5],\n",
" [5.6, 2.5, 3.9, 1.1],\n",
" [5.9, 3.2, 4.8, 1.8],\n",
" [6.1, 2.8, 4. , 1.3],\n",
" [6.3, 2.5, 4.9, 1.5],\n",
" [6.1, 2.8, 4.7, 1.2],\n",
" [6.4, 2.9, 4.3, 1.3],\n",
" [6.6, 3. , 4.4, 1.4],\n",
" [6.8, 2.8, 4.8, 1.4],\n",
" [6.7, 3. , 5. , 1.7],\n",
" [6. , 2.9, 4.5, 1.5],\n",
" [5.7, 2.6, 3.5, 1. ],\n",
" [5.5, 2.4, 3.8, 1.1],\n",
" [5.5, 2.4, 3.7, 1. ],\n",
" [5.8, 2.7, 3.9, 1.2],\n",
" [6. , 2.7, 5.1, 1.6],\n",
" [5.4, 3. , 4.5, 1.5],\n",
" [6. , 3.4, 4.5, 1.6],\n",
" [6.7, 3.1, 4.7, 1.5],\n",
" [6.3, 2.3, 4.4, 1.3],\n",
" [5.6, 3. , 4.1, 1.3],\n",
" [5.5, 2.5, 4. , 1.3],\n",
" [5.5, 2.6, 4.4, 1.2],\n",
" [6.1, 3. , 4.6, 1.4],\n",
" [5.8, 2.6, 4. , 1.2],\n",
" [5. , 2.3, 3.3, 1. ],\n",
" [5.6, 2.7, 4.2, 1.3],\n",
" [5.7, 3. , 4.2, 1.2],\n",
" [5.7, 2.9, 4.2, 1.3],\n",
" [6.2, 2.9, 4.3, 1.3],\n",
" [5.1, 2.5, 3. , 1.1],\n",
" [5.7, 2.8, 4.1, 1.3],\n",
" [6.3, 3.3, 6. , 2.5],\n",
" [5.8, 2.7, 5.1, 1.9],\n",
" [7.1, 3. , 5.9, 2.1],\n",
" [6.3, 2.9, 5.6, 1.8],\n",
" [6.5, 3. , 5.8, 2.2],\n",
" [7.6, 3. , 6.6, 2.1],\n",
" [4.9, 2.5, 4.5, 1.7],\n",
" [7.3, 2.9, 6.3, 1.8],\n",
" [6.7, 2.5, 5.8, 1.8],\n",
" [7.2, 3.6, 6.1, 2.5],\n",
" [6.5, 3.2, 5.1, 2. ],\n",
" [6.4, 2.7, 5.3, 1.9],\n",
" [6.8, 3. , 5.5, 2.1],\n",
" [5.7, 2.5, 5. , 2. ],\n",
" [5.8, 2.8, 5.1, 2.4],\n",
" [6.4, 3.2, 5.3, 2.3],\n",
" [6.5, 3. , 5.5, 1.8],\n",
" [7.7, 3.8, 6.7, 2.2],\n",
" [7.7, 2.6, 6.9, 2.3],\n",
" [6. , 2.2, 5. , 1.5],\n",
" [6.9, 3.2, 5.7, 2.3],\n",
" [5.6, 2.8, 4.9, 2. ],\n",
" [7.7, 2.8, 6.7, 2. ],\n",
" [6.3, 2.7, 4.9, 1.8],\n",
" [6.7, 3.3, 5.7, 2.1],\n",
" [7.2, 3.2, 6. , 1.8],\n",
" [6.2, 2.8, 4.8, 1.8],\n",
" [6.1, 3. , 4.9, 1.8],\n",
" [6.4, 2.8, 5.6, 2.1],\n",
" [7.2, 3. , 5.8, 1.6],\n",
" [7.4, 2.8, 6.1, 1.9],\n",
" [7.9, 3.8, 6.4, 2. ],\n",
" [6.4, 2.8, 5.6, 2.2],\n",
" [6.3, 2.8, 5.1, 1.5],\n",
" [6.1, 2.6, 5.6, 1.4],\n",
" [7.7, 3. , 6.1, 2.3],\n",
" [6.3, 3.4, 5.6, 2.4],\n",
" [6.4, 3.1, 5.5, 1.8],\n",
" [6. , 3. , 4.8, 1.8],\n",
" [6.9, 3.1, 5.4, 2.1],\n",
" [6.7, 3.1, 5.6, 2.4],\n",
" [6.9, 3.1, 5.1, 2.3],\n",
" [5.8, 2.7, 5.1, 1.9],\n",
" [6.8, 3.2, 5.9, 2.3],\n",
" [6.7, 3.3, 5.7, 2.5],\n",
" [6.7, 3. , 5.2, 2.3],\n",
" [6.3, 2.5, 5. , 1.9],\n",
" [6.5, 3. , 5.2, 2. ],\n",
" [6.2, 3.4, 5.4, 2.3],\n",
" [5.9, 3. , 5.1, 1.8]])"
]
},
2019-12-05 12:27:39 +01:00
"execution_count": 21,
2018-03-21 17:42:59 +01:00
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"iris.data"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To find out what the four features are, we can list them:"
]
},
{
"cell_type": "code",
2019-12-05 12:27:39 +01:00
"execution_count": 22,
2018-03-21 17:42:59 +01:00
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['sepal length (cm)',\n",
" 'sepal width (cm)',\n",
" 'petal length (cm)',\n",
" 'petal width (cm)']"
]
},
2019-12-05 12:27:39 +01:00
"execution_count": 22,
2018-03-21 17:42:59 +01:00
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"iris.feature_names"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Similarly, we can also print the flowers' labels (a.k.a. targets):"
]
},
{
"cell_type": "code",
2019-12-05 12:27:39 +01:00
"execution_count": 23,
2018-03-21 17:42:59 +01:00
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
" 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
" 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
" 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
" 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
" 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2])"
]
},
2019-12-05 12:27:39 +01:00
"execution_count": 23,
2018-03-21 17:42:59 +01:00
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"iris.target"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The three flower classes are encoded with integers. Let's show the corresponding names:"
]
},
{
"cell_type": "code",
2019-12-05 12:27:39 +01:00
"execution_count": 24,
2018-03-21 17:42:59 +01:00
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array(['setosa', 'versicolor', 'virginica'], dtype='<U10')"
]
},
2019-12-05 12:27:39 +01:00
"execution_count": 24,
2018-03-21 17:42:59 +01:00
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"iris.target_names"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-12-05 12:27:39 +01:00
"## Case Study (continued): Simple Visualizations"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2018-03-21 17:37:55 +01:00
"Since the data is four dimensional, we cannot visualize all features together. Instead, we can plot the distribution of the flower classes by a single feature using histograms."
]
},
{
"cell_type": "code",
2019-12-05 12:27:39 +01:00
"execution_count": 25,
2018-03-21 17:42:59 +01:00
"metadata": {},
"outputs": [
{
"data": {
2019-12-05 12:27:39 +01:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEGCAYAAAB8Ys7jAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAZxUlEQVR4nO3df3RU9bnv8fcDiYYItShpjYQIPS0gEDAQBY+noiDVo6nWpVzai1agLkQUtVot2lawq7ftWbdLj+teqkVFvMpBadD21NVbBQ7Uq6VKkoLhh4hi1ATkZ0VAfhh57h+zoUmAZDKzk8k3fF5rzcrMnv3j2Un4sPOdvZ9t7o6IiISnU6YLEBGR1CjARUQCpQAXEQmUAlxEJFAKcBGRQGW15cZ69OjhvXv3bstNiogEr6KiYru75zWe3qYB3rt3b8rLy9tykyIiwTOz9481XUMoIiKBUoCLiARKAS4iEqg2HQMXkY7ls88+o6amhv3792e6lA4hJyeHgoICsrOzk5pfAS4iKaupqaFbt2707t0bM8t0OUFzd3bs2EFNTQ19+vRJahkNoYhIyvbv38/pp5+u8I6BmXH66ae36K8ZBbiIpEXhHZ+Wfi8V4CIigVKAi0hszOJ9xG3u3Lls2rQp/hVnyAkZ4Jn4xRGRzFOAi4i0I3v37uWKK65gyJAhDBo0iOeee46KigpGjhzJsGHDuPTSS9m8eTNlZWWUl5czfvx4zjnnHPbt28eSJUsoLi6mqKiISZMmceDAAQCmT5/OgAEDGDx4MD/4wQ8A+MMf/sDw4cMpLi7mkksuYcuWLZnc7QR3b7PHsGHDvD2A5h8i0ry1a9c2eJ3Mv62WPJJRVlbmN95445HXH3/8sZ9//vm+detWd3d/9tlnfeLEie7uPnLkSF+xYoW7u+/bt88LCgp8/fr17u5+/fXX+0MPPeTbt2/3vn37+qFDh9zd/e9//7u7u+/cufPItMcee8zvvPPOFL9rTWv8PXV3B8r9GJmq88BFJGhFRUXcdddd/PCHP6S0tJTu3buzevVqxowZA8Dnn39Ofn7+UcutX7+ePn360LdvXwBuuOEGZs2axa233kpOTg7f+973KC0tpbS0FEic8z5u3Dg2b97MwYMHkz5XuzVpCEVEgta3b18qKyspKirixz/+MQsXLmTgwIGsXLmSlStXUlVVxcsvv5z0+rKysnjjjTe49tprefHFF7nssssAmDZtGrfeeitVVVX85je/aRdXnyrARSRomzZtIjc3l+uuu467776b119/nW3btrF8+XIgcbn/mjVrAOjWrRu7d+8GoF+/flRXV/POO+8A8PTTTzNy5Ej27NnDrl27uPzyy3nooYdYtWoVALt27aJnz54APPXUU229m8ekIRQRiY1722+zqqqKu+++m06dOpGdnc0jjzxCVlYWt912G7t27aKuro477riDgQMHMmHCBKZMmUKXLl1Yvnw5Tz75JGPHjqWuro5zzz2XKVOmsHPnTq666ir279+Pu/Pggw8CMHPmTMaOHUv37t0ZNWoU7733XtvvbCPmbfgdLykp8fZwQ4dkThPMxC+iSGjWrVvH2WefnekyOpRjfU/NrMLdSxrPqyEUEZFAKcBFRAKlABcRCZQCXEQkUM0GuJnNMbOtZra63rT/aWZvmdmbZvaCmX2xdcsUEZHGkjkCnwtc1mjaImCQuw8G3gbujbkuERFpRrMB7u6vADsbTXvZ3euil38FClqhNhEJTXvvJ5uE+++/n8WLF7d4uWXLlh257L6txHEhzyTguRjWIyLSJg43g+rU6ehj2J/+9KdtUkNdXR1ZWelFcFpLm9mPgDpgXhPzTAYmAxQWFqazORFpAXug5UewPiOsK9imT59Or169uOWWW4DE1ZJdu3bF3VmwYAEHDhzg6quv5oEHHqC6uppLL72U4cOHU1FRwR//+EdmzJhBeXk5ZsakSZP4/ve/z4QJEygtLeXaa69lxYoV3H777ezdu5eTTz6ZJUuWkJ2dzc0330x5eTlZWVk8+OCDXHzxxQ3q2rlzJ5MmTWLjxo3k5uYye/ZsBg8ezMyZM3n33XfZuHEjhYWFzJ8/P639T/ksFDObAJQC472Jyzndfba7l7h7SV5eXqqbExE5yrhx41iwYMGR1wsWLCAvL48NGzbwxhtvsHLlSioqKnjllVcA2LBhA1OnTmXNmjVs376d2tpaVq9eTVVVFRMnTmyw7oMHDzJu3DgefvhhVq1axeLFi+nSpQuzZs3CzKiqqmL+/PnccMMNRzW2mjFjBsXFxbz55pv8/Oc/57vf/e6R99auXcvixYvTDm9I8QjczC4D7gFGuvunaVchIpKC4uJitm7dyqZNm9i2bRvdu3c/0n2wuLgYgD179rBhwwYKCws566yzGDFiBABf+cpX2LhxI9OmTeOKK67gG9/4RoN1r1+/nvz8fM4991wAvvCFLwDw6quvMm3aNAD69+/PWWedxdtvv91g2VdffZWFCxcCMGrUKHbs2MEnn3wCwJVXXkmXLl1i2f9mA9zM5gMXAT3MrAaYQeKsk5OBRdFdlP/q7lNiqUhEpAXGjh1LWVkZH330EePGjeP999/n3nvv5aabbmowX3V1NaeccsqR1927d2fVqlW89NJLPProoyxYsIA5c+a0er31a0hXMmehfMfd8909290L3P0Jd/+qu/dy93Oih8JbRDJi3LhxPPvss5SVlTF27FguvfRS5syZw549ewCora1l69atRy23fft2Dh06xDXXXMPPfvYzKisrG7zfr18/Nm/ezIoVKwDYvXs3dXV1fP3rX2fevMTHfm+//TYffPAB/fr1a7Bs/XmWLVtGjx49jhzBx0ntZEUkPhlo4zlw4EB2795Nz549yc/PJz8/n3Xr1nH++ecD0LVrV5555hk6d+7cYLna2lomTpzIoUOHAPjFL37R4P2TTjqJ5557jmnTprFv3z66dOnC4sWLmTp1KjfffDNFRUVkZWUxd+5cTj755AbLzpw5k0mTJjF48GByc3NbrX+42skeh9rJSuja4iwUtZONn9rJioicABTgIiKBUoCLiARKAS4iEigFuIhIoBTgIiKB0nngIhKbVE5dbEoqzbU2bdrEbbfdRllZWYuWu/HGG7nzzjsZMGDAced59NFHyc3NbdDbJJMU4CLSoZx55pnHDO/m2rc+/vjjza57ypT2ddG5hlBEJFjTp09n1qxZR17PnDmTX/3qVwwaNAiAuXPncuWVVzJq1ChGjx7NoUOHmDp1Kv3792fMmDFcfvnlR8L+oosu4vCFhl27duVHP/oRQ4YMYcSIEWzZsqXB+gHeeecdLrnkEoYMGcLQoUN599132bNnD6NHj2bo0KEUFRXx+9//vlX3XwEuIsE6VjvZ4cOHN5insrKSsrIy/vznP/P8889TXV3N2rVrefrpp1m+fPkx17t3715GjBjBqlWruPDCC3nssceOmmf8+PHccsstrFq1ir/85S/k5+eTk5PDCy+8QGVlJUuXLuWuu+6iNa921xCKiATrWO1ke/Xq1WCeMWPGcNpppwGJNq9jx46lU6dOnHHGGUfdiOGwk0466cjt0YYNG8aiRYsavL97925qa2u5+uqrAcjJyQHgs88+47777uOVV16hU6dO1NbWsmXLFs4444xY9/swBbiIBK1xO9nGUmnfmp2dTdQqm86dO1NXV9fMEgnz5s1j27ZtVFRUkJ2dTe/evY+62UOcNIQiIkFr3E62KRdccAELFy7k0KFDbNmyhWXLlqW0zW7dulFQUMDvfvc7AA4cOMCnn37Krl27+NKXvkR2djZLly7l/fffT2n9ydIRuIjEJhP31GzcTra6uvq4815zzTUsWbKEAQMG0KtXL4YOHcqpp56a0naffvppbrrpJu6//36ys7P57W9/y/jx4/nmN79JUVERJSUl9O/fP8W9So7ayR6H2slK6NRO9tj27NlD165d2bFjB+eddx6vvfZaq41Rp6Il7WR1BC4iJ5TS0lI+/vhjDh48yE9+8pN2Fd4tpQAXkRNKquPe7ZE+xBSRtLTlMGxH19LvpQJcRFK
2018-03-21 17:42:59 +01:00
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
2018-12-05 13:28:52 +01:00
"metadata": {
"needs_background": "light"
},
2018-03-21 17:42:59 +01:00
"output_type": "display_data"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"feature_index = 2\n",
"colors = ['blue', 'red', 'green']\n",
"\n",
"for label, color in zip(range(len(iris.target_names)), colors):\n",
" plt.hist(iris.data[iris.target==label, feature_index], \n",
" label=iris.target_names[label],\n",
" color=color)\n",
"\n",
"plt.xlabel(iris.feature_names[feature_index])\n",
"plt.legend(loc='upper right')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Also, we can draw scatter plots of two features."
]
},
{
"cell_type": "code",
2019-12-05 12:27:39 +01:00
"execution_count": 26,
2018-03-21 17:42:59 +01:00
"metadata": {},
"outputs": [
{
"data": {
2019-12-05 12:27:39 +01:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEGCAYAAABhMDI9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3de3wU9bn48c+TTShEFFHpTyqSSK1YIdy9IG2lBo/WC57+lIP9RQVbT0pS9Vhr1ZYW0Dbtae1Raytoaq3W5Hgpaqu22oqVem0VEIhCUasEAS+IGkRQgTy/P2aC2c3uZmd2d3Z29nm/XvNK5rtz+c7Mki8zz3eer6gqxhhjTJeyQlfAGGNMuFjDYIwxJo41DMYYY+JYw2CMMSaONQzGGGPilBe6Al7tt99+Wl1dXehqGGNMUVm6dOlbqjook2WLrmGorq5myZIlha6GMcYUFRFpz3RZe5RkjDEmjjUMxhhj4uS1YRCRb4rI8yLynIjcJiJ9Ez7/hIjcISIvicg/RKQ6n/UxxhjTu7zFGETkAOAC4DBV3S4idwJnADd3W+xrwDuqerCInAH8BJjudV87duxg/fr1fPDBBzmouenbty9DhgyhoqKi0FUxxhRAvoPP5UA/EdkBVAIbEz4/FZjn/r4Q+KWIiHpM4LR+/Xr23HNPqqurEZFs61zSVJXNmzezfv16DjrooEJXxxhTAHl7lKSqG4CfAeuA14AOVf1LwmIHAK+6y+8EOoB9E7clIvUiskRElmzatKnHvj744AP23XdfaxRyQETYd9997e7LmBKWt4ZBRAbi3BEcBHwK2ENEzvSzLVVtVtUJqjph0KDk3XCtUcgdO5fGlLZ8Bp+nAK+o6iZV3QHcDRydsMwG4EAAESkHBgCb81gnY0wItLa1Un1NNWWXl1F9TTWtba2FrpLpJp8NwzrgKBGpFOe/oLXA6oRl7gVmuL+fDvzVa3yhGN18881s3JgYbjGmNLS2tVJ/Xz3tHe0oSntHO/X31VvjECL5jDH8AyegvAxoc/fVLCJXiMhUd7FfA/uKyEvARcBl+apPmFjDYErZ7Idns23HtriybTu2Mfvh2QWqkUmU1/cYVHWuqh6qqiNV9SxV/VBV56jqve7nH6jqNFU9WFWPUNWX81mfLq2tUF0NZWXOz9Yc/Efl/fff56STTmL06NGMHDmSO+64g6VLl3LMMccwfvx4jj/+eF577TUWLlzIkiVLqKurY8yYMWzfvp2HH36YsWPHUlNTw1e/+lU+/PBDAC677DIOO+wwRo0axcUXXwzAfffdx5FHHsnYsWOZMmUKb7zxRvaVNyZA6zrWeSo3BaCqRTWNHz9eE61atapHWSotLaqVlarw8VRZ6ZRnY+HChXruuefunn/33Xd14sSJ+uabb6qq6u23367nnHOOqqoec8wx+swzz6iq6vbt23XIkCG6Zs0aVVU966yz9Oqrr9a33npLDznkEO3s7FRV1XfeeUdVVd9+++3dZb/61a/0oosuyq7iKXg5p8Z4UXV1lTKPHlPV1VWFrlqkAUs0w7+zJZcSY/Zs2BZ/F8u2bU55NmpqanjooYe49NJLeeyxx3j11Vd57rnnOO644xgzZgw//OEPWb9+fY/11qxZw0EHHcQhhxwCwIwZM3j00UcZMGAAffv25Wtf+xp33303lZWVgPPOxvHHH09NTQ1XXnklzz//fHYVNyZgTbVNVFZUxpVVVlTSVNtUoBqZRCXXMKxLcbeaqjxThxxyCMuWLaOmpobvfe973HXXXYwYMYLly5ezfPly2tra+MtfEl/jSK28vJynn36a008/nfvvv58TTjgBgPPPP5/zzjuPtrY2brjhBnvfwBSdupo6mk9ppmpAFYJQNaCK5lOaqaupK3TVjKvo0m5na+hQaE+SfHbo0Oy2u3HjRvbZZx/OPPNM9t57b+bPn8+mTZt46qmnmDhxIjt27OCFF15gxIgR7Lnnnrz33nsADB8+nLVr1/LSSy9x8MEHc+utt3LMMcewdetWtm3bxoknnsikSZMYNmwYAB0dHRxwwAEA3HLLLdlV2pgCqaups4YgxEquYWhqgvr6+MdJlZVOeTba2tr49re/TVlZGRUVFSxYsIDy8nIuuOACOjo62LlzJxdeeCEjRoxg5syZzJo1i379+vHUU0/xm9/8hmnTprFz504OP/xwZs2axdtvv82pp57KBx98gKpy1VVXATBv3jymTZvGwIEDOfbYY3nllVeyq7gxxiQQLbLXBiZMmKCJA/WsXr2az372sxlvo7XViSmsW+fcKTQ1QZ395yWO13NqjAk3EVmqqhMyWbbk7hjAaQSsITDGmORKLvhsMhdE2oKo7MN4Y9ck3EryjsH07v2P3qf+j/W731DtSlsA5Cxo2JUaodj3YbyxaxJ+dsdgknrng3fynrYgiNQIln4hfOyahJ81DCapXZ27kpbnMm1BEKkRLP1C+Ng1CT9rGExSsbJY0vKhA7J84SODbRXbPow3dk3CzxqGkJozZw6LFi3yvN7ixYs5+eSTs97/wL4D8562IIjUCJZ+IXzsmoSfNQwFpKp0dnYm/eyKK65gypQpea/Dzp07k5bv0WePvKctCCI1Ql1NHTNGzyAmzh1QTGLMGD3DgpwFZCkxikCm2fbCMmWbXVVVnVSqVVWqIs7PLFOrXnrppfrLX/5y9/zcuXP1yiuv1J/+9Kc6YcIEramp0Tlz5qiq6iuvvKKHHHKInnXWWXrYYYfp2rVrdcaMGTpixAgdOXKkXnXVVaqqOmPGDP3d736nqqpPP/20Tpw4UUeNGqWHH364btmyRbdv364zZ87UkSNH6pgxY/Svf/2rqqo+8sgjetJJJ6mq6ubNm/XUU0/VmpoaPfLII3XFihW763fmmWfq0UcfrWeccUbSY4pKdtWWlS1a2VQZl8WzsqlSW1ZmmU7XmCKDZVdNo7XVyYnR3u5k3W5vd+azGJRh+vTp3Hnnnbvn77zzTgYNGsSLL77I008/zfLly1m6dCmPPvooAC+++CKNjY08//zzvPXWW2zYsIHnnnuOtrY2zjnnnLhtf/TRR0yfPp2f//znrFixgkWLFtGvXz+uu+46RIS2tjZuu+02ZsyY0SOh3ty5cxk7diwrV67kRz/6EWefffbuz1atWsWiRYu47bbbfB93MbAeMMZ4V3oNQx7ybo8dO5Y333yTjRs3smLFCgYOHLg7m+rYsWMZN24c//znP3nxxRcBqKqq4qijjgJg2LBhvPzyy5x//vk8+OCD7LXXXnHbXrNmDYMHD+bwww8HYK+99qK8vJzHH3+cM888E4BDDz2UqqoqXnjhhbh1H3/8cc466ywAjj32WDZv3syWLVsAmDp1Kv369fN9zMXCesAY413pveCWp7zb06ZNY+HChbz++utMnz6d9vZ2vvOd7/D1r389brm1a9eyxx577J4fOHAgK1as4M9//jPXX389d955JzfddFNWdclE9zpE2dABQ2nv6JlO13rAGJNa3u4YRGS4iCzvNm0RkQsTlpksIh3dlpmTr/rsliq/dpZ5t6dPn87tt9/OwoULmTZtGscffzw33XQTW7duBWDDhg28+eabPdZ766236Ozs5LTTTuOHP/why5Yti/t8+PDhvPbaazzzzDMAvPfee+zcuZPPf/7ztLqPv1544QXWrVvH8OHD49btvszixYvZb7/9etyRRJ31gDHGu7zdMajqGmAMgIjEgA3APUkWfUxVs+9fmak85d0eMWIE7733HgcccACDBw9m8ODBrF69mokTJwLQv39/WlpaiMXi3w/YsGED55xzzu7eST/+8Y/jPu/Tpw933HEH559/Ptu3b6dfv34sWrSIxsZGGhoaqKmpoby8nJtvvplPfOITcevOmzePr371q4waNYrKysqSHL+hq6fL7Idns65jHUMHDKWptsl6wBiTRiBpt0Xk34C5qjopoXwycLGXhiEXabct73bvLO22MdESxrTbZwCpur9MFJEVwEacRqLHIMYiUg/UAwzNdqg1sLzbxhiTRt57JYlIH2Aq8LskHy8DqlR1NPAL4PfJtqGqzao
2018-03-21 17:42:59 +01:00
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
2018-12-05 13:28:52 +01:00
"metadata": {
"needs_background": "light"
},
2018-03-21 17:42:59 +01:00
"output_type": "display_data"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"first_feature_index = 1\n",
"second_feature_index = 0\n",
"\n",
"colors = ['blue', 'red', 'green']\n",
"\n",
"for label, color in zip(range(len(iris.target_names)), colors):\n",
" plt.scatter(iris.data[iris.target==label, first_feature_index], \n",
" iris.data[iris.target==label, second_feature_index],\n",
" label=iris.target_names[label],\n",
" c=color)\n",
"\n",
"plt.xlabel(iris.feature_names[first_feature_index])\n",
"plt.ylabel(iris.feature_names[second_feature_index])\n",
"plt.legend(loc='upper left')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Using the higher level library pandas, one can easily create a so-called **scatterplot matrix**."
]
},
{
"cell_type": "code",
2019-12-05 12:27:39 +01:00
"execution_count": 27,
2018-03-21 17:42:59 +01:00
"metadata": {},
"outputs": [
{
"data": {
2019-12-05 12:27:39 +01:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAHjCAYAAAD2Xrx8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nOy9d5Rcx3ng+6vbOafJGZhBzpEASRAMkiiJpERSsiVLlCxbwVqnZ3u9a/vZZ9dhvbafvWvv8+6xpWc5ypatYJnKpExJJCUmRILIYQaTU89M53xvvT9uT09Az6AxGcD9ndNnprvr3vrurepbVV99QUgpMTAwMDAwMFgbKKstgIGBgYGBgcEUxsBsYGBgYGCwhjAGZgMDAwMDgzWEMTAbGBgYGBisIYyB2cDAwMDAYA1hDMwGBgYGBgZrCPNqCwBQVVUl29raVlsMgwVy/fp1jPa7fTHa7/bFaLvbmxMnToSllNWzP18TA3NbWxvHjx9fbTEMFsj+/fuN9lskx6+P0x9Jc7g9RI3HvqJ1G+23Nrg4FOPiYJydTT7WV7srOsZou6UnlSvw4qVRrGaFoxurMZuWT7EshOgu9/maGJgNDO5mJpI5Xr4SBiCvSt6/r2mVJTJYaaSUPH9uGFWTjMQzfKrCgdlg6TnVE+HiUByAOp+dbQ2+FZfB2GM2MFhlHFYTbps+R6722FZZGoPVQAhBlVtve6MPrC6T99+kTLXJSmOsmA0MVhm7xcQzh1qJpvPUeo2H8t3K+/c1EU5kqTEG5lVlY62H4GErFkXB57SsigzGwDyLtl//ZkXlrv/hY8ssicHdhMNqwmE1rbYYBquI1azQ4HesthgGsGor5UkMVbaBgYGBgcEawlgxG9xWZPIq/3qyn1gmz2M76mkOOm8oE0nl+NeT/UjgqT2NBF3WG8r0R9J8480BXDYz79vbtKqrVVWTfP3NAfojaR7eXMOWeu+qyWKw/MQyeb56sp+8qvHe3Y3z7imf6J7gf3/vCh67hd9+Yhtv9kc4PxBjT4ufe9urVlDqu4eRWIZnTw9gsyg8tacRj/1GdXbnaILnzg0Tcll5ck8juYLG73z9HCPxLJ84so4jG27wgLoljBWzwW1F30Sa4ViGdE7lwmCsbJlrowmi6TyxdJ6rI4myZS4MxEjlVEbjWXonUssp8k2ZSOXoCifJFTTe6ouuqiwGy0/XaJLxZI54psDl4fi8Zb9/cYRUTmU4luFE9xhv9kbIFTRO90ZWSNq7j0vDcRLZAmOJHN1j5Z8NZwdiZPIq/RH9eXRuIErPeIpMXuWFCyOLlsFYMa8Ale5bg7F3fTOaAg6qPDbimfycK8v1VW5O9egPrvZqV9kym+s9nB+M4bGbaQqs7r5ewGmlJehkIJJmW6OxWr7Taaty4XNYyKsaG2rmdovKqxqH24Oc6Y/gtVvY1xpCURTODcTY2ehfQYnvLjbWejg/EMNqVmgN6Ro5TZNkC1pJs7a13kvveIqQy0qt107AaaXB7yCcyPLgpsWtlsEYmA1uM+wWEx851DpvmYDLyieOrJ+3TLagAfrDr6DJJZNvIZgUwfsM3+W7Bp/Dwk/fv27eMuPJHP9yrJeCqvH7T+4obdk8sqWWR7bUroSYdy21Xjs/c7S99F7TJF883stgNMOh9SEOt4foqHHTUdNRKmM1K/zpB3YvmQyGKtvgrqRrNImqSZJZleFoZrXFMTCYQd+ErhYtaJKe8dXdarnbSeYKDBafEVdHy2+NLTXzrpiFEIeBZ4AjQD2QBs4C3wQ+L6U0NsQMbkt2NfsZimVw28y0hsqruw0MVosNNR4uDyfIqxrbGoztjdXEY7ewp8XP9XCSQ+uCK1LnnAOzEOLbwADwLPD7wAhgBzYCDwHPCiH+p5TyayshqIHBUlLtsfHMTVTiBgarhcNqMkKzriEe3FQDm1auvvlWzB+RUoZnfZYAThZf/0MIYdjrGxgYGBgYLCFz7jHPHpSFEF4hRHDyVa6MgcFaoT+Spm+V3aAMDOZiMJqm19g7XhOkcypXhuOkc+pqi1LiplbZQoifAX4HyACT5qsSmN/s1cBglbg2muBrpwcAeHxnPRtqPasskYHBFD1jKb5ysg+Ad2yrXZXsRQZTfPlkH+F4dk1tb1XiLvWrwHZjdWxwu5DIFEr/x7OFeUoaGKw8iWl9MpldO6u0u5XJ50ViDT0rKhmYrwGGzsXgtmF7o49ktoAEdjbqq5FwIkv3WJINtR68ZULsGRisFJvrPMQyefKqxu5mP+mcyvnBGI1+B3U++2qLd8dzeThOrqCxtd6Logge31nPxaE4W+rXjmatkoH5N4BXhBCvA9nJD6WUv7hsUhkYLAKTIri3Y8ouUdMkXz7RVwzjGV8z6iqDuxNFERxaHyq9/+qpPq6HU1hMgk8cWY/dYmQZWy6ujSb45plBAHKqxt6WAM1BZ9mY+6tJJQPzZ4DvAW8B2vKKY2CwPEg5+Xd1o3wZGMxmqm9O/W+wPEz//a/lZ0ElA7NFSvkryy7JAjByJxtUgqII3revka7RJJvr9GAN2YLKsa4JXDYTe1oCZY+LZ/Kc6J6g3udgU93C1Vy94ymujiTY1uilxmOoKu8mIqkcp3ojtASdtFe7uTAYYziawaQITCbBgbYgj26r49xAjMaAw8jJvUjO9kcZS+Y40BbAab1xeGuvdtMUcJDOq2xfYaM7KSUnuifIFTQOrAtiMc0deLOSgfnbQohPAV9npip7fPGiGhisDDUe+4xB8fXOcU50TwB6Eom2qhujf33/0ijXRhKcFhFqPDYCZdJH3gxVkzx7up+8KukeS/Kx++aPkWxwZ/H8uWH6I2nO9EZ5395GvnN2iHAiy3gyx8ZaDxaTwoG2IAdXKKLUncxQNMN3zw8DenrYR7fV3VDm0nCcvok0AKd7I9wzbUthubk0HOflK7oNtUkR89ZdycD8E8W/vzHtM8NdyuC2ZnJlIgRz7uk5ip+bFYHFvLCw8krx/Hm1YKyG7kIm29xqVrBbTVhMQu9PxdWSw9hPXjJsZgWTIlA1Oed9dVqmhrxyK+rl5FbqvqlkUkpjim9wx7G5zsNbfVECLis1cySqf2hTNS1BJ1VuK27bwn7EQgg+cKCZvok068qsyg3ubB7dVseG2gTxTIEfXQ1zX3sVTpsZm1lBgtEnlpCAy8oHDzQTSefpqC6fTrMl5OTH9jeRK2isn6NMOd7sjXB9LMnBdUHqfQtLE3srdd90GSCE+DkhhH/a+4AQ4mcrOO6dQogfFF+DQognK5LewGAFOH59gmg6z/Vwks5wsmwZs0lhU52HkLv8wF0pHruFLfVew9r2LsRqVthY4+HVa2N0jiY53j3BpjoPbVUuY1BeBmq8djbWelAUMWeZpoDzlgblZLbA9y6O0Dma5PsXRxclX6V1V6Kf+6SUMjL5Rko5AXzyZgdJKb8jpXxQSvkg0AP8ewV1GRisCFXFwdasCAJOw6/ZYPlQFEGwaJ9Q5bl1OwWD1cVqVvA69GdElXtl2q8S/ZxJCCFk0bZcCGECKpZOCLEeGJZSrkwiSwODCtjR5KPWa8NmMeFzlB+Ye8dTvHh5lHqfnYc31yDE3LNwgzuD0XiWf78wjNdu4dFttZjnsZy9FX58fzOjiSy1c2ybGNycl6+Mcn0sxb3tIdpvYcW7WCwmhQ/f08J4Mkedd2W8Kirpdd8B/kUI8YgQ4hHgC8XPKuVp4KsLEc7AYDmp8drnHJQB3ugaZzSe5UxflNFEds5yBncOJ3smGIpmuDwcp2cJk0xYzQqNfseSDfR3G/FMnuPXJwjHs7xybWzF67dbTDT4HfOqyJeSSnrJr6EHGPkPxdcLwH++hTqeAG7I2SyE+JQQ4rgQ4vjo6OL09gYGy0FblR4NKOC0zDuAG9w5tIacCAEum4lqY3W7ZnBazdQWV6vrQnf+3nwlVtka8JfF1y0hhKgDclLKG6Y4UsrPAp8F2L9//9oNwWJwV5DLqfz03x+nL5LimXta+cSR9XjsFoQAt92MWTFWOncDm+u8tASdWEwKFpPC9y4Oc2EwzoG2IJvrPXz1ZD8FTfL
2018-03-21 17:42:59 +01:00
"text/plain": [
"<Figure size 576x576 with 16 Axes>"
]
},
2018-12-05 13:28:52 +01:00
"metadata": {
"needs_background": "light"
},
2018-03-21 17:42:59 +01:00
"output_type": "display_data"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"iris_df = pd.DataFrame(iris.data, columns=iris.feature_names)\n",
2019-12-05 12:27:39 +01:00
"\n",
2018-12-05 13:28:52 +01:00
"pd.plotting.scatter_matrix(iris_df, figsize=(8, 8));"
2018-03-21 17:37:55 +01:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-12-05 12:27:39 +01:00
"## Concept of Generalization"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2018-03-21 17:37:55 +01:00
"The goal of a supervised machine learning model is to make predictions on new (i.e., previously unseen) data.\n",
"\n",
"In a real-world application, we are not interested in marking an already labeled email as spam or not. Instead, we want to make the user's life easier by automatically classifying new incoming mail.\n",
"\n",
"In order to get an idea of how good a model generalizes, a best practice is to split the available data into a training and a test set. Only the former is used to train the model. Then predictions are made on the test data and the predictions can be compared with the actual labels.\n",
"\n",
2019-12-05 12:27:39 +01:00
"Common splits are 75/25 or 60/40."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-10-02 09:06:44 +02:00
"<img src=\"raw/generalization.png\" width=\"60%\">"
2018-03-21 17:37:55 +01:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-12-05 12:27:39 +01:00
"## Case Study (continued): Train/Test Split for the Iris data"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2018-03-21 17:37:55 +01:00
"It is common practice to refer to the feature matrix as X and the vector of labels as y."
]
},
{
"cell_type": "code",
2019-12-05 12:27:39 +01:00
"execution_count": 28,
2018-03-21 17:37:55 +01:00
"metadata": {},
"outputs": [],
"source": [
"X, y = iris.data, iris.target"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"A naive splitting approach could be to use array slicing."
]
},
{
"cell_type": "code",
2019-12-05 12:27:39 +01:00
"execution_count": 29,
2018-03-21 17:37:55 +01:00
"metadata": {},
"outputs": [],
"source": [
"X_train, X_test, y_train, y_test = X[0:100, :], X[100:150, :], y[0:100], y[100:150]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"However, this would lead to unbalanced label distributions. For example, the test set would only be made up of flowers of the same type."
]
},
{
"cell_type": "code",
2019-12-05 12:27:39 +01:00
"execution_count": 30,
2018-03-21 17:42:59 +01:00
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
" 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
" 2, 2, 2, 2, 2, 2])"
]
},
2019-12-05 12:27:39 +01:00
"execution_count": 30,
2018-03-21 17:42:59 +01:00
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"y_test"
]
},
{
"cell_type": "code",
2019-12-05 12:27:39 +01:00
"execution_count": 31,
2018-03-21 17:42:59 +01:00
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([ 0, 0, 50])"
]
},
2019-12-05 12:27:39 +01:00
"execution_count": 31,
2018-03-21 17:42:59 +01:00
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"np.bincount(y_test)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"sklearn provides a function that not only randomizes the split but also ensures that the resulting label distribution is proportionate to the overall distribution (called **stratification**)."
]
},
{
"cell_type": "code",
2019-12-05 12:27:39 +01:00
"execution_count": 32,
"metadata": {},
"outputs": [],
"source": [
"from sklearn.model_selection import train_test_split"
]
},
{
"cell_type": "code",
"execution_count": 33,
2018-03-21 17:42:59 +01:00
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2019-12-05 12:27:39 +01:00
"array([1, 0, 1, 2, 2, 2, 2, 1, 0, 1, 0, 0, 2, 0, 0, 1, 0, 1, 2, 2, 0, 2,\n",
" 1, 2, 0, 0, 2, 2, 1, 2, 1, 1, 0, 0, 1, 1, 1, 0, 1, 2, 0, 0, 2, 2,\n",
" 1])"
2018-03-21 17:42:59 +01:00
]
},
2019-12-05 12:27:39 +01:00
"execution_count": 33,
2018-03-21 17:42:59 +01:00
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
2018-12-05 13:28:52 +01:00
"X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.7, test_size=0.3, stratify=y)\n",
2019-12-05 12:27:39 +01:00
"\n",
2018-03-21 17:37:55 +01:00
"y_test"
]
},
{
"cell_type": "code",
2019-12-05 12:27:39 +01:00
"execution_count": 34,
2018-03-21 17:42:59 +01:00
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([15, 15, 15])"
]
},
2019-12-05 12:27:39 +01:00
"execution_count": 34,
2018-03-21 17:42:59 +01:00
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"np.bincount(y_test)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-12-05 12:27:39 +01:00
"## A simple Classification Model: k-Nearest Neighbors"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To predict the label for any observation, just determine the k \"nearest\" observations in the training set (e.g., by Euclidean distance) and use a simple majority vote."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-10-02 09:06:44 +02:00
"<img src=\"raw/knn.png\" width=\"60%\">"
2018-03-21 17:37:55 +01:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-12-05 12:27:39 +01:00
"## Case Study (continued): Train and Predict the Iris data"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2018-03-21 17:37:55 +01:00
"sklearn provides a uniform interface for all its classification models. They all have a **fit()** and a **predict()** method that abstract away the actual machine learning algorithm."
]
},
{
"cell_type": "code",
2019-12-05 12:27:39 +01:00
"execution_count": 35,
"metadata": {},
"outputs": [],
"source": [
"from sklearn.neighbors import KNeighborsClassifier"
]
},
{
"cell_type": "code",
"execution_count": 36,
2018-03-21 17:37:55 +01:00
"metadata": {},
"outputs": [],
"source": [
"knn = KNeighborsClassifier(n_neighbors=5)\n",
"\n",
"knn.fit(X_train, y_train)\n",
"\n",
"y_pred = knn.predict(X_test)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let us list the labels predicted for the test set ..."
]
},
{
"cell_type": "code",
2019-12-05 12:27:39 +01:00
"execution_count": 37,
2018-03-21 17:42:59 +01:00
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2019-12-05 12:27:39 +01:00
"array([1, 0, 1, 1, 2, 2, 2, 1, 0, 1, 0, 0, 2, 0, 0, 1, 0, 1, 2, 2, 0, 2,\n",
" 1, 1, 0, 0, 1, 2, 1, 2, 1, 1, 0, 0, 1, 1, 1, 0, 1, 2, 0, 0, 2, 2,\n",
" 1])"
2018-03-21 17:42:59 +01:00
]
},
2019-12-05 12:27:39 +01:00
"execution_count": 37,
2018-03-21 17:42:59 +01:00
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"y_pred"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"... and compare them with the actual labels."
]
},
{
"cell_type": "code",
2019-12-05 12:27:39 +01:00
"execution_count": 38,
2018-03-21 17:42:59 +01:00
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2019-12-05 12:27:39 +01:00
"array([1, 0, 1, 2, 2, 2, 2, 1, 0, 1, 0, 0, 2, 0, 0, 1, 0, 1, 2, 2, 0, 2,\n",
" 1, 2, 0, 0, 2, 2, 1, 2, 1, 1, 0, 0, 1, 1, 1, 0, 1, 2, 0, 0, 2, 2,\n",
" 1])"
2018-03-21 17:42:59 +01:00
]
},
2019-12-05 12:27:39 +01:00
"execution_count": 38,
2018-03-21 17:42:59 +01:00
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"y_test"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Numpy can show us the indices where the predictions are wrong."
]
},
{
"cell_type": "code",
2019-12-05 12:27:39 +01:00
"execution_count": 39,
2018-03-21 17:42:59 +01:00
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2019-12-05 12:27:39 +01:00
"(array([ 3, 23, 26]),)"
2018-03-21 17:42:59 +01:00
]
},
2019-12-05 12:27:39 +01:00
"execution_count": 39,
2018-03-21 17:42:59 +01:00
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"np.where(y_pred != y_test)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Alternatively, we can calculate the fraction of correctly predicted flowers."
]
},
{
"cell_type": "code",
2019-12-05 12:27:39 +01:00
"execution_count": 40,
2018-03-21 17:42:59 +01:00
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2019-12-05 12:27:39 +01:00
"0.9333333333333333"
2018-03-21 17:42:59 +01:00
]
},
2019-12-05 12:27:39 +01:00
"execution_count": 40,
2018-03-21 17:42:59 +01:00
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"np.sum(y_pred == y_test) / len(y_test)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"It is important to mention that we can also \"predict\" the training set. Surprisingly, the model does not get the training set 100% correct."
]
},
{
"cell_type": "code",
2019-12-05 12:27:39 +01:00
"execution_count": 41,
2018-03-21 17:42:59 +01:00
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2019-12-05 12:27:39 +01:00
"0.9523809523809523"
2018-03-21 17:42:59 +01:00
]
},
2019-12-05 12:27:39 +01:00
"execution_count": 41,
2018-03-21 17:42:59 +01:00
"metadata": {},
"output_type": "execute_result"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"y_train_pred = knn.predict(X_train)\n",
2019-12-05 12:27:39 +01:00
"\n",
2018-03-21 17:37:55 +01:00
"np.sum(y_train_pred == y_train) / len(y_train)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"A visualization reveals that the misclassified flowers are right \"at the borderline\" between two neighboring clusters of flower classes."
]
},
{
"cell_type": "code",
2019-12-05 12:27:39 +01:00
"execution_count": 42,
2018-03-21 17:42:59 +01:00
"metadata": {},
"outputs": [
{
"data": {
2019-12-05 12:27:39 +01:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEWCAYAAABi5jCmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3de3xU1bnw8d8TEoSoRBA8IpgEjwIFwh0NgtcIXsH6Ci/2RBS1RsVLfVsVbHq8cMipF1S8VGm0CtK0gigqtioYRcWKCghEIyBqwk0hAgYwogGe94+9J50Jk8nMJHPLPN/PZz7MrL332k92wjyz11qzlqgqxhhjjEdKrAMwxhgTXywxGGOM8WGJwRhjjA9LDMYYY3xYYjDGGOPDEoMxxhgflhhMsxGRfBFZGIF6J4jIkuau16v+10Tkcq/XU0XkOxH5VkQyRWSPiLSKwHn3iMhxzV1vtIjIXSLy11jHYZqfJQYTNBGpEJGzGtquqiWqOjLMus8WkXdFZLeIVInIOyIyOvxog6eq56rqLDeOTOB3QC9VPVpVN6jqYaq6vynnEJHFIvLreuc9TFW/akq98UJEskVERSQ11rGYprPEYJpFU94QRGQM8DzwLNAV+A/gDmBU80QXkkxgu6pui8G5m529UZtwWGIwYXGbd94XkYdEZDtwl3eTjzgeEpFtIrJLRMpEpI+fegR4EPgfVX1KVatV9YCqvqOqVzdw7odFZKNb73IROcVr24kisszdtlVEHnTL24jIX0Vku4h8LyIfi8h/uNsWi8iv3buhRcAxbjPPzPqfhEWkg4g8IyJbRGSniLzklrcXkVfdu52d7vOu7rYi4BTgMbfex9xyFZHj3ecZIvKse3yliPxBRFK8rvUSEZnm1v21iJwb4HdTISKTRGQ18IOIpIrIMSLyglv/1yJyUxDX7HQR2eSnbn93je+6/37v/oxDReR4986v2m2am9NQzCa+WGIwTXES8BXOJ/yiettGAqcC3YEM4P8C2/3U0QM4FpgXwnk/BvoDHYC/Ac+LSBt328PAw6raDvhPYK5bfrkbx7HAkcC1wI/elarqm8C5wBa3mWeCn3PPBtKB3sBRwENueQrwDJCFc9fxI/CYW28h8B5wg1vvDX7qfdSN7zjgNOAy4Aqv7ScBa4GOwH3AX9yk2pBfAecDRwAHgAXAKqALkAfcLCJnu/s2dM1Ccar77xHuz/gB8D/AQqA9zp3go2HUa2LAEoNpii2q+qiq7lPVH+ttqwUOB3oCoqqfq+o3fuo40v3X3za/VPWvqrrdPe8DwCE4CcZz3uNFpKOq7lHVpV7lRwLHq+p+VV2uqruCPSeAiHTGSRzXqupOVa1V1XfcmLar6guqWqOqu3ES5WlB1tsKuAS4XVV3q2oF8AAw3mu3SlV90u3rmAV0xknIDXlEVTe6v5chQCdVnaKqP7v9Gk+654SGr1lT1eIkymNUda+qRmwAgWlelhhMU2xsaIOqvoXziflPwDYRKRaRdn529dxFdA72pCJyi4h87jZRfI/zSbuju/kqnLuUNW5z0QVu+WzgDeA5txnoPhFJC/acrmOBHaq6009M6SLyZ7cZaBdO08oREtxopo5AGlDpVVaJ8+ne41vPE1WtcZ8eFqBO799NFk7z2PeeB/B7/p1YGrpmTXUbIMBHIvKZiFzZTPWaCLPEYJoi4NS8qvqIqg4CeuG88dzqZ7e1OG9iFwdzQrc/4Tacpqn2qnoEUI3zBoSqfqGqv8Jp5rkXmCcih7qf7u9W1V7AycAFOM01odgIdBCRI/xs+x3OXctJbpOMp2nF09wT6Fp9x78/XXtkAptDjM+b9/k2Al+r6hFej8NV9Txo+JoBP+A0mzk/iJPkOgVxPtx6v1XVq1X1GOAa4HFPn4qJb5YYTESIyBAROcn9VP4DsBenrduHOvO+/xb4bxG5QkTaiUiKiAwXkWI/VR8O7AOqgFQRuQOouxMRkUtFpJOqHgC+d4sPiMgZIpLjvrntwnkjPiieQNymsNdw3uDai0iaiHgSwOE4/Qrfi0gH4M56h2/F6T/wV+9+nHb9IhE5XESy3GvSXN8R+AjY7XZItxWRViLSR0SGQMPXDFgHtBGR893f4x9wmu38qXKPqfsZRWSspwMe2ImTPEK65iY2LDGYSGmH0469E6dZZDtwv78dVXUeMA64EtiC8yY6FXjZz+5vAK/jvGlV4iQc72aTc4DPRGQPTqfqJW47+9E4Hdy7gM+Bd3Cal0I1HieprAG2ATe75dOBtjif/pe6MXp7GBjjjip6xE+9N+Ik0K+AJTid6k+HEd9B3MRzAU6H/ddujE/hNMFBA9dMVauBie6+m934NuGH27xVBLzvNlfl4vRtfOjW+wrwm5byvY2WTmyhHmOMMd7sjsEYY4wPSwzGGGN8WGIwxhjjwxKDMcYYHwk3wVbHjh01Ozs71mEYY0xCWb58+Xeq2tD3UHwkXGLIzs5m2bJlsQ7DGGMSiohUNr6Xw5qSjDHG+LDEYIwxxoclBmOMMT4Sro/Bn9raWjZt2sTevXtjHYoB2rRpQ9euXUlLC3XyUmNMPGgRiWHTpk0cfvjhZGdnE3jtEhNpqsr27dvZtGkT3bp1i3U4xpgwtIimpL1793LkkUdaUogDIsKRRx5pd2/GJLAWkRgASwpxxH4XxiS2FpMYjDHGNI8W0ceQaGbOnMnIkSM55phjYh2KMSYMZWVllJaWUl1dTUZGBnl5eeTk5DT7MbFidwwxMHPmTLZs2RLrMIwxYSgrK2PBggVUV1cDUF1dzYIFCygrK2vWY2IpKRNDSUk52dnFpKRMIzu7mJKS8ibX+cMPP3D++efTr18/+vTpw5w5c1i+fDmnnXYagwYN4uyzz+abb75h3rx5LFu2jPz8fPr378+PP/5IaWkpAwYMICcnhyuvvJKffvoJgMmTJ9OrVy/69u3LLbfcAsCCBQs46aSTGDBgAGeddRZbt25tcuzGmOCVlpZSW1vrU1ZbW0tpaWmzHhNLSdeUVFJSTkHBQmpq9gFQWbmLgoKFAOTn9wq73tdff51jjjmGf/zjH4DzieDcc8/l5ZdfplOnTsyZM4fCwkKefvppHnvsMaZNm8bgwYPZu3cvEyZMoLS0lO7du3PZZZfxxBNPMH78eObPn8+aNWsQEb7/3lmKd/jw4SxduhQR4amnnuK+++7jgQceaOJVMcYEy/OpP9jycI+JpaS7YygsXFKXFDxqavZRWLikSfXm5OSwaNEiJk2axHvvvcfGjRv59NNPGTFiBP3792fq1Kls2nTwcrlr166lW7dudO/eHYDLL7+cd999l4yMDNq0acNVV13Fiy++SHp6OuB8Z+Pss88mJyeH+++/n88++6xJcRtjQpORkRFSebjHxFLSJYYNG3aFVB6s7t27s2LFCnJycvjDH/7ACy+8QO/evVm5ciUrV66krKyMhQsXBl1famoqH330EWPGjOHVV1/lnHPOAeDGG2/khhtuoKysjD//+c/2fQFjoiwvL++gb/WnpaWRl5fXrMfEUtIlhszMdiGVB2vLli2kp6dz6aWXcuutt/Lhhx9SVVXFBx98ADjtiZ5P94cffji7d+8GoEePHlRUVLB+/XoAZs+ezWmnncaePXuorq7mvPPO46GHHmLVqlWAc+vZpUsXAGbNmtWkmI0xocvJyWHUqFF1n/YzMjIYNWpUwBFG4RwTS0nXx1BUNNynjwEgPT2VoqLhTaq3rKyMW2+9lZSUFNLS0njiiSdITU3lpptuorq6mn379nHzzTfTu3dvJkyYwLXXXkvbtm354IMPeOaZZxg7diz79u1jyJAhXHvttezYsYMLL7yQvXv3oqo8+OCDANx1112MHTuW9u3bc+aZZ/L11183KW5jTOhycnJCflMP55hYEVWNdQwhGTx4sNZfqOfzzz/nF7/4RdB1lJSUU1i4hA0bdpGZ2Y6iouFN6ng2Bwv1d2KMiSwRWa6qg4PZN+nuGMAZfWSJwBhj/Eu6PgZjjDGBRSwxiEgPEVnp9dglIjfX2+d0Ean22ueOSMVjjDEmOBFrSlLVtUB/ABFpBWw
2018-03-21 17:42:59 +01:00
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
2018-12-05 13:28:52 +01:00
"metadata": {
"needs_background": "light"
},
2018-03-21 17:42:59 +01:00
"output_type": "display_data"
}
],
2018-03-21 17:37:55 +01:00
"source": [
"first_feature_index = 3\n",
"second_feature_index = 0\n",
"\n",
"correct_idx = np.where(y_pred == y_test)[0]\n",
"incorrect_idx = np.where(y_pred != y_test)[0]\n",
"\n",
"colors = [\"darkblue\", \"darkgreen\", \"gray\"]\n",
"\n",
"for n, color in enumerate(colors):\n",
" idx = np.where(y_test == n)[0]\n",
" plt.scatter(X_test[idx, first_feature_index], X_test[idx, second_feature_index], color=color,\n",
" label=iris.target_names[n])\n",
"\n",
"plt.scatter(X_test[incorrect_idx, first_feature_index], X_test[incorrect_idx, second_feature_index],\n",
" color=\"darkred\", label='misclassified')\n",
"\n",
"plt.xlabel('sepal width [cm]')\n",
"plt.ylabel('petal length [cm]')\n",
"plt.legend(loc='best')\n",
"plt.title(\"Iris Classification results\")\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In practice, the number of neighbors is to chosen before the model is trained. Therefore, it is possible to \"optimize\" it. This process is referred to as **hyper-parameter** tuning. For the Iris dataset this does not make much of a difference."
]
},
{
"cell_type": "code",
2019-12-05 12:27:39 +01:00
"execution_count": 43,
2018-03-21 17:42:59 +01:00
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
2019-12-05 12:27:39 +01:00
"1 0.9777777777777777\n",
2019-10-02 09:06:44 +02:00
"2 0.9333333333333333\n",
"3 0.9555555555555556\n",
2019-12-05 12:27:39 +01:00
"4 0.8888888888888888\n",
"5 0.9333333333333333\n",
"6 0.9333333333333333\n",
"7 0.9333333333333333\n",
"8 0.9333333333333333\n",
"9 0.9555555555555556\n",
2019-10-02 09:06:44 +02:00
"10 0.9333333333333333\n",
2019-12-05 12:27:39 +01:00
"11 0.9555555555555556\n",
2019-10-02 09:06:44 +02:00
"12 0.9333333333333333\n",
"13 0.9333333333333333\n",
"14 0.9333333333333333\n",
"15 0.9333333333333333\n",
"16 0.9333333333333333\n",
"17 0.9333333333333333\n",
"18 0.9333333333333333\n",
"19 0.9333333333333333\n",
"20 0.9333333333333333\n",
"21 0.9333333333333333\n",
2019-12-05 12:27:39 +01:00
"22 0.9333333333333333\n",
2019-10-02 09:06:44 +02:00
"23 0.9111111111111111\n",
2019-12-05 12:27:39 +01:00
"24 0.9333333333333333\n",
2019-10-02 09:06:44 +02:00
"25 0.9111111111111111\n",
2019-12-05 12:27:39 +01:00
"26 0.9333333333333333\n",
2019-10-02 09:06:44 +02:00
"27 0.9111111111111111\n",
"28 0.9111111111111111\n",
2019-12-05 12:27:39 +01:00
"29 0.9333333333333333\n",
"30 0.9333333333333333\n"
2018-03-21 17:42:59 +01:00
]
}
],
2018-03-21 17:37:55 +01:00
"source": [
"for i in range(1, 31):\n",
" knn = KNeighborsClassifier(n_neighbors=i)\n",
" knn.fit(X_train, y_train)\n",
" y_pred = knn.predict(X_test)\n",
" correct = np.sum(y_pred == y_test) / len(y_test)\n",
2018-12-05 13:28:52 +01:00
" print(i, correct)"
2018-03-21 17:37:55 +01:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-12-05 12:27:39 +01:00
"## WHU's Python Course in the BSc program"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"- free [online book](https://github.com/webartifex/intro-to-python) by the author of this workshop"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Literature on Machine Learning"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Depending on the programming language one chooses, the following books are recommended:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"- [Python Machine Learning](https://www.amazon.de/Python-Machine-Learning-scikit-learn-TensorFlow/dp/1787125939/ref=sr_1_1?__mk_de_DE=%C3%85M%C3%85%C5%BD%C3%95%C3%91&keywords=python+machine+learning&qid=1575545025&sr=8-1) by Sebastian Raschka\n",
2018-03-21 17:37:55 +01:00
"\n",
2019-12-05 12:27:39 +01:00
"<img src=\"raw/python_ml.png\">"
2018-03-21 17:37:55 +01:00
]
2019-10-02 09:06:44 +02:00
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-12-05 12:27:39 +01:00
"- [An Introduction to Statistical Learning](http://faculty.marshall.usc.edu/gareth-james/ISL/)\n",
"\n",
"<img src=\"raw/r.png\">"
2019-10-02 09:06:44 +02:00
]
2018-03-21 17:37:55 +01:00
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
2019-12-05 12:27:39 +01:00
"version": "3.6.9"
2019-10-02 09:06:44 +02:00
},
"toc": {
"base_numbering": 1,
"nav_menu": {},
"number_sections": false,
"sideBar": true,
"skip_h1_title": false,
"title_cell": "Table of Contents",
"title_sidebar": "Contents",
"toc_cell": false,
"toc_position": {},
"toc_section_display": true,
"toc_window_display": false
2018-03-21 17:37:55 +01:00
}
},
"nbformat": 4,
"nbformat_minor": 2
}
