"**Note**: Click on \"*Kernel*\" > \"*Restart Kernel and Clear All Outputs*\" in [JupyterLab](https://jupyterlab.readthedocs.io/en/stable/) *before* reading this notebook to reset its output. If you cannot run this file on your machine, you may want to open it [in the cloud <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_mb.png\">](https://mybinder.org/v2/gh/webartifex/intro-to-python/develop?urlpath=lab/tree/11_classes/03_content.ipynb)."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Chapter 11: Classes & Instances (continued)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"The implementations of our `Vector` and `Matrix` classes so far do not know any of the rules of [linear algebra <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_wiki.png\">](https://en.wikipedia.org/wiki/Linear_algebra). In this third part of the chapter, we add a couple of typical vector and matrix operations, for example, vector addition or matrix-vector multiplication, and some others. Before we do so, we briefly talk about how we can often model the *same* underlying data with a *different* data type."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Representations of Data"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"> \"If you change the way you look at things, the things you look at change.\"\n",
"> -- philosopher and personal coach [Dr. Wayne Dyer <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_wiki.png\">](https://en.wikipedia.org/wiki/Wayne_Dyer)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"Sometimes, it is helpful to view a `Vector` as a `Matrix` with either one row or one column. On the contrary, such a `Matrix` can always be interpreted as a `Vector` again. Changing the representation of the same underlying data (i.e., the `_entries`) can be viewed as \"changing\" an object's data type, for which, however, there is no built-in syntax.\n",
"\n",
"Thus, we implement the `.as_matrix()` and `.as_vector()` methods below that create *new* `Matrix` or `Vector` instances out of existing `Vector` or `Matrix` instances, respectively. Internally, both methods rely on the sequence protocol again (i.e., `for x in self`). Also, `.as_matrix()` interprets the `Vector` instance as a column vector by default (i.e., the `column=True` flag)."
"Interpreting a matrix as a vector only works if one of the two dimensions, $m$ or $n$, is $1$. If this requirement is not satisfied, we get the `RuntimeError` raised in `.as_vector()` above."
"\u001b[0;31mRuntimeError\u001b[0m: one dimension (m or n) must be 1"
]
}
],
"source": [
"m.as_vector()"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Operator Overloading"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"By implementing special methods such as `.__add__()`, `.__sub__()`, `.__mul__()`, and some others, we can make user-defined data types emulate how numeric types operate with each other (cf., [reference <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/reference/datamodel.html#emulating-numeric-types)): Then, `Vector` and `Matrix` instances can be added together, subtracted from one another, or be multiplied together. We use them to implement the arithmetic rules from linear algebra.\n",
"\n",
"The OOP concept behind this is **[operator overloading <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_wiki.png\">](https://en.wikipedia.org/wiki/Operator_overloading)** as first mentioned in the context of `str` concatenation in [Chapter 1 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/develop/01_elements/00_content.ipynb#Operator-Overloading)."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Arithmetic Operators"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"To understand the protocol behind arithmetic operators, we first look at the simple case of how an `int` object and a `float` object are added. The expression `1 + 2.0` is \"translated\" by Python into a method invocation of the form `1.__add__(2.0)`. This is why all the special methods behind binary operators take two arguments, `self` and, by convention, `other`. To allow binary operators to work with objects of *different* data types, Python expects the `.__add__()` method on the `1` object to return `NotImplemented` if it does not know how to deal with the `2.0` object and then proceeds by invoking the *reverse* special method `2.0.__radd__(1)`. With this protocol, one can create *new* data types that know how to execute arithmetic operations with *existing* data types *without* having to change the latter. By convention, the result of a binary operation should always be a *new* instance object and *not* a mutation of an existing one."
"Before implementing the arithmetic operators, we must first determine what other data types are allowed to interact with our `Vector` and `Matrix` instances and also how the two interact with each other. Conceptually, this is the same as to ask how strict we want the rules from linear algebra to be enforced in our model world. For example, while it is obvious that two vectors with the same number of entries may be added or subtracted, we could also allow a scalar value to be added to a vector. That seems awkward at first because it is an illegal operation in linear algebra. However, for convenience in our programs, we could interpret any scalar as a \"constants\" vector of the \"right size\" and add it to each entry in a `Vector`. This idea can be generalized into what is called **[broadcasting](https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html)** in [numpy](https://docs.scipy.org/doc/numpy/index.html). We often see \"dirty hacks\" like this in code. They are no bugs but features supposed to make the user of a library more productive.\n",
"\n",
"In this chapter, we model the following binary arithmetic operations:"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"- **Addition** / **Subtraction**\n",
" - `Vector` with `Vector` (if number of entries match; commutative)\n",
" - `Matrix` with `Matrix` (if dimensions $m$ and $n$ match; commutative)\n",
" - `Matrix` / `Vector` with scalar (the scalar is broadcasted; non-commutative for subtraction)\n",
"- **Multiplication**\n",
" - `Vector` with `Vector` ([dot product <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_wiki.png\">](https://en.wikipedia.org/wiki/Dot_product) if number of entries match; commutative)\n",
" - `Matrix` with `Vector` (if dimensions are compatible; vector interpreted as column vector; non-commutative)\n",
" - `Vector` with `Matrix` (if dimensions are compatible; vector interpreted as row vector; non-commutative)\n",
" - `Matrix` with `Matrix` ([matrix-matrix multiplication <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_wiki.png\">](https://en.wikipedia.org/wiki/Matrix_multiplication) if dimensions are compatible; generally non-commutative)\n",
"- **Division**\n",
" - `Matrix` / `Vector` by a scalar (inverse of [scalar multiplication <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_wiki.png\">](https://en.wikipedia.org/wiki/Scalar_multiplication); non-commutative)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"This listing shows the conceptual complexity behind the task of writing a \"little\" linear algebra library. Not to mention that some of the operations are [commutative <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_wiki.png\">](https://en.wikipedia.org/wiki/Commutative_property) while others are not.\n",
"\n",
"As the available special methods correspond to the high-level grouping in the listing, we must implement a lot of **type dispatching** within them. This is why you see the built-in [isinstance() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#isinstance) function in most of the methods below. We use it to check if the `other` argument passed in is a `Vector` or `Matrix` instance or a scalar."
"To check if `other` is a scalar, we need to specify what data type constitutes a scalar. We use a goose typing strategy as explained in [Chapter 5 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/develop/05_numbers/02_content.ipynb#Goose-Typing): Any object that behaves like a `numbers.Number` from the [numbers <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/numbers.html) module in the [standard library <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/index.html) is considered a scalar.\n",
"\n",
"For example, the integer `1` is an instance of the built-in `int` type. At the same time, [isinstance() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#isinstance) also confirms that it is a `numbers.Number` in the abstract sense."
"Now with all the preparation work done, let's look at a \"minimal\" implementation of `Vector` that supports all the arithmetic operations specified above. *None* of the special methods inside the `Vector` class is aware that the `Matrix` class exists! Thus, all operations involving at least one `Matrix` instance are implemented only in the `Matrix` class."
"`.__mul__()` implements both scalar multiplication and the dot product of two `Vector`s. As both operations are commutative, `.__rmul__()` dispatches to `.__mul__()` via the `self * other` expression."
"`.__truediv__()` implements the ordinary division operator `/` while `.__floordiv__()` would implement the integer division operator `//`. Here, `.__truediv__()` dispatches to `.__mul__()` after inverting the `other` argument via the `self * (1 / other)` expression."
"`.__add__()` and `.__sub__()` implement vector addition and subtraction according to standard linear algebra rules, meaning that both `Vector`s must have the same number of entries or a `ValueError` is raised. Furthermore, both methods are able to broadcast the `other` argument to the dimension of a `Vector` and then execute either vector addition or subtraction. As addition is commutative, `.__radd__()` dispatches to `.__add__()`. For now, we have to explicitly implement `.__rsub__()`. Further below, we see how it can be re-factored to be commutative."
"For `Matrix` instances, the implementation is a bit more involved as we need to distinguish between matrix-matrix, matrix-vector, vector-matrix, and scalar multiplication and check for compatible dimensions. To review the underlying rules, check this [article <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_wiki.png\">](https://en.wikipedia.org/wiki/Matrix_multiplication) or watch the video below."
"<IPython.lib.display.YouTubeVideo at 0x7fa62c6bfa90>"
]
},
"execution_count": 34,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from IPython.display import YouTubeVideo\n",
"YouTubeVideo(\"OMA2Mwo0aZg\", width=\"60%\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"To summarize the video, the multiplication of two matrices $\\bf{A}$ and $\\bf{B}$ with dimensions $m$ by $n$ and $n$ by $p$ yields a matrix $\\bf{C}$ with dimensions $m$ and $p$. To obtain an entry $c_{ij}$ of matrix $\\bf{C}$ where $i$ and $j$ represent the index labels of the rows and columns, we have to calculate the dot product of the $i$th row vector of $\\bf{A}$ with the $j$th column vector of $\\bf{B}$. So, it makes a difference if we multiply $\\bf{A}$ with $\\bf{B}$ from the right or left.\n",
"\n",
"When multiplying a `Matrix` with a `Vector`, we follow the convention that a `Vector` on the left is interpreted as a row vector and a `Vector` on the right as a column vector. The `Vector`'s length must match the `Matrix`'s corresponding dimension. As row and column vectors can be viewed as a `Matrix` as well, matrix-vector multiplication is a special case of matrix-matrix multiplication."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"In the revised `Matrix` class, the `.__add__()`, `.__radd__()`, `.__sub__()`, `.__rsub__()`, and `.__truediv__()` methods follow the same logic as in the `Vector` class above.\n",
"\n",
"Besides implementing scalar multiplication, `.__mul__()` and `.__rmul__()` are responsible for converting `Vector`s into `Matrix`s and back. In particular, all the different ways of performing a multiplication are reduced into *one* generic form, which is a matrix-matrix multiplication. That is achieved by the `._matrix_multiply()` method, another implementation detail."
"The same holds for matrix-vector and vector-matrix multiplication. These operations always return `Vector` instances in line with standard linear algebra. If a `Vector`'s length is not compatible with the respective dimension of a `Matrix`, we receive a `ValueError`."
"We do not allow addition or subtraction of matrices with vectors and raise a `TypeError` instead. Alternatively, we could have implemented broadcasting here as well, just like [numpy](https://www.numpy.org/) does."
"\u001b[0;31mTypeError\u001b[0m: vectors and matrices cannot be added"
]
}
],
"source": [
"m + v"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Relational Operators"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"As we have seen before, two different `Vector`s with the same `._entries` do *not* compare equal. The reason is that for user-defined types Python by default only assumes two instances to be equal if they are actually the same object. This is, of course, semantically wrong for our `Vector`s and `Matrix`s."
]
},
{
"cell_type": "code",
"execution_count": 49,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [],
"source": [
"v = Vector([1, 2, 3])\n",
"w = Vector([1, 2, 3])"
]
},
{
"cell_type": "code",
"execution_count": 50,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"text/plain": [
"False"
]
},
"execution_count": 50,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"v == w"
]
},
{
"cell_type": "code",
"execution_count": 51,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 51,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"v == v"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"We implement the `.__eq__()` (cf., [reference <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/reference/datamodel.html#object.__eq__)) method to control how the comparison operator `==` is carried out. For brevity, we show this only for the `Vector` class. The `.__eq__()` method exits early as soon as the first pair of entries does not match. Also, for reasons discussed in [Chapter 5 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/develop/05_numbers/01_content.ipynb#Imprecision), we compare the absolute difference of two corresponding entries to a very small `zero_threshold` that is stored as a class attribute shared among all `Vector` instances. If the `Vector`s differ in their numbers of entries, we fail loudly and raise a `ValueError`."
" return False # exit early if two corresponding entries differ\n",
" return True\n",
" return NotImplemented"
]
},
{
"cell_type": "code",
"execution_count": 53,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [],
"source": [
"v = Vector([1, 2, 3])\n",
"w = Vector([1, 2, 3])"
]
},
{
"cell_type": "code",
"execution_count": 54,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 54,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"v == w"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"Besides `.__eq__()`, there are other special methods to implement the `!=`, `<`, `>`, `<=`, and `>=` operators, the last four of which are not semantically meaningful in the context of linear algebra.\n",
"\n",
"Python implicitly creates the `!=` for us in that it just inverts the result of `.__eq__()`."
"Our `Vector` and `Matrix` classes do not fully behave like a `numbers.Number` in the abstract sense. Besides the not yet talked about but useful unary `+` and `-` operators, numbers in Python usually support being passed to built-in functions like [abs() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#abs), [bin() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#bin), [bool() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#bool), [complex() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#complex), [float() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#float), [hex() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#hex), [int() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#int), and others (cf., the [reference <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/reference/datamodel.html#emulating-numeric-types) for an exhaustive list).\n",
"\n",
"To see that our classes lack simple but expected behaviors, let's try to invert the signs of all entries in the vector `v` ..."
"\u001b[0;31mTypeError\u001b[0m: bad operand type for abs(): 'Vector'"
]
}
],
"source": [
"abs(v)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"For our example, we decide to implement the unary `+` and `-` operators, [abs() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#abs) as the Euclidean / Frobenius norm (i.e., the [magnitude <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_wiki.png\">](https://en.wikipedia.org/wiki/Magnitude_%28mathematics%29#Euclidean_vector_space)), [bool() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#bool) to check if that norm is greater than `0` or not, and [float() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#float) to obtain a single scalar if the vector or matrix consists of only one entry. To achieve that, we implement the `.__pos__()`, `.__neg__()`, `.__abs__()`, `.__bool__()`, and `.__float__()` methods. For brevity, we do this only for the `Vector` class."
"Only an all `0`s `Vector` is `False` in a boolean context. As mentioned in [Chapter 3 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/develop/03_conditionals/00_content.ipynb#Truthy-vs.-Falsy), commonly we view an *empty* sequence as falsy; however, as we do not allow `Vector`s without any entries, we choose the all `0`s alternative. In that regard, the `Vector` class does not behave like the built-in sequence types."
"Cell \u001b[0;32mIn[75], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[38;5;28;43mfloat\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mv\u001b[49m\u001b[43m)\u001b[49m\n",
"Cell \u001b[0;32mIn[63], line 31\u001b[0m, in \u001b[0;36mVector.__float__\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m 29\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m__float__\u001b[39m(\u001b[38;5;28mself\u001b[39m):\n\u001b[1;32m 30\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mlen\u001b[39m(\u001b[38;5;28mself\u001b[39m) \u001b[38;5;241m!=\u001b[39m \u001b[38;5;241m1\u001b[39m:\n\u001b[0;32m---> 31\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mRuntimeError\u001b[39;00m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mvector must have exactly one entry to become a scalar\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[1;32m 32\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_entries[\u001b[38;5;241m0\u001b[39m]\n",
