"**Note**: Click on \"*Kernel*\" > \"*Restart Kernel and Clear All Outputs*\" in [JupyterLab](https://jupyterlab.readthedocs.io/en/stable/) *before* reading this notebook to reset its output. If you cannot run this file on your machine, you may want to open it [in the cloud <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_mb.png\">](https://mybinder.org/v2/gh/webartifex/intro-to-python/develop?urlpath=lab/tree/11_classes/04_content.ipynb)."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Chapter 11: Classes & Instances (continued)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"In this fourth part of the chapter, we finalize our `Vector` and `Matrix` classes. As both `class` definitions have become rather lengthy, we learn how we to organize them into a Python package and import them in this Jupyter notebook. "
"In [Chapter 2 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/develop/02_functions/02_content.ipynb#Local-Modules-and-Packages), we introduce the concept of a Python module that is imported with the `import` statement. Essentially, a **module** is a single plain text \\*.py file on disk that contains Python code (e.g., [*sample_module.py* <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/blob/develop/02_functions/sample_module.py) in [Chapter 2's folder <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/tree/develop/02_functions)).\n",
"\n",
"Conceptually, a **package** is a generalization of a module whose code is split across several \\*.py to achieve a better organization of the individual parts. The \\*.py files are stored within a folder (e.g., [*sample_package* <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/tree/develop/11_classes/sample_package) in [Chapter 11's folder <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/tree/develop/11_classes)). In addition to that, a \"*\\_\\_init\\_\\_.py*\" file that may be empty must be put inside the folder. The latter is what the Python interpreter looks for to decide if a folder is a package or not.\n",
"\n",
"Let's look at an example with the final version of our `Vector` and `Matrix` classes.\n",
"\n",
"`!pwd` shows the location of this Jupyter notebook on the computer you are running [JupyterLab](https://jupyterlab.readthedocs.io/en/stable/) on: It is the local equivalent of [Chapter 11's folder <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/tree/develop/11_classes) in this book's [GitHub repository <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python)."
"`!ls` lists all the files and folders in the current location: These are Chapter 11's Jupyter notebooks (i.e., the \\*.ipynb files) and the [*sample_package* <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/tree/develop/11_classes/sample_package) folder. "
"If we run `!ls` with the `sample_package` folder as the argument, we see the folder's contents: Four \\*.py files. Alternatively, you can use [JupyterLab' File Browser](https://jupyterlab.readthedocs.io/en/stable/user/interface.html?highlight=file%20browser#left-sidebar) on the left to navigate into the package."
"The package is organized such that the [*matrix.py* <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/matrix.py) and [*vector.py* <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/vector.py) modules each define just one class, `Matrix` and `Vector`. That is intentional as both classes consist of several hundred lines of code and comments.\n",
"\n",
"The [*utils.py* <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/utils.py) module contains code that is shared by both classes. Such code snippets are commonly called \"utilities\" or \"helpers,\" which explains the module's name.\n",
"\n",
"Finally, the [*\\_\\_init\\_\\_.py* <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/__init__.py) file contains mostly meta information and defines what objects should be importable from the package's top level.\n",
"\n",
"With the `import` statement, we can import the entire package just as we would import a module from the [standard library <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/index.html)."
"The above cell runs the code in the [*\\_\\_init\\_\\_.py* <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/__init__.py) file from top to bottom, which in turn runs the [*matrix.py* <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/matrix.py), [*utils.py* <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/utils.py), and [*vector.py* <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/vector.py) modules (cf., look at the `import` statements in the four \\*.py files to get the idea). As both [*matrix.py* <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/matrix.py) and [*vector.py* <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/vector.py) depend on each other (i.e., the `Matrix` class needs the `Vector` class to work and vice versa), understanding the order in that the modules are executed is not trivial. Without going into detail, we mention that Python guarantees that each \\*.py file is run only once and figures out the order on its own. If Python is unable to do that, for example, due to unresolvable cirular imports, it aborts with an `ImportError`.\n",
"... and we use the built-in [dir() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#dir) function to check what attributes `pkg` comes with."
"The package's meta information and documentation are automatically parsed from the [*\\_\\_init\\_\\_.py* <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/__init__.py) file."
"A common practice by package authors is to put all the objects on the package's top level that they want the package users to work with directly. That is achieved via the `import` statements in the [*\\_\\_init\\_\\_.py* <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/__init__.py) file.\n",
"\n",
"However, users can always reach into a package and work with its internals.\n",
"\n",
"For example, the `Vector` and `Matrix` classes are also available via their **qualified name** (cf., [PEP 3155 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://www.python.org/dev/peps/pep-3155/)): First, we access the `vector` and `matrix` modules on `pkg`, and then the `Vector` and `Matrix` classes on the modules."
"Also, let's import the [*utils.py* <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/utils.py) module with the `norm()` function into the global scope. As this function is integrated into the `Vector.__abs__()` and `Matrix.__abs__()` methods, there is actually no need to work with it explicitly."
"Many tutorials on the internet begin by importing \"everything\" from a package into the global scope with `from ... import *`.\n",
"\n",
"That is commonly considered a *bad* practice as it may overwrite already existing variables. However, if the package's [*\\_\\_init\\_\\_.py* <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/__init__.py) file defines an `__all__` attribute, a `list` with all the names to be \"exported,\" the **star import** is safe to be used, in particular, in *interactive* sessions like Jupyter notebooks. We emphasize that the star import should *not* be used *within* packages and modules as then it is not directly evident from a name where the corresponding object is defined.\n",
"\n",
"For more best practices regarding importing we refer to, among others, [Google's Python Style Guide](https://google.github.io/styleguide/pyguide.html#22-imports).\n",
"\n",
"The following `import` statement makes the `Vector` and `Matrix` classes available in the global scope."
"For further information on modules and packages, we refer to the [official tutorial <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/tutorial/modules.html)."
"The final implementations of the `Vector` and `Matrix` classes are in the [*matrix.py* <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/matrix.py) and [*vector.py* <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/vector.py) files: They integrate all of the functionalities introduced in this chapter. In addition, the code is cleaned up and fully documented, including examples of common usages.\n",
"\n",
"We strongly suggest the eager student go over the files in the [*sample_package* <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/tree/develop/11_classes/sample_package) in detail at some point to understand what well-written and (re-)usable code looks like."
"Furthermore, the classes are designed for easier maintenence in the long-run.\n",
"\n",
"For example, the `Matrix/Vector.storage` and `Matrix/Vector.typing` class attributes replace the \"hard coded\" [tuple() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#func-tuple) and [float() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#float) built-ins in the `.__init__()` methods: As `self.storage` and `self.typing` are not defined on the *instances*, Python automatically looks them up on the *classes*."
"\u001b[0;34m\u001b[0m \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"a vector must have at least one entry\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"Both `Matrix/Vector.storage` and `Matrix/Vector.typing` themselves reference the `DEFAULT_ENTRIES_STORAGE` and `DEFAULT_ENTRY_TYPE` constants in the [*utils.py* <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com/webartifex/intro-to-python/blob/develop/11_classes/sample_package/utils.py) module. This way, we could, for example, change only the constants and thereby also change how the `._entries` are stored internally in both classes. Also, this single **[single source of truth <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_wiki.png\">](https://en.wikipedia.org/wiki/Single_source_of_truth)** ensures that both classes are consistent with each other at all times."
"Of course, we could also use the [type() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#type) built-in instead."
"In order to not have to \"hard code\" the name of *another* class (e.g., the `Vector.as_matrix()` method references the `Matrix` class), we apply the following \"hack:\" First, we store a reference to the other class as a class attribute (e.g., `Matrix.vector_cls` and `Vector.matrix_cls`), and then reference that attribute within the methods, just like `.storage` and `.typing` above."
"For completeness sake, we mention that in the final `Vector` and `Matrix` classes, the `.__sub__()` and `.__rsub__()` methods use the negation operator implemented in `.__neg__()` and then dispatch to `.__add__()` instead of implementing the subtraction logic themselves."
"## Comparison with [numpy](https://www.numpy.org/)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"We started out in this chapter by realizing that Python provides us no good data type to model a vector $\\vec{x}$ or a matrix $\\bf{A}$. Then, we built up two custom data types, `Vector` and `Matrix`, that wrap a simple `tuple` object for $\\vec{x}$ and a `tuple` of `tuple`s for $\\bf{A}$ so that we can interact with their `._entries` in a \"natural\" way, which is similar to how we write linear algebra tasks by hand. By doing this, we extend Python with our own little \"dialect\" or **[domain-specific language <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_wiki.png\">](https://en.wikipedia.org/wiki/Domain-specific_language)** (DSL).\n",
"\n",
"If we feel like sharing our linear algebra library with the world, we could easily do so on either [GitHub <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_gh.png\">](https://github.com) or [PyPI](https://pypi.org). However, for the domain of linear algebra this would be rather pointless as there is already a widely adopted library with [numpy](https://www.numpy.org/) that not only has a lot more features than ours but also is implemented in C, which makes it a lot faster with big data.\n",
"\n",
"Let's model the example in the [first part <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/develop/11_classes/00_content.ipynb#Example:-Vectors-&-Matrices) with both [numpy](https://www.numpy.org/) and our own DSL and compare them."
]
},
{
"cell_type": "code",
"execution_count": 54,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [],
"source": [
"x = (1, 2, 3)\n",
"A = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]"
]
},
{
"cell_type": "code",
"execution_count": 55,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"ename": "TypeError",
"evalue": "can't multiply sequence by non-int of type 'tuple'",
"\u001b[0;32m<ipython-input-55-fd81d962f516>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mA\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;31mTypeError\u001b[0m: can't multiply sequence by non-int of type 'tuple'"
]
}
],
"source": [
"A * x"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"The creation of vectors and matrices is similar to our DSL. However, numpy uses the more general concept of an **n-dimensional array** (i.e., the `ndarray` type) where a vector is only a special case of a matrix and a matrix is yet another special case of an even higher dimensional structure."
]
},
{
"cell_type": "code",
"execution_count": 56,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [],
"source": [
"import numpy as np"
]
},
{
"cell_type": "code",
"execution_count": 57,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [],
"source": [
"x_arr = np.array(x)\n",
"A_arr = np.array(A)"
]
},
{
"cell_type": "code",
"execution_count": 58,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [],
"source": [
"x_vec = Vector(x)\n",
"A_mat = Matrix(A)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"The text representations are very similar. However, [numpy](https://www.numpy.org/)'s `ndarray`s keep the entries as `int`s while our `Vector` and `Matrix` objects contain `float`s."
"... while `Matrix` objects come with `.n_rows` and `.n_cols` properties."
]
},
{
"cell_type": "code",
"execution_count": 65,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"text/plain": [
"(3, 3)"
]
},
"execution_count": 65,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"A_mat.n_rows, A_mat.n_cols"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"The built-in [len() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#len) function does not return the number of entries in an `ndarray` but the number of the rows instead. This is equivalent to the first element in the `.shape` attribute."
]
},
{
"cell_type": "code",
"execution_count": 66,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"data": {
"text/plain": [
"3"
]
},
"execution_count": 66,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(x_arr)"
]
},
{
"cell_type": "code",
"execution_count": 67,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"text/plain": [
"3"
]
},
"execution_count": 67,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(x_vec)"
]
},
{
"cell_type": "code",
"execution_count": 68,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"text/plain": [
"3"
]
},
"execution_count": 68,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(A_arr)"
]
},
{
"cell_type": "code",
"execution_count": 69,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"text/plain": [
"9"
]
},
"execution_count": 69,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(A_mat)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"The `.transpose()` method also exists for `ndarray`s."
"To perform matrix-matrix, matrix-vector, or vector-matrix multiplication in [numpy](https://www.numpy.org/), we use the `.dot()` method. If we use the `*` operator with `ndarray`s, an *entry-wise* multiplication is performed."
"Because we implemented our classes to support the sequence protocol, [numpy](https://www.numpy.org/)'s *one*-dimensional `ndarray`s are actually able to work with them: The `*` operator is applied on a per-entry basis."
]
},
{
"cell_type": "code",
"execution_count": 77,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"data": {
"text/plain": [
"array([2., 4., 6.])"
]
},
"execution_count": 77,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x_arr + x_vec"
]
},
{
"cell_type": "code",
"execution_count": 78,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"text/plain": [
"array([1., 4., 9.])"
]
},
"execution_count": 78,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x_arr * x_vec"
]
},
{
"cell_type": "code",
"execution_count": 79,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"ename": "ValueError",
"evalue": "operands could not be broadcast together with shapes (3,3) (9,) ",
"\u001b[0;32m<ipython-input-79-032c970f8cfc>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mA_arr\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mA_mat\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;31mValueError\u001b[0m: operands could not be broadcast together with shapes (3,3) (9,) "
]
}
],
"source": [
"A_arr + A_mat"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"We conclude that it is rather easy to extend Python in a way that makes the resulting application code read like core Python again. As there are many well established third-party packages out there, it is unlikely that we have to implement a fundamental library ourselves. Yet, we can apply the concepts introduced in this chapter to organize the code in the applications we write."
