alexander/intro-to-python
1
1
Fork 0
Code Issues Pull requests Releases Activity

Release 0.1.0

Browse source 
After refurbishing the project we prepare a new relaease.
There are no changes with respect to the contents as compared to v0.0.0
that are noteworthy release notes.
This commit is contained in:
Alexander Hess 2024-04-08 22:13:31 +02:00
commit 94e5112f10
Signed by: alexander
GPG key ID: 344EA5AB10D868E0
65 changed files with 387 additions and 387 deletions
Download patch file Download diff file Expand all files Collapse all files

22
08_mfr/01_content.ipynb
View file

@ -8,7 +8,7 @@
}
},
"source": [
"**Note**: Click on \"*Kernel*\" > \"*Restart Kernel and Clear All Outputs*\" in [JupyterLab](https://jupyterlab.readthedocs.io/en/stable/) *before* reading this notebook to reset its output. If you cannot run this file on your machine, you may want to open it [in the cloud <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_mb.png\">](https://mybinder.org/v2/gh/webartifex/intro-to-python/develop?urlpath=lab/tree/08_mfr/01_content.ipynb)."
"**Note**: Click on \"*Kernel*\" > \"*Restart Kernel and Clear All Outputs*\" in [JupyterLab](https://jupyterlab.readthedocs.io/en/stable/) *before* reading this notebook to reset its output. If you cannot run this file on your machine, you may want to open it [in the cloud <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_mb.png\">](https://mybinder.org/v2/gh/webartifex/intro-to-python/main?urlpath=lab/tree/08_mfr/01_content.ipynb)."
]
},
{
@ -30,7 +30,7 @@
}
},
"source": [
"After introducing the Map-Filter-Reduce paradigm in the [first part <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/develop/08_mfr/00_content.ipynb) of this chapter, we first see how `list` comprehensions can replace the [map() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#map) and [filter() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#filter) built-ins in many cases. Then, we learn how `generator` expressions are like `list` comprehensions *without* using the memory. We end this part with a short discussion of the built-in [all() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#all) and [any() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#any) functions."
"After introducing the Map-Filter-Reduce paradigm in the [first part <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/main/08_mfr/00_content.ipynb) of this chapter, we first see how `list` comprehensions can replace the [map() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#map) and [filter() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#filter) built-ins in many cases. Then, we learn how `generator` expressions are like `list` comprehensions *without* using the memory. We end this part with a short discussion of the built-in [all() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#all) and [any() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#any) functions."
]
},
{
@ -52,7 +52,7 @@
}
},
"source": [
"Consider again the original \"*A simple Filter*\" example from [Chapter 4 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/develop/04_iteration/03_content.ipynb#Example:-A-simple-Filter), re-written such that both the mapping and the filtering are done in *one* `for`-loop instead of the *two* above."
"Consider again the original \"*A simple Filter*\" example from [Chapter 4 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/main/04_iteration/03_content.ipynb#Example:-A-simple-Filter), re-written such that both the mapping and the filtering are done in *one* `for`-loop instead of the *two* above."
]
},
{
@ -156,7 +156,7 @@
"source": [
"A list comprehension may always be used in a place where otherwise a `list` object would work.\n",
"\n",
"For example, let's rewrite the \"*A simple Filter*\" example from [Chapter 4 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/develop/04_iteration/03_content.ipynb#Example:-A-simple-Filter) in just one line. As a caveat, the code below *materializes* all elements in memory *before* summing them up, and may, therefore, cause a `MemoryError` when executed with a bigger `numbers` list. We see with [PythonTutor <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](http://pythontutor.com/visualize.html#code=numbers%20%3D%20range%281,%2013%29%0Aresult%20%3D%20sum%28%5B%28n%20**%202%29%20%2B%201%20for%20n%20in%20numbers%20if%20n%20%25%202%20%3D%3D%200%5D%29&cumulative=false&curstr=0&heapPrimitives=nevernest&mode=display&origin=opt-frontend.js&py=3&rawInputLstJSON=%5B%5D&textReferences=false) how a `list` object exists in memory at step 17 and then \"gets lost\" right after. As the next section shows, this downside may be mitigated."
"For example, let's rewrite the \"*A simple Filter*\" example from [Chapter 4 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/main/04_iteration/03_content.ipynb#Example:-A-simple-Filter) in just one line. As a caveat, the code below *materializes* all elements in memory *before* summing them up, and may, therefore, cause a `MemoryError` when executed with a bigger `numbers` list. We see with [PythonTutor <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](http://pythontutor.com/visualize.html#code=numbers%20%3D%20range%281,%2013%29%0Aresult%20%3D%20sum%28%5B%28n%20**%202%29%20%2B%201%20for%20n%20in%20numbers%20if%20n%20%25%202%20%3D%3D%200%5D%29&cumulative=false&curstr=0&heapPrimitives=nevernest&mode=display&origin=opt-frontend.js&py=3&rawInputLstJSON=%5B%5D&textReferences=false) how a `list` object exists in memory at step 17 and then \"gets lost\" right after. As the next section shows, this downside may be mitigated."
]
},
{
@ -639,7 +639,7 @@
}
},
"source": [
"To find the overall product, we *unpack* the list comprehension into the `product()` function from the \"*Function Definitions & Calls*\" sub-section in [Chapter 7 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/develop/07_sequences/03_content.ipynb#Function-Definitions-&-Calls)."
"To find the overall product, we *unpack* the list comprehension into the `product()` function from the \"*Function Definitions & Calls*\" sub-section in [Chapter 7 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/main/07_sequences/03_content.ipynb#Function-Definitions-&-Calls)."
]
},
{
@ -766,7 +766,7 @@
"source": [
"Because of the high memory consumption, Pythonistas avoid materialized `list` objects, and, thus, also `list` comprehensions, whenever possible. Instead, they prefer to work with **[`generator` expressions <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/reference/expressions.html#generator-expressions)**. Syntactically, they work like list comprehensions except that parentheses, `(` and `)`, replace brackets, `[` and `]`.\n",
"\n",
"Let's go back to the original \"*A simple Filter*\" example from [Chapter 4 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/develop/04_iteration/03_content.ipynb#Example:-A-simple-Filter) one more time, apply the transformation $y := x^2 + 1$ to all even `numbers`, and sum them up."
"Let's go back to the original \"*A simple Filter*\" example from [Chapter 4 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/main/04_iteration/03_content.ipynb#Example:-A-simple-Filter) one more time, apply the transformation $y := x^2 + 1$ to all even `numbers`, and sum them up."
]
},
{
@ -1050,7 +1050,7 @@
}
},
"source": [
"`generator` objects work just like the `map` and `filter` objects in the [first part <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/develop/08_mfr/00_content.ipynb) of this chapter. So, with the [next() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#next) function, we can generate elements one by one."
"`generator` objects work just like the `map` and `filter` objects in the [first part <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/main/08_mfr/00_content.ipynb) of this chapter. So, with the [next() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#next) function, we can generate elements one by one."
]
},
{
@ -1564,7 +1564,7 @@
"source": [
"Now, the first of the two alternative solutions may be more appealing to many readers. In general, many practitioners seem to dislike `lambda` expressions.\n",
"\n",
"In the first solution, we *unpack* the elements produced by `(1 + (x / y) for x in first for y in second)` into the `product()` function from the \"*Function Definitions & Calls*\" sub-section in [Chapter 7 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/develop/07_sequences/03_content.ipynb#Function-Definitions-&-Calls). However, inside `product()`, the elements are *packed* into `args`, a *materialized* `tuple` object! So, all the memory efficiency gained by using a generator expression is lost! [PythonTutor <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](http://pythontutor.com/visualize.html#code=def%20product%28*args%29%3A%0A%20%20%20%20result%20%3D%20args%5B0%5D%0A%20%20%20%20for%20arg%20in%20args%5B1%3A%5D%3A%0A%20%20%20%20%20%20%20%20result%20*%3D%20arg%0A%20%20%20%20return%20result%0A%0Afirst%20%3D%20range%2810,%2031,%2010%29%0Asecond%20%3D%20range%2840,%2061,%2010%29%0A%0Aresult%20%3D%20product%28*%281%20%2B%20%28x%20/%20y%29%20for%20x%20in%20first%20for%20y%20in%20second%29%29&cumulative=false&curstr=0&heapPrimitives=nevernest&mode=display&origin=opt-frontend.js&py=3&rawInputLstJSON=%5B%5D&textReferences=false) shows how a `tuple` object exists in steps 38-58."
"In the first solution, we *unpack* the elements produced by `(1 + (x / y) for x in first for y in second)` into the `product()` function from the \"*Function Definitions & Calls*\" sub-section in [Chapter 7 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/main/07_sequences/03_content.ipynb#Function-Definitions-&-Calls). However, inside `product()`, the elements are *packed* into `args`, a *materialized* `tuple` object! So, all the memory efficiency gained by using a generator expression is lost! [PythonTutor <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](http://pythontutor.com/visualize.html#code=def%20product%28*args%29%3A%0A%20%20%20%20result%20%3D%20args%5B0%5D%0A%20%20%20%20for%20arg%20in%20args%5B1%3A%5D%3A%0A%20%20%20%20%20%20%20%20result%20*%3D%20arg%0A%20%20%20%20return%20result%0A%0Afirst%20%3D%20range%2810,%2031,%2010%29%0Asecond%20%3D%20range%2840,%2061,%2010%29%0A%0Aresult%20%3D%20product%28*%281%20%2B%20%28x%20/%20y%29%20for%20x%20in%20first%20for%20y%20in%20second%29%29&cumulative=false&curstr=0&heapPrimitives=nevernest&mode=display&origin=opt-frontend.js&py=3&rawInputLstJSON=%5B%5D&textReferences=false) shows how a `tuple` object exists in steps 38-58."
]
},
{
@ -1656,13 +1656,13 @@
}
},
"source": [
"With the new concepts in this chapter, let's rewrite the book's introductory \"*Averaging all even Numbers in a List*\" example from [Chapter 1 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/develop/01_elements/00_content.ipynb#Example:-Averaging-all-even-Numbers-in-a-List) such that it efficiently handles a large sequence of numbers. We continue from its latest implementation, the `average_evens()` function in the \"*Keyword-only Arguments*\" section in [Chapter 2 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/develop/02_functions/00_content.ipynb#Keyword-only-Arguments).\n",
"With the new concepts in this chapter, let's rewrite the book's introductory \"*Averaging all even Numbers in a List*\" example from [Chapter 1 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/main/01_elements/00_content.ipynb#Example:-Averaging-all-even-Numbers-in-a-List) such that it efficiently handles a large sequence of numbers. We continue from its latest implementation, the `average_evens()` function in the \"*Keyword-only Arguments*\" section in [Chapter 2 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/main/02_functions/00_content.ipynb#Keyword-only-Arguments).\n",
"\n",
"We assume that `average_evens()` is called with a *finite* and *iterable* object that generates a **stream** of numeric objects that can be cast as `int` objects. After all, the idea of even and odd numbers makes sense only in the context of whole numbers.\n",
"\n",
"The `generator` expression `(round(n) for n in numbers)` implements the type casting, and, when it is evaluated during a function call, *nothing* happens except that a `generator` object is assigned to `integers`. Then, with the [reduce() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functools.html#functools.reduce) function from the [functools <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functools.html) module, we *simultaneously* add up *and* count the even numbers with the `generator` object to which the inner `generator` expression `((n, 1) for n in integers if n % 2 == 0)` evaluates to. That `generator` object takes the `integers` generator as its source and produces `tuple` objects consisting of the next *even* number in line and `1`. Two such `tuple` objects are then iteratively passed to the `function` object to which the `lambda` expression evaluates to. `x` represents the total and the count of the even numbers processed so far, while `y`'s first element, `y[0]`, is the next *even* number to be added to the running total. The result of calling [reduce() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functools.html#functools.reduce) is again a `tuple` object, namely the final `total` and `count`. Lastly, we calculate the simple average and scale it.\n",
"\n",
"In summary, this implementation of `average_evens()` does *not* keep materialized `list` objects internally like its predecessors in [Chapter 2 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/develop/02_functions/00_content.ipynb) but processes the elements of the `numbers` argument on a one-by-one basis."
"In summary, this implementation of `average_evens()` does *not* keep materialized `list` objects internally like its predecessors in [Chapter 2 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/main/02_functions/00_content.ipynb) but processes the elements of the `numbers` argument on a one-by-one basis."
]
},
{
@ -2040,7 +2040,7 @@
}
},
"source": [
"[all() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#all) can be viewed as syntactic sugar replacing a `for`-loop: Internally, [all() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#all) implements the *short-circuiting* strategy explained in [Chapter 3 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/develop/03_conditionals/00_content.ipynb#Short-Circuiting), and we mimic that by testing for the *opposite* condition in the `if` statement and stopping the `for`-loop early with the `break` statement. In the worst case, if `threshold` were, for example, `150`, we would loop over *all* elements in the `iterable` argument, which must be *finite* for the code to work. So, [all() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#all) is a *linear search* in disguise."
"[all() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#all) can be viewed as syntactic sugar replacing a `for`-loop: Internally, [all() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#all) implements the *short-circuiting* strategy explained in [Chapter 3 <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_nb.png\">](https://nbviewer.jupyter.org/github/webartifex/intro-to-python/blob/main/03_conditionals/00_content.ipynb#Short-Circuiting), and we mimic that by testing for the *opposite* condition in the `if` statement and stopping the `for`-loop early with the `break` statement. In the worst case, if `threshold` were, for example, `150`, we would loop over *all* elements in the `iterable` argument, which must be *finite* for the code to work. So, [all() <img height=\"12\" style=\"display: inline-block\" src=\"../static/link/to_py.png\">](https://docs.python.org/3/library/functions.html#all) is a *linear search* in disguise."
]
},
{