Commit graph

149 commits

Author SHA1 Message Date
419c7e6bf6
Add tests for the Order ORM model
- so far, the check constraints were untested
  + the tests mirror the tests for the `ReplayedOrder` model
    with some variations
- shorten some import names
- fix some typos
2021-09-16 12:05:22 +02:00
e4e543bd40
Add ORM models for the simulation data
- add new ORM models `ReplaySimulation` and `ReplayedOrder`
  to store the data generated by the routing simulations
- add migrations script to create the corresponding database tables
  + create "replay_simulations" and "replayed_orders" tables
  + add missing check constraints to "orders" table
  + add unique constraint to "orders" table to enable compound key
  + drop unnecessary check constraints from the "orders" table
- add tests for the new ORM models
  + add `simulation`, `replayed_order`, `make_replay_order()`, and
    `pre_order` fixtures
  + add `ReplayedOrderFactor` faker class
- fix some typos
2021-09-16 11:58:55 +02:00
41f75f507d
Use a unified DATE constant for convenience 2021-09-16 11:53:40 +02:00
f0eb9d3b6f
There is no 'too complex function'
We check a function's cognitive complexity only with `mccabe` (=C901)
and not with WPS231 as the two overlap in most cases.
2021-09-15 14:51:56 +02:00
6091ad95c6
Fix failing tests due to randomization 2021-09-15 12:00:03 +02:00
f891fac3dc
Refactor tests.db.fake_data.factories into a package 2021-09-15 11:58:17 +02:00
7e23033d84
There is no 'too deep nesting' 2021-09-15 11:58:13 +02:00
5978b0e92f
Modules are never 'too complex' 2021-09-15 11:47:38 +02:00
d03c2f91fb
Merge branch 'main' into develop 2021-09-13 11:37:37 +02:00
0f60640bc0
Show how the Google Maps API is integrated
Some checks failed
CI / fast (without R) (push) Has been cancelled
CI / slow (with R) (push) Has been cancelled
- show an example order's path from the restaurant to the customer's
  delivery address as it is travelled by a courier
- explain how to use the Google Maps API directly
- show how the API is integrated into the data model
2021-09-13 11:30:17 +02:00
f6b331883e
Merge branch 'release-0.4.0' into main
Some checks failed
CI / fast (without R) (push) Has been cancelled
CI / slow (with R) (push) Has been cancelled
2021-09-13 11:22:42 +02:00
5eabd541e8
Merge branch 'release-0.4.0' into develop 2021-09-13 11:22:01 +02:00
19d7b6f3cc
Finalize release 0.4.0 2021-09-13 11:20:53 +02:00
97b75f2579
Merge branch 'google-maps' into develop 2021-09-13 11:11:53 +02:00
d83ff2e273
Add Order.draw() and Path.draw()
- `Order.draw()` plots a `Courier`'s path from the
  `Order.pickup_address` to the `Order.delivery_address`
- `Path.draw()` plots a `Courier`'s path between any two
  `Address` objects
2021-09-13 10:33:35 +02:00
2449492aba
Add Path.from_order() convenience method
Special case of `Path.from_addresses()` to create a single
`Path` object from an `Order.restaurant` to the `Customer`.
2021-09-13 09:57:25 +02:00
2d324b77eb
Rename DistanceMatrix into Path
- a `Path` is a better description for an instance of the model
- the `Location`s en route are renamed into `.waypoints`
- generic `assoc` is renamed into `path` in the test suite
2021-09-12 17:33:48 +02:00
2d08afa309
Upgrade sqlalchemy
Adapt code to prevent new warnings and errors (as of SQLAlchemy 1.4):
- Overlapping foreign key columns could be set in a conflicting way
  => This is prevented by the application logic
  => Ignore the warning by setting a `overlaps` flag
- Transaction already rolled back
  => This only happens when tests cause an `IntegrityError` on purpose
  => Filter away the corresponding warning in the fixture
- Query returns `Row` objects and not scalars
  => Add genexpr to pull out `primary_id`
2021-09-12 16:52:51 +02:00
3bef9ca38d
Remove flake8-expression-complexity ...
... from the dev dependencies.

Longer queries in SQLAlchemy get flagged even though they are not
complicated. Other expressions are generally not that complicated.
2021-09-12 16:52:44 +02:00
1c19da2f70
Solve all issues detected by PyCharm
- as of September 2021, PyCharm is used to write some of the code
- PyCharm's built-in code styler, linter, and type checker issued
  some warnings that are resolved in this commit
  + spelling mistakes
  + all instance attributes must be specified explicitly
    in a class's __init__() method
    => use `functools.cached_property` for caching
  + make `tuple`s explicit with `(...)`
  + one test failed randomly although everything is ok
    => adjust the fixture's return value (stub for Google Directions API)
  + reformulate SQL so that PyCharm can understand the symbols
2021-09-12 16:51:12 +02:00
1268aba017
Pin the dependencies ...
... after upgrading:
- alembic
- geopy
- googlemaps
- psycopg2
- rpy2
- dev dependencies
  + coverage       + darglint      + flake8      + flake8-annotations
  + flake8-black   + flake8-comprehensions       + flake8-docstrings
  + flake8-pytest-style            + pre-commit  + pytest
  + pytest-cov     + pytest-mock   + sphinx      + sphinx-autodoc-typehints
- research dependencies
  + jupyterlab     + matplotlib    + numpy       + pandas
- transient dependencies
  + argcomplete    + argon2-cffi   + attrs       + babel
  + bleach         + certifi       + cffi        + cfgv
  + chardset-normalizer            + colorlog    + debugpy
  + decorator      + defusedxml    + distlib     + geographiclib
  + gitdb          + gitpython     + greenlet    + identify
  + idna           + importlib-resources         + ipykernel
  + ipython        + jinja2        + json5       + jupyter-client
  + kiwisolver     + mako          + markupsafe
  + nbclient       + nbconvert     + nbformat    + nodeenv
  + notebook       + parso         + pathspec    + pbr
  + pillow         + platformdirs  + pluggy      + prometheus-client
  + prompt-toolkit + pycodestyle   + pydocstyle  + pyflakes
  + pygments       + pyristent     + python-dateutil
  + pywin32        + pywinpty      + pyzmq       + regex
  + requests       + scipy         + send2trash  + six
  + smmap          + sphinxcontrib-htmlhelp
  + sphinxcontrib-serializinghtml  + stevedore   + terminado
  + textfixtures   + testpath      + traitlets   + typed-ast
  + typing-extensions              + tzdata      + tzlocal
  + urllib3        + virtualenv    + zipp
2021-09-12 16:51:12 +02:00
6636e56ec8
Upgrade flake8-pytest-style
The newly introduced "P023" error code must be disabled explicitly.
2021-09-12 16:51:11 +02:00
fa3b761054
Remove pytest-randomly from the dev dependencies
- problem: because of the randomization of test cases, every once in a
  while, the schema holding the test db already exists and cannot be
  created a second time (background: we run all tests against a db
  created with `metadate.create_all()` by SQLAlchemy at once and
  a db created by running all incremental Alchemy migration scripts)
- quick fix: possible, we could find a way to run all tests against
  one of the two test db's in random order and then against the other
  => in the interest of time, we simply do not randomize the test cases
2021-09-12 16:51:11 +02:00
322ce57062
Make mypy a bit stricter 2021-09-12 16:51:11 +02:00
2ba4914af7
Ignore PyCharm's .idea/ folder 2021-09-12 16:51:10 +02:00
db715edd6d
Add constructor for the DistanceMatrix class
- `DistanceMatrix.from_addresses()` takes a variable number of
  `Address` objects and creates distance matrix entries for them
- as a base measure, the air distance between two `Address`
  objects is calculated
- in addition, an integration with the Google Maps Directions API is
  implemented that provides a more realistic measure of the distance
  and duration a rider on a bicycle would need to travel between two
  `Address` objects
- add a `Location.lat_lng` convenience property that provides the
  `.latitude` and `.longitude` of an `Address` as a 2-`tuple`
2021-09-12 16:51:10 +02:00
5e9307523c
Add ordered-set to the dependencies 2021-09-12 16:51:10 +02:00
cc75307e5a
Add DistanceMatrix class
- the class stores the data of a distance matrix between all addresses
  + air distances
  + bicycle distances
- in addition, the "path" returned by the Google Directions API are
  also stored as a JSON serialized sequence of latitude-longitude pairs
- we assume a symmetric graph
2021-09-12 16:51:10 +02:00
28368cc30a
Add geopy to the dependencies 2021-09-12 16:51:09 +02:00
3dd848605c
Add googlemaps to the dependencies 2021-09-12 16:51:09 +02:00
6c03261b07
Merge branch 'main' into develop 2021-03-01 14:45:24 +01:00
f5ced933d6
Merge branch 'tactical-forecasting' into main
Some checks failed
CI / fast (without R) (push) Has been cancelled
CI / slow (with R) (push) Has been cancelled
2021-03-01 14:13:21 +01:00
9d6de9d98c
Create tactical demand forecasts
- the first notebook runs the tactical-forecasts command
- the second notebook describes the tactical demand forecasting process
  + demand aggregation on a per-pixel level
  + time series generation: horizontal, vertical, and real-time time series
  + STL decomposition into seasonal, trend, and residual components
  + choosing the most promising forecasting model
  + predicting demand with various models
- fix where to re-start the forecasting process after it was interrupted
- enable the heuristic for choosing the most promising model
  to also work for 7 training weeks
2021-02-09 17:06:37 +01:00
21d012050c
Add visualization scripts for customers/restaurants
- the two notebook files are helpful in visualizing all relevant
  pickup (red) or delivery (blue) locations from the point of view
  of either e restaurant or a customer
2021-02-04 16:19:12 +01:00
40471b883c
Add infos on tactical demand forecasting 2021-02-04 16:12:25 +01:00
d494a7908b
Merge branch 'gridification' into main
Some checks failed
CI / fast (without R) (push) Has been cancelled
CI / slow (with R) (push) Has been cancelled
2021-02-04 15:39:46 +01:00
a89b9497f2
Visualize the pixel grids
- add notebook that runs the plotting code
- add three visualizations per city:
  + all addresses, colored by zip code
  + all restaurants, incl. the number of received orders
  + all restaurants on a grid with pixel side length of 1000m
2021-02-04 15:37:16 +01:00
1e263a4b98
Run the gridification script 2021-02-04 15:29:42 +01:00
28a7c7451c
Rename existing notebooks using order numbers 2021-02-04 15:10:14 +01:00
915aa4d3b4
Merge branch 'release-0.3.0' into develop 2021-02-04 13:23:39 +01:00
241e7ed81f
Merge branch 'release-0.3.0' into main
Some checks failed
CI / fast (without R) (push) Has been cancelled
CI / slow (with R) (push) Has been cancelled
2021-02-04 13:18:32 +01:00
d4ca85b55a
Finalize release 0.3.0 2021-02-04 13:13:26 +01:00
0da86e5f07
Pin the dependencies ...
... after upgrading:
- alembic
- matplotlib
- pandas
- rpy2
- sqlalchemy
- statsmodels
- dev dependencies
  + coverage     + factory-boy    + faker                 + nox
  + packaging    + pre-commit     + flake8-annotations    + pytest
  + pytest-cov   + sphinx
- research dependencies
  + numpy        + pyty
- transient dependencies
  + astpretty    + atomicwrites   + bleach                + chardet
  + colorlog     + darglint       + flake8-comprehensions + gitpython
  + identify     + ipykernel      + ipython               + jedi
  + jinja2       + jupyter-client + jupyter-core          + mako
  + nbformat     + nest-asyncio   + notebook              + parso
  + pluggy       + prompt-toolkit + ptyprocess            + pygments
  + pyyaml       + pyzmq          + requests              + smmap
  + terminado    + textfixtures   + snowballstemmer       + typed-ast
  + urllib3      + virtualenv

- fix SQL statements written in raw text
2021-02-04 13:12:47 +01:00
50b35a8284
Add CLI script to run tactical forecasting heuristic 2021-02-04 12:05:43 +01:00
23391c2fa4
Adjust OrderHistory.choose_tactical_model() heuristic
- use the `HorizontalSMAModel` for low demand
- use the `TrivialModel` for no demand
2021-02-02 15:20:02 +01:00
3f5b4a50bb
Rename Forecast.training_horizon into .train_horizon
- we use that shorter name in `urban_meal_delivery.forecasts.*`
  and want to be consistent in the ORM layer as well
2021-02-02 13:09:09 +01:00
6fd16f2a6c
Add TrivialModel
- the trivial model simply predicts `0` demand for all time steps
2021-02-02 12:45:26 +01:00
015d304306
Add HorizontalSMAModel
- the model applies a simple moving average on horizontal time series
- refactor `db.Forecast.from_dataframe()` to correctly convert
  `float('NaN')` values into `None`; otherwise, SQLAlchemy complains
2021-02-02 12:40:53 +01:00
af82951485
Add OrderHistory.choose_tactical_model()
- the method implements a heuristic from the first research paper
  that chooses the most promising forecasting `*Model` based on
  the average daily demand in a `Pixel` for a given `train_horizon`
- adjust the test scenario => `LONG_TRAIN_HORIZON` becomes `8`
  as that is part of the rule implemented in the heuristic
2021-02-02 11:29:27 +01:00
8926e9ff28
Fix nox session for slow CI tests
- when running tests marked with "r" we still must not run tests
  marked with "db" on the CI server
2021-02-01 22:00:47 +01:00