Commit graph

128 commits

Author SHA1 Message Date
6636e56ec8
Upgrade flake8-pytest-style
The newly introduced "P023" error code must be disabled explicitly.
2021-09-12 16:51:11 +02:00
fa3b761054
Remove pytest-randomly from the dev dependencies
- problem: because of the randomization of test cases, every once in a
  while, the schema holding the test db already exists and cannot be
  created a second time (background: we run all tests against a db
  created with `metadate.create_all()` by SQLAlchemy at once and
  a db created by running all incremental Alchemy migration scripts)
- quick fix: possible, we could find a way to run all tests against
  one of the two test db's in random order and then against the other
  => in the interest of time, we simply do not randomize the test cases
2021-09-12 16:51:11 +02:00
322ce57062
Make mypy a bit stricter 2021-09-12 16:51:11 +02:00
2ba4914af7
Ignore PyCharm's .idea/ folder 2021-09-12 16:51:10 +02:00
db715edd6d
Add constructor for the DistanceMatrix class
- `DistanceMatrix.from_addresses()` takes a variable number of
  `Address` objects and creates distance matrix entries for them
- as a base measure, the air distance between two `Address`
  objects is calculated
- in addition, an integration with the Google Maps Directions API is
  implemented that provides a more realistic measure of the distance
  and duration a rider on a bicycle would need to travel between two
  `Address` objects
- add a `Location.lat_lng` convenience property that provides the
  `.latitude` and `.longitude` of an `Address` as a 2-`tuple`
2021-09-12 16:51:10 +02:00
5e9307523c
Add ordered-set to the dependencies 2021-09-12 16:51:10 +02:00
cc75307e5a
Add DistanceMatrix class
- the class stores the data of a distance matrix between all addresses
  + air distances
  + bicycle distances
- in addition, the "path" returned by the Google Directions API are
  also stored as a JSON serialized sequence of latitude-longitude pairs
- we assume a symmetric graph
2021-09-12 16:51:10 +02:00
28368cc30a
Add geopy to the dependencies 2021-09-12 16:51:09 +02:00
3dd848605c
Add googlemaps to the dependencies 2021-09-12 16:51:09 +02:00
6c03261b07
Merge branch 'main' into develop 2021-03-01 14:45:24 +01:00
f5ced933d6
Merge branch 'tactical-forecasting' into main
Some checks failed
CI / fast (without R) (push) Has been cancelled
CI / slow (with R) (push) Has been cancelled
2021-03-01 14:13:21 +01:00
9d6de9d98c
Create tactical demand forecasts
- the first notebook runs the tactical-forecasts command
- the second notebook describes the tactical demand forecasting process
  + demand aggregation on a per-pixel level
  + time series generation: horizontal, vertical, and real-time time series
  + STL decomposition into seasonal, trend, and residual components
  + choosing the most promising forecasting model
  + predicting demand with various models
- fix where to re-start the forecasting process after it was interrupted
- enable the heuristic for choosing the most promising model
  to also work for 7 training weeks
2021-02-09 17:06:37 +01:00
21d012050c
Add visualization scripts for customers/restaurants
- the two notebook files are helpful in visualizing all relevant
  pickup (red) or delivery (blue) locations from the point of view
  of either e restaurant or a customer
2021-02-04 16:19:12 +01:00
40471b883c
Add infos on tactical demand forecasting 2021-02-04 16:12:25 +01:00
d494a7908b
Merge branch 'gridification' into main
Some checks failed
CI / fast (without R) (push) Has been cancelled
CI / slow (with R) (push) Has been cancelled
2021-02-04 15:39:46 +01:00
a89b9497f2
Visualize the pixel grids
- add notebook that runs the plotting code
- add three visualizations per city:
  + all addresses, colored by zip code
  + all restaurants, incl. the number of received orders
  + all restaurants on a grid with pixel side length of 1000m
2021-02-04 15:37:16 +01:00
1e263a4b98
Run the gridification script 2021-02-04 15:29:42 +01:00
28a7c7451c
Rename existing notebooks using order numbers 2021-02-04 15:10:14 +01:00
915aa4d3b4
Merge branch 'release-0.3.0' into develop 2021-02-04 13:23:39 +01:00
241e7ed81f
Merge branch 'release-0.3.0' into main
Some checks failed
CI / fast (without R) (push) Has been cancelled
CI / slow (with R) (push) Has been cancelled
2021-02-04 13:18:32 +01:00
d4ca85b55a
Finalize release 0.3.0 2021-02-04 13:13:26 +01:00
0da86e5f07
Pin the dependencies ...
... after upgrading:
- alembic
- matplotlib
- pandas
- rpy2
- sqlalchemy
- statsmodels
- dev dependencies
  + coverage     + factory-boy    + faker                 + nox
  + packaging    + pre-commit     + flake8-annotations    + pytest
  + pytest-cov   + sphinx
- research dependencies
  + numpy        + pyty
- transient dependencies
  + astpretty    + atomicwrites   + bleach                + chardet
  + colorlog     + darglint       + flake8-comprehensions + gitpython
  + identify     + ipykernel      + ipython               + jedi
  + jinja2       + jupyter-client + jupyter-core          + mako
  + nbformat     + nest-asyncio   + notebook              + parso
  + pluggy       + prompt-toolkit + ptyprocess            + pygments
  + pyyaml       + pyzmq          + requests              + smmap
  + terminado    + textfixtures   + snowballstemmer       + typed-ast
  + urllib3      + virtualenv

- fix SQL statements written in raw text
2021-02-04 13:12:47 +01:00
50b35a8284
Add CLI script to run tactical forecasting heuristic 2021-02-04 12:05:43 +01:00
23391c2fa4
Adjust OrderHistory.choose_tactical_model() heuristic
- use the `HorizontalSMAModel` for low demand
- use the `TrivialModel` for no demand
2021-02-02 15:20:02 +01:00
3f5b4a50bb
Rename Forecast.training_horizon into .train_horizon
- we use that shorter name in `urban_meal_delivery.forecasts.*`
  and want to be consistent in the ORM layer as well
2021-02-02 13:09:09 +01:00
6fd16f2a6c
Add TrivialModel
- the trivial model simply predicts `0` demand for all time steps
2021-02-02 12:45:26 +01:00
015d304306
Add HorizontalSMAModel
- the model applies a simple moving average on horizontal time series
- refactor `db.Forecast.from_dataframe()` to correctly convert
  `float('NaN')` values into `None`; otherwise, SQLAlchemy complains
2021-02-02 12:40:53 +01:00
af82951485
Add OrderHistory.choose_tactical_model()
- the method implements a heuristic from the first research paper
  that chooses the most promising forecasting `*Model` based on
  the average daily demand in a `Pixel` for a given `train_horizon`
- adjust the test scenario => `LONG_TRAIN_HORIZON` becomes `8`
  as that is part of the rule implemented in the heuristic
2021-02-02 11:29:27 +01:00
8926e9ff28
Fix nox session for slow CI tests
- when running tests marked with "r" we still must not run tests
  marked with "db" on the CI server
2021-02-01 22:00:47 +01:00
cb7611d587
Add OrderHistory.avg_daily_demand()
- the method calculates the number of daily `Order`s in a `Pixel`
  withing the `train_horizon` preceding the `predict_day`
2021-02-01 21:50:42 +01:00
67cd58cf16
Add urban_meal_delivery.forecasts.models sub-package
- `*Model`s use the `methods.*.predict()` functions to predict demand
  given an order time series generated by `timify.OrderHistory`
- `models.base.ForecastingModelABC` unifies how all `*Model`s work
  and implements a caching strategy
- implement three `*Model`s for tactical forecasting, based on the
  hets, varima, and rtarima models described in the first research paper
- add overall documentation for `urban_meal_delivery.forecasts` package
- move the fixtures in `tests.forecasts.timify.conftest` to
  `tests.forecasts.conftest` and adjust the horizon of the test horizon
  from two to three weeks
2021-02-01 20:39:52 +01:00
796fdc919c
Add Forecast.from_dataframe() constructor
- this alternative constructor takes the `pd.DataFrame`s from the
  `*Model.predict()` methods and converts them into ORM models
2021-02-01 15:50:30 +01:00
b8952213d8
Add extrapolate_season.predict() function
- the function implements a forecasting "method" similar to the
  seasonal naive method
  => instead of simply taking the last observation given a seasonal lag,
     it linearly extrapolates all observations of the same seasonal lag
     from the past into the future; conceptually, it is like the
     seasonal naive method with built-in smoothing
- the function is tested just like the `arima.predict()` and
  `ets.predict()` functions
  + rename the `tests.forecasts.methods.test_ts_methods` module
    into `tests.forecasts.methods.test_predictions`
- re-organize some constants in the `tests` package
- streamline some docstrings
2021-02-01 11:32:10 +01:00
1d63623dfc
Add Forecast.__repr__() 2021-01-31 21:57:05 +01:00
47ef1f8759
Add OrderHistory.first/last_order() methods
- get the `datetime` of the first or last order within a pixel
- unify some fixtures in `tests.forecasts.timify.conftest`
2021-01-31 21:46:20 +01:00
7b824a4a12
Shorten a couple of names
- rename "total_orders" columns into "n_orders"
- rename `.make_*_time_series()` methods into `.make_*_ts()`
2021-01-31 20:20:55 +01:00
d45c60b764
Add OrderHistory.time_step property 2021-01-31 20:06:23 +01:00
fd404e2b89
Adjust Pixel.__repr__() a tiny bit 2021-01-31 19:34:05 +01:00
63e8e94145
Add Pixel.restaurants property
- the property loads all `Restaurant`s from the database that
  are within the `Pixel`
2021-01-31 19:29:18 +01:00
08b748c867
Move decomposition module into methods sub-package
- move the module
- unify the corresponding tests in `tests.forecasts.methods` sub-package
- make all `predict()` and the `stl()` function round results
- streamline documentation
2021-01-31 18:54:58 +01:00
a5b590b24c
Add Forecast.actual column 2021-01-31 18:29:53 +01:00
6429165aaf
Add statsmodels to the dependencies 2021-01-31 18:24:03 +01:00
4b6d92958d
Add functionality for drawing folium.Maps
- this code is not unit-tested due to the complexity involving
  interactive `folium.Map`s => visual checks give high confidence
2021-01-26 17:07:50 +01:00
605ade4078
Add Pixel.northeast/southwest properties
- the properties are needed for the drawing functionalitites
2021-01-26 17:05:36 +01:00
ca2ba0c9d5
Fix missing dependencies in test session 2021-01-24 19:18:09 +01:00
1bfc7db916
Make Grid.gridify() use only pickup addresses
- ensure a `Restaurant` only has one unique `Order.pickup_address`
- rework `Grid.gridify()` so that only pickup addresses are assigned
  into `Pixel`s
- include database migrations to ensure the data adhere to these
  tighter constraints
2021-01-24 19:04:39 +01:00
0c1ff5338d
Check if predict_at/day is in .totals
- this is a minor sanity check
2021-01-24 18:40:08 +01:00
f36fffdd4d
Add pytest-mock to the dev dependencies 2021-01-24 18:32:07 +01:00
de3e489b39
Adjust flake8 to not consider constants magic 2021-01-24 18:31:02 +01:00
a1da1e9af8
Add matplotlib to the dependencies 2021-01-21 17:15:39 +01:00