Commit graph

117 commits

Author SHA1 Message Date
f5ced933d6
Merge branch 'tactical-forecasting' into main
Some checks failed
CI / fast (without R) (push) Has been cancelled
CI / slow (with R) (push) Has been cancelled
2021-03-01 14:13:21 +01:00
9d6de9d98c
Create tactical demand forecasts
- the first notebook runs the tactical-forecasts command
- the second notebook describes the tactical demand forecasting process
  + demand aggregation on a per-pixel level
  + time series generation: horizontal, vertical, and real-time time series
  + STL decomposition into seasonal, trend, and residual components
  + choosing the most promising forecasting model
  + predicting demand with various models
- fix where to re-start the forecasting process after it was interrupted
- enable the heuristic for choosing the most promising model
  to also work for 7 training weeks
2021-02-09 17:06:37 +01:00
21d012050c
Add visualization scripts for customers/restaurants
- the two notebook files are helpful in visualizing all relevant
  pickup (red) or delivery (blue) locations from the point of view
  of either e restaurant or a customer
2021-02-04 16:19:12 +01:00
40471b883c
Add infos on tactical demand forecasting 2021-02-04 16:12:25 +01:00
d494a7908b
Merge branch 'gridification' into main
Some checks failed
CI / fast (without R) (push) Has been cancelled
CI / slow (with R) (push) Has been cancelled
2021-02-04 15:39:46 +01:00
a89b9497f2
Visualize the pixel grids
- add notebook that runs the plotting code
- add three visualizations per city:
  + all addresses, colored by zip code
  + all restaurants, incl. the number of received orders
  + all restaurants on a grid with pixel side length of 1000m
2021-02-04 15:37:16 +01:00
1e263a4b98
Run the gridification script 2021-02-04 15:29:42 +01:00
28a7c7451c
Rename existing notebooks using order numbers 2021-02-04 15:10:14 +01:00
241e7ed81f
Merge branch 'release-0.3.0' into main
Some checks failed
CI / fast (without R) (push) Has been cancelled
CI / slow (with R) (push) Has been cancelled
2021-02-04 13:18:32 +01:00
d4ca85b55a
Finalize release 0.3.0 2021-02-04 13:13:26 +01:00
0da86e5f07
Pin the dependencies ...
... after upgrading:
- alembic
- matplotlib
- pandas
- rpy2
- sqlalchemy
- statsmodels
- dev dependencies
  + coverage     + factory-boy    + faker                 + nox
  + packaging    + pre-commit     + flake8-annotations    + pytest
  + pytest-cov   + sphinx
- research dependencies
  + numpy        + pyty
- transient dependencies
  + astpretty    + atomicwrites   + bleach                + chardet
  + colorlog     + darglint       + flake8-comprehensions + gitpython
  + identify     + ipykernel      + ipython               + jedi
  + jinja2       + jupyter-client + jupyter-core          + mako
  + nbformat     + nest-asyncio   + notebook              + parso
  + pluggy       + prompt-toolkit + ptyprocess            + pygments
  + pyyaml       + pyzmq          + requests              + smmap
  + terminado    + textfixtures   + snowballstemmer       + typed-ast
  + urllib3      + virtualenv

- fix SQL statements written in raw text
2021-02-04 13:12:47 +01:00
50b35a8284
Add CLI script to run tactical forecasting heuristic 2021-02-04 12:05:43 +01:00
23391c2fa4
Adjust OrderHistory.choose_tactical_model() heuristic
- use the `HorizontalSMAModel` for low demand
- use the `TrivialModel` for no demand
2021-02-02 15:20:02 +01:00
3f5b4a50bb
Rename Forecast.training_horizon into .train_horizon
- we use that shorter name in `urban_meal_delivery.forecasts.*`
  and want to be consistent in the ORM layer as well
2021-02-02 13:09:09 +01:00
6fd16f2a6c
Add TrivialModel
- the trivial model simply predicts `0` demand for all time steps
2021-02-02 12:45:26 +01:00
015d304306
Add HorizontalSMAModel
- the model applies a simple moving average on horizontal time series
- refactor `db.Forecast.from_dataframe()` to correctly convert
  `float('NaN')` values into `None`; otherwise, SQLAlchemy complains
2021-02-02 12:40:53 +01:00
af82951485
Add OrderHistory.choose_tactical_model()
- the method implements a heuristic from the first research paper
  that chooses the most promising forecasting `*Model` based on
  the average daily demand in a `Pixel` for a given `train_horizon`
- adjust the test scenario => `LONG_TRAIN_HORIZON` becomes `8`
  as that is part of the rule implemented in the heuristic
2021-02-02 11:29:27 +01:00
8926e9ff28
Fix nox session for slow CI tests
- when running tests marked with "r" we still must not run tests
  marked with "db" on the CI server
2021-02-01 22:00:47 +01:00
cb7611d587
Add OrderHistory.avg_daily_demand()
- the method calculates the number of daily `Order`s in a `Pixel`
  withing the `train_horizon` preceding the `predict_day`
2021-02-01 21:50:42 +01:00
67cd58cf16
Add urban_meal_delivery.forecasts.models sub-package
- `*Model`s use the `methods.*.predict()` functions to predict demand
  given an order time series generated by `timify.OrderHistory`
- `models.base.ForecastingModelABC` unifies how all `*Model`s work
  and implements a caching strategy
- implement three `*Model`s for tactical forecasting, based on the
  hets, varima, and rtarima models described in the first research paper
- add overall documentation for `urban_meal_delivery.forecasts` package
- move the fixtures in `tests.forecasts.timify.conftest` to
  `tests.forecasts.conftest` and adjust the horizon of the test horizon
  from two to three weeks
2021-02-01 20:39:52 +01:00
796fdc919c
Add Forecast.from_dataframe() constructor
- this alternative constructor takes the `pd.DataFrame`s from the
  `*Model.predict()` methods and converts them into ORM models
2021-02-01 15:50:30 +01:00
b8952213d8
Add extrapolate_season.predict() function
- the function implements a forecasting "method" similar to the
  seasonal naive method
  => instead of simply taking the last observation given a seasonal lag,
     it linearly extrapolates all observations of the same seasonal lag
     from the past into the future; conceptually, it is like the
     seasonal naive method with built-in smoothing
- the function is tested just like the `arima.predict()` and
  `ets.predict()` functions
  + rename the `tests.forecasts.methods.test_ts_methods` module
    into `tests.forecasts.methods.test_predictions`
- re-organize some constants in the `tests` package
- streamline some docstrings
2021-02-01 11:32:10 +01:00
1d63623dfc
Add Forecast.__repr__() 2021-01-31 21:57:05 +01:00
47ef1f8759
Add OrderHistory.first/last_order() methods
- get the `datetime` of the first or last order within a pixel
- unify some fixtures in `tests.forecasts.timify.conftest`
2021-01-31 21:46:20 +01:00
7b824a4a12
Shorten a couple of names
- rename "total_orders" columns into "n_orders"
- rename `.make_*_time_series()` methods into `.make_*_ts()`
2021-01-31 20:20:55 +01:00
d45c60b764
Add OrderHistory.time_step property 2021-01-31 20:06:23 +01:00
fd404e2b89
Adjust Pixel.__repr__() a tiny bit 2021-01-31 19:34:05 +01:00
63e8e94145
Add Pixel.restaurants property
- the property loads all `Restaurant`s from the database that
  are within the `Pixel`
2021-01-31 19:29:18 +01:00
08b748c867
Move decomposition module into methods sub-package
- move the module
- unify the corresponding tests in `tests.forecasts.methods` sub-package
- make all `predict()` and the `stl()` function round results
- streamline documentation
2021-01-31 18:54:58 +01:00
a5b590b24c
Add Forecast.actual column 2021-01-31 18:29:53 +01:00
6429165aaf
Add statsmodels to the dependencies 2021-01-31 18:24:03 +01:00
4b6d92958d
Add functionality for drawing folium.Maps
- this code is not unit-tested due to the complexity involving
  interactive `folium.Map`s => visual checks give high confidence
2021-01-26 17:07:50 +01:00
605ade4078
Add Pixel.northeast/southwest properties
- the properties are needed for the drawing functionalitites
2021-01-26 17:05:36 +01:00
ca2ba0c9d5
Fix missing dependencies in test session 2021-01-24 19:18:09 +01:00
1bfc7db916
Make Grid.gridify() use only pickup addresses
- ensure a `Restaurant` only has one unique `Order.pickup_address`
- rework `Grid.gridify()` so that only pickup addresses are assigned
  into `Pixel`s
- include database migrations to ensure the data adhere to these
  tighter constraints
2021-01-24 19:04:39 +01:00
0c1ff5338d
Check if predict_at/day is in .totals
- this is a minor sanity check
2021-01-24 18:40:08 +01:00
f36fffdd4d
Add pytest-mock to the dev dependencies 2021-01-24 18:32:07 +01:00
de3e489b39
Adjust flake8 to not consider constants magic 2021-01-24 18:31:02 +01:00
a1da1e9af8
Add matplotlib to the dependencies 2021-01-21 17:15:39 +01:00
2339100371
Add folium to the dependencies 2021-01-21 11:47:22 +01:00
f37d8adb9d
Add confidence intervals to Forecast model
- add `.low80`, `.high80`, `.low95`, and `.high95` columns
- add check contraints for the confidence intervals
- rename the `.method` column into `.model` for consistency
2021-01-20 16:57:39 +01:00
64482f48d0
Add wrappers for R's "arima" and "ets" functions 2021-01-20 13:06:32 +01:00
98b6830b46
Add stl() function
- `stl()` wraps R's "stl" function in Python
- STL is a decomposition method for time series
2021-01-11 16:10:45 +01:00
b0f2fdde10
Add rpy2 to the dependencies
- add a Jupyter notebook that allows to install all project-external
  dependencies regarding R and R packages
- adjust the GitHub Action workflow to also install R and the R packages
  used within the project
- add a `init_r` module that initializes all R packages globally
  once the `urban_meal_delivery` package is imported
2021-01-11 16:06:58 +01:00
84876047c1
Return the resulting time series as pd.Series 2021-01-10 16:11:40 +01:00
9196c88ed4
Remove pylint from the project 2021-01-09 17:47:45 +01:00
100fac659a
Add OrderHistory.make_real_time_time_series()
- the method slices out a real-time time series from the data within
  an `OrderHistory` object
2021-01-09 17:30:00 +01:00
5330ceb771
Add OrderHistory.make_vertical_time_series()
- the method slices out a vertical time series from the data within
  an `OrderHistory` object
2021-01-09 17:28:55 +01:00
b61db734b6
Add OrderHistory.make_horizontal_time_series()
- the method slices out a horizontal time series from the data within
  an `OrderHistory` object
2021-01-09 16:34:42 +01:00
65d1632e98
Add OrderHistory class
- the main purpose of this class is to manage querying the order totals
  from the database and slice various kinds of time series out of the
  data
- the class holds the former `aggregate_orders()` function as a method
- modularize the corresponding tests
- add `tests.config` with globals used when testing to provide a
  single source of truth for various settings
2021-01-09 16:29:58 +01:00