Commit graph

49 commits

Author SHA1 Message Date
cb7611d587
Add OrderHistory.avg_daily_demand()
- the method calculates the number of daily `Order`s in a `Pixel`
  withing the `train_horizon` preceding the `predict_day`
2021-02-01 21:50:42 +01:00
67cd58cf16
Add urban_meal_delivery.forecasts.models sub-package
- `*Model`s use the `methods.*.predict()` functions to predict demand
  given an order time series generated by `timify.OrderHistory`
- `models.base.ForecastingModelABC` unifies how all `*Model`s work
  and implements a caching strategy
- implement three `*Model`s for tactical forecasting, based on the
  hets, varima, and rtarima models described in the first research paper
- add overall documentation for `urban_meal_delivery.forecasts` package
- move the fixtures in `tests.forecasts.timify.conftest` to
  `tests.forecasts.conftest` and adjust the horizon of the test horizon
  from two to three weeks
2021-02-01 20:39:52 +01:00
796fdc919c
Add Forecast.from_dataframe() constructor
- this alternative constructor takes the `pd.DataFrame`s from the
  `*Model.predict()` methods and converts them into ORM models
2021-02-01 15:50:30 +01:00
b8952213d8
Add extrapolate_season.predict() function
- the function implements a forecasting "method" similar to the
  seasonal naive method
  => instead of simply taking the last observation given a seasonal lag,
     it linearly extrapolates all observations of the same seasonal lag
     from the past into the future; conceptually, it is like the
     seasonal naive method with built-in smoothing
- the function is tested just like the `arima.predict()` and
  `ets.predict()` functions
  + rename the `tests.forecasts.methods.test_ts_methods` module
    into `tests.forecasts.methods.test_predictions`
- re-organize some constants in the `tests` package
- streamline some docstrings
2021-02-01 11:32:10 +01:00
1d63623dfc
Add Forecast.__repr__() 2021-01-31 21:57:05 +01:00
47ef1f8759
Add OrderHistory.first/last_order() methods
- get the `datetime` of the first or last order within a pixel
- unify some fixtures in `tests.forecasts.timify.conftest`
2021-01-31 21:46:20 +01:00
7b824a4a12
Shorten a couple of names
- rename "total_orders" columns into "n_orders"
- rename `.make_*_time_series()` methods into `.make_*_ts()`
2021-01-31 20:20:55 +01:00
d45c60b764
Add OrderHistory.time_step property 2021-01-31 20:06:23 +01:00
fd404e2b89
Adjust Pixel.__repr__() a tiny bit 2021-01-31 19:34:05 +01:00
63e8e94145
Add Pixel.restaurants property
- the property loads all `Restaurant`s from the database that
  are within the `Pixel`
2021-01-31 19:29:18 +01:00
08b748c867
Move decomposition module into methods sub-package
- move the module
- unify the corresponding tests in `tests.forecasts.methods` sub-package
- make all `predict()` and the `stl()` function round results
- streamline documentation
2021-01-31 18:54:58 +01:00
a5b590b24c
Add Forecast.actual column 2021-01-31 18:29:53 +01:00
4b6d92958d
Add functionality for drawing folium.Maps
- this code is not unit-tested due to the complexity involving
  interactive `folium.Map`s => visual checks give high confidence
2021-01-26 17:07:50 +01:00
605ade4078
Add Pixel.northeast/southwest properties
- the properties are needed for the drawing functionalitites
2021-01-26 17:05:36 +01:00
1bfc7db916
Make Grid.gridify() use only pickup addresses
- ensure a `Restaurant` only has one unique `Order.pickup_address`
- rework `Grid.gridify()` so that only pickup addresses are assigned
  into `Pixel`s
- include database migrations to ensure the data adhere to these
  tighter constraints
2021-01-24 19:04:39 +01:00
0c1ff5338d
Check if predict_at/day is in .totals
- this is a minor sanity check
2021-01-24 18:40:08 +01:00
de3e489b39
Adjust flake8 to not consider constants magic 2021-01-24 18:31:02 +01:00
f37d8adb9d
Add confidence intervals to Forecast model
- add `.low80`, `.high80`, `.low95`, and `.high95` columns
- add check contraints for the confidence intervals
- rename the `.method` column into `.model` for consistency
2021-01-20 16:57:39 +01:00
64482f48d0
Add wrappers for R's "arima" and "ets" functions 2021-01-20 13:06:32 +01:00
98b6830b46
Add stl() function
- `stl()` wraps R's "stl" function in Python
- STL is a decomposition method for time series
2021-01-11 16:10:45 +01:00
b0f2fdde10
Add rpy2 to the dependencies
- add a Jupyter notebook that allows to install all project-external
  dependencies regarding R and R packages
- adjust the GitHub Action workflow to also install R and the R packages
  used within the project
- add a `init_r` module that initializes all R packages globally
  once the `urban_meal_delivery` package is imported
2021-01-11 16:06:58 +01:00
84876047c1
Return the resulting time series as pd.Series 2021-01-10 16:11:40 +01:00
9196c88ed4
Remove pylint from the project 2021-01-09 17:47:45 +01:00
100fac659a
Add OrderHistory.make_real_time_time_series()
- the method slices out a real-time time series from the data within
  an `OrderHistory` object
2021-01-09 17:30:00 +01:00
5330ceb771
Add OrderHistory.make_vertical_time_series()
- the method slices out a vertical time series from the data within
  an `OrderHistory` object
2021-01-09 17:28:55 +01:00
b61db734b6
Add OrderHistory.make_horizontal_time_series()
- the method slices out a horizontal time series from the data within
  an `OrderHistory` object
2021-01-09 16:34:42 +01:00
65d1632e98
Add OrderHistory class
- the main purpose of this class is to manage querying the order totals
  from the database and slice various kinds of time series out of the
  data
- the class holds the former `aggregate_orders()` function as a method
- modularize the corresponding tests
- add `tests.config` with globals used when testing to provide a
  single source of truth for various settings
2021-01-09 16:29:58 +01:00
d5b3efbca1
Add aggregate_orders() function
- the function queries the database and aggregates the ad-hoc orders
  by pixel and time steps into a demand time series
- implement "heavy" integration tests for `aggregate_orders()`
- make `pandas` a package dependency
- streamline the `Config`
2021-01-07 23:35:13 +01:00
e8c97dd7da
Add Forecast model to ORM layer
- the model handles the caching of demand forecasting results
- include the database migration script
2021-01-07 12:59:30 +01:00
54ff377579
Add CLI script to gridify all cities
- reorganize `urban_meal_delivery.console` into a sub-package
- move `tests.db.conftest` fixtures into `tests.conftest`
  => some integration tests regarding CLI scripts need a database
- add `urban_meal_delivery.console.decorators.db_revision` decorator
  to ensure the database is at a certain state before a CLI script runs
- refactor the `urban_meal_delivery.db.grids.Grid.gridify()` constructor:
  - bug fix: even empty `Pixel`s end up in the database temporarily
    => create `Pixel` objects only if an `Address` is to be assigned
       to it
  - streamline code and docstring
  - add further test cases
2021-01-06 16:17:05 +01:00
daa224d041
Rename _*_id columns into just *_id 2021-01-05 22:37:12 +01:00
776112d609
Add Grid.gridify() constructor
- the purpose of this constructor method is to generate all `Pixel`s
  for a `Grid` that have at least one `Address` assigned to them
- fix missing `UniqueConstraint` in `Grid` class => it was not possible
  to create two `Grid`s with the same `.side_length` in different cities
- change the `City.viewport` property into two separate `City.southwest`
  and `City.northeast` properties; also add `City.total_x` and
  `City.total_y` properties for convenience
2021-01-05 18:58:48 +01:00
a1cbb808fd
Integrate the new Location class
- the old `UTMCoordinate` class becomes the new `Location` class
- its main purpose is to represent locations in both lat-long
  coordinates as well as in the UTM system
- remove `Address.__init__()` and `City.__init__()` methods as they
  are not executed for entries retrieved from the database
- simplfiy the `Location.__init__()` => remove `relative_to` argument
2021-01-04 20:33:10 +01:00
2e3ccd14d5
Use globals for the database connection
- remove the factory functions for creating engines and sessions
- define global engine, connection, and session objects to be used
  everywhere in the urban_meal_delivery package
2021-01-04 20:23:55 +01:00
f996376b13
Add ORM models for the pixel grids
- add Grid, Pixel, and AddressPixelAssociation ORM models
- each Grid belongs to a City an is characterized by the side_length
  of all the square Pixels contained in it
- Pixels aggregate Addresses => many-to-many relationship (that is
  modeled with SQLAlchemy's Association Pattern to implement a couple
  of constraints)
2021-01-03 19:33:36 +01:00
6cb4be80f6
Add Address.x and Address.y coordinates
- the Address.x and Address.y properties use the UTMCoordinate class
  behind the scenes
- x and y are simple coordinates in an x-y plane
- the (0, 0) origin is the southwest corner of Address.city.viewport
2021-01-02 16:29:50 +01:00
6f9935072e
Add UTMCoordinate class
- the class is a utility to abstract working with latitude-longitude
  coordinates in their UTM representation (~ "cartesian plane")
- the class's .x and .y properties enable working with simple x-y
  coordinates where the (0, 0) origin is the lower-left of a city's
  viewport
2021-01-02 14:31:59 +01:00
78dba23d5d
Re-factor the ORM tests to use randomized fake data
- create `*Factory` classes with fakerboy and faker that generate
  randomized instances of the ORM models
- add new pytest marker: "db" are the integration tests involving the
  database whereas "e2e" will be all other integration tests
- streamline the docstrings in the ORM models
2020-12-29 15:40:32 +01:00
3e0300cb0e
Disable too-few-public-methods error in pylint 2020-12-16 11:04:43 +01:00
c1064673aa
Isolate configuration related code better
- create the global `config` object inside the
  `urban_meal_delivery.configuration` module
- streamline documentation and comments
2020-12-14 15:15:08 +01:00
9ee9c04a69
Remove python-dotenv from the dependencies
zsh-dotenv automatically loads the environment variables upon entering
the project's root.
2020-12-14 14:26:12 +01:00
a16c260543
Add database migrations
- use Alembic to migrate the PostgreSQL database
  + create initial migration script to set up the database,
    as an alternative to db.Base.metadata.create_all()
  + integrate Alembic into the test suite; the db_engine fixture
    now has two modes:
    * create the latest version of tables all at once
    * invoke `alembic upgrade head`
    => the "e2e" tests are all run twice, once in each mode; this
       ensures that the migration scripts re-create the same database
       schema as db.Base.metadata.create_all() would
    * in both modes, a temporary PostgreSQL schema is used to create the
      tables in
    => could now run "e2e" tests against production database and still
       have isolation
- make the configuration module public (to be used by Alembic)
- adjust linting rules for Alembic
2020-08-11 10:29:58 +02:00
fdcc93a1ea
Add an ORM layer
- use SQLAlchemy (and PostgreSQL) to model the ORM layer
- add the following models:
  + Address => modelling all kinds of addresses
  + City => model the three target cities
  + Courier => model the UDP's couriers
  + Customer => model the UDP's customers
  + Order => model the orders received by the UDP
  + Restaurant => model the restaurants active on the UDP
- so far, the emphasis lies on expression the Foreign Key
  and Check Constraints that are used to validate the assumptions
  inherent to the cleanded data
- provide database-independent unit tests with 100% coverage
- provide additional integration tests ("e2e") that commit data to
  a PostgreSQL instance to validate that the constraints work
- adapt linting rules a bit
2020-08-11 10:28:17 +02:00
9456f86d65
Add a config object
- add the following file:
  + src/urban_meal_delivery/_config.py
- a config module is created holding two sets of configurations:
  + production => against the real database
  + testing => against a database with test data
- the module is "protected" (i.e., underscore) and imported at the
  top level via a proxy-like object `config` that detects in which of
  the two environments the package is being run
2020-08-11 10:27:11 +02:00
882226f0a9
Add technical documentation for the package
- use sphinx to document the developed package
- create a nox session "docs" that builds the docs
- include a skeleton in the docs/ folder
  + how to install the package
  + how to use nox
  + license
2020-08-05 01:44:29 +02:00
126dcf7c39
Include doctests in the test suite
- use xdoctest to validate all code snippets in docstrings
- add xdoctest to the nox session "test"
2020-08-05 00:02:40 +02:00
97d714d9ee
Add CLI entry point umd
- add the following file:
  + src/urban_meal_delivery/console.py => click-based CLI tools
  + tests/test_console.py => tests for the module above
- add a CLI entry point `umd`:
  + implement the --version / -V option
    to show the installed package's version
  + rework to package's top-level:
    * add a __pkg_name__ variable to parameterize the package name
  + add unit and integration tests
- fix that pylint cannot know the proper order of imports in the
  isolated nox session
2020-08-04 21:14:40 +02:00
9fc5b4816a
Add a testing tool chain
- use pytest as the base, measure coverage with pytest-cov
  + configure coverage to include branches and specify source locations
  + configure pytest to enforce explicit markers
- add a package for the test suite under tests/
- add a `__version__` identifier at the package's root
  + it is dynamically assigned the version of the installed package
  + the version is PEP440 compliant and follows a strict subset of
    semantic versioning: x.y.z[.devN] where x, y, z, and N are all
    non-negative integers
  + add module with tests for the __version__
- add a nox session "test" that runs the test suite
- use flake8 to lint pytest for consistent style
2020-08-04 00:09:29 +02:00
de8afa6335
Initial commit
- add the following files:
  + .gitignore => ignore poetry's and pyenv's artifacts
  + LICENSE.txt => MIT license as the project is scientific research
  + README.md => rough description of the project
  + poetry.lock
  + pyproject.toml
  + src/urban_meal_delivery/__init__.py => source code package
- use a "src" layout structure:
  + ensure that pytest runs the tests against a packaged and pip
    installed version of the source code, not the *.py files in the
    project directory
  + more info: https://hynek.me/articles/testing-packaging/
2020-08-03 20:19:42 +02:00