alexander/urban-meal-delivery
1
1
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity Actions

Compare commits

base: alexander:main
Branches Tags
alexander:main
alexander:develop
alexander:v0.4.0.post0
alexander:v0.4.0
alexander:v0.3.0.post1
alexander:v0.3.0.post0
alexander:v0.3.0
alexander:v0.2.0.post0
alexander:v0.2.0
alexander:v0.1.0
..
compare: alexander:v0.1.0
Branches Tags
alexander:develop
alexander:main
alexander:v0.4.0.post0
alexander:v0.4.0
alexander:v0.3.0.post1
alexander:v0.3.0.post0
alexander:v0.3.0
alexander:v0.2.0.post0
alexander:v0.2.0
alexander:v0.1.0

No commits in common. "main" and "v0.1.0" have entirely different histories.
main ... v0.1.0

118 changed files with 940 additions and 643680 deletions
Download patch file Download diff file Expand all files Collapse all files

24
.github/workflows/tests.yml vendored
View file

@ -1,8 +1,7 @@
name: CI name: CI
on: push on: push
jobs: jobs:
fast-tests: tests:
name: fast (without R)
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v2 - uses: actions/checkout@v2
@ -11,22 +10,5 @@ jobs:
python-version: 3.8 python-version: 3.8
architecture: x64 architecture: x64
- run: pip install nox==2020.5.24 - run: pip install nox==2020.5.24
- run: pip install poetry==1.1.4 - run: pip install poetry==1.0.10
- run: nox -s format lint ci-tests-fast safety docs - run: nox
slow-tests:
name: slow (with R)
runs-on: ubuntu-latest
env:
R_LIBS: .r_libs
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v1
with:
python-version: 3.8
architecture: x64
- run: mkdir .r_libs
- run: sudo apt-get install r-base r-base-dev libcurl4-openssl-dev libxml2-dev patchelf
- run: R -e "install.packages('forecast')"
- run: pip install nox==2020.5.24
- run: pip install poetry==1.1.4
- run: nox -s ci-tests-slow

5
.gitignore vendored
View file

@ -1,7 +1,4 @@
.cache/ .cache/
**/*.egg-info/ *.egg-info/
.env
.idea/
**/.ipynb_checkpoints/
.python-version .python-version
.venv/ .venv/

3
.gitmodules vendored
View file

@ -1,3 +0,0 @@
[submodule "research/papers/demand-forecasting"]
path = research/papers/demand-forecasting
url = git@github.com:webartifex/urban-meal-delivery-demand-forecasting.git

26
.pre-commit-config.yaml
View file

@ -4,30 +4,18 @@ repos:
# Run the local formatting, linting, and testing tool chains. # Run the local formatting, linting, and testing tool chains.
- repo: local - repo: local
hooks: hooks:
- id: local-fix-branch-references - id: local-pre-commit-checks
name: Check for wrong branch references name: Run code formatters and linters
entry: poetry run nox -s fix-branch-references -- entry: poetry run nox -s pre-commit --
language: system
stages: [commit]
types: [text]
- id: local-format
name: Format the source files
entry: poetry run nox -s format --
language: system language: system
stages: [commit] stages: [commit]
types: [python] types: [python]
- id: local-lint - id: local-pre-merge-checks
name: Lint the source files
entry: poetry run nox -s lint --
language: system
stages: [commit]
types: [python]
- id: local-test-suite
name: Run the entire test suite name: Run the entire test suite
entry: poetry run nox -s test-suite -- entry: poetry run nox -s pre-merge --
language: system language: system
stages: [merge-commit] stages: [merge-commit, push]
types: [text] types: [python]
# Enable hooks provided by the pre-commit project to # Enable hooks provided by the pre-commit project to
# enforce rules that local tools could not that easily. # enforce rules that local tools could not that easily.
- repo: https://github.com/pre-commit/pre-commit-hooks - repo: https://github.com/pre-commit/pre-commit-hooks

75
README.md
View file

@ -1,81 +1,16 @@
# Urban Meal Delivery # Urban Meal Delivery
This repository holds code This repository holds code
analyzing the data of an undisclosed urban meal delivery platform (UDP) analyzing the data of an undisclosed urban meal delivery platform
operating in France from January 2016 to January 2017. operating in France from January 2016 to January 2017.
The goal is to The goal is to
optimize the platform's delivery process involving independent couriers. optimize the platform's delivery process involving independent couriers.
The analysis is structured into three aspects
## Structure
The analysis is structured into the following stages
that iteratively build on each other. that iteratively build on each other.
## Real-time Demand Forecasting
### Data Cleaning ## Predictive Routing
The UDP provided its raw data as a PostgreSQL dump. ## Shift & Capacity Planning
This [notebook](https://nbviewer.jupyter.org/github/webartifex/urban-meal-delivery/blob/main/research/01_clean_data.ipynb)
cleans the data extensively
and maps them onto the [ORM models](https://github.com/webartifex/urban-meal-delivery/tree/main/src/urban_meal_delivery/db)
defined in the `urban-meal-delivery` package
that is developed in the [src/](https://github.com/webartifex/urban-meal-delivery/tree/main/src) folder
and contains all source code to drive the analyses.
Due to a non-disclosure agreement with the UDP,
neither the raw nor the cleaned data are published as of now.
However, previews of the data can be seen throughout the [research/](https://github.com/webartifex/urban-meal-delivery/tree/main/research) folder.
### Tactical Demand Forecasting
Before any optimizations of the UDP's operations are done,
a **demand forecasting** system for *tactical* purposes is implemented.
To achieve that, the cities first undergo a **gridification** step
where each *pickup* location is assigned into a pixel on a "checker board"-like grid.
The main part of the source code that implements that is in this [file](https://github.com/webartifex/urban-meal-delivery/blob/main/src/urban_meal_delivery/db/grids.py#L60).
Visualizations of the various grids can be found in the [visualizations/](https://github.com/webartifex/urban-meal-delivery/tree/main/research/visualizations) folder
and in this [notebook](https://nbviewer.jupyter.org/github/webartifex/urban-meal-delivery/blob/main/research/03_grid_visualizations.ipynb).
Then, demand is aggregated on a per-pixel level
and different kinds of order time series are generated.
The latter are the input to different kinds of forecasting `*Model`s.
They all have in common that they predict demand into the *short-term* future (e.g., one hour)
and are thus used for tactical purposes, in particular predictive routing (cf., next section).
The details of how this works can be found in the first academic paper
published in the context of this research project
and titled "*Real-time Demand Forecasting for an Urban Delivery Platform*"
(cf., the [repository](https://github.com/webartifex/urban-meal-delivery-demand-forecasting) with the LaTeX files).
All demand forecasting related code is in the [forecasts/](https://github.com/webartifex/urban-meal-delivery/tree/main/src/urban_meal_delivery/forecasts) sub-package.
### Predictive Routing
### Shift & Capacity Planning
## Installation & Contribution
To play with the code developed for the analyses,
you can clone the project with [git](https://git-scm.com/)
and install the contained `urban-meal-delivery` package
and all its dependencies
in a [virtual environment](https://docs.python.org/3/tutorial/venv.html)
with [poetry](https://python-poetry.org/docs/):
`git clone https://github.com/webartifex/urban-meal-delivery.git`
and
`poetry install --extras research`
The `--extras` option is necessary as the non-develop dependencies
are structured in the [pyproject.toml](https://github.com/webartifex/urban-meal-delivery/blob/main/pyproject.toml) file
into dependencies related to only the `urban-meal-delivery` source code package
and dependencies used to run the [Jupyter](https://jupyter.org/) environment
with the analyses.
Contributions are welcome.
Use the [issues](https://github.com/webartifex/urban-meal-delivery/issues) tab.
The project is licensed under the [MIT license](https://github.com/webartifex/urban-meal-delivery/blob/main/LICENSE.txt).

44
alembic.ini
View file

@ -1,44 +0,0 @@
[alembic]
file_template = rev_%%(year)d%%(month).2d%%(day).2d_%%(hour).2d_%%(rev)s_%%(slug)s
script_location = %(here)s/migrations
[post_write_hooks]
hooks=black
black.type=console_scripts
black.entrypoint=black
# The following is taken from the default alembic.ini file.
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARN
handlers = console
qualname =
[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S

2
docs/conf.py
View file

@ -5,7 +5,7 @@ import urban_meal_delivery as umd
project = umd.__pkg_name__ project = umd.__pkg_name__
author = umd.__author__ author = umd.__author__
copyright = f'2020, {author}' copyright = f'2020, {author}' # pylint:disable=redefined-builtin
version = release = umd.__version__ version = release = umd.__version__
extensions = [ extensions = [

4
migrations/README.md
View file

@ -1,4 +0,0 @@
# Database Migrations
This project uses [alembic](https://alembic.sqlalchemy.org/en/latest)
to run the database migrations

49
migrations/env.py
View file

@ -1,49 +0,0 @@
"""Configure Alembic's migration environment."""
import os
from logging import config as log_config
import sqlalchemy as sa
from alembic import context
from urban_meal_delivery import config as umd_config
from urban_meal_delivery import db
# Disable the --sql option, a.k.a, the "offline mode".
if context.is_offline_mode():
raise NotImplementedError('The --sql option is not implemented in this project')
# Set up the default Python logger from the alembic.ini file.
log_config.fileConfig(context.config.config_file_name)
def include_object(obj, _name, type_, _reflected, _compare_to):
"""Only include the clean schema into --autogenerate migrations."""
if ( # noqa:WPS337
type_ in {'table', 'column'}
and hasattr(obj, 'schema') # noqa:WPS421 => fix for rare edge case
and obj.schema != umd_config.CLEAN_SCHEMA
):
return False
return True
engine = sa.create_engine(umd_config.DATABASE_URI)
with engine.connect() as connection:
context.configure(
connection=connection,
include_object=include_object,
target_metadata=db.Base.metadata,
version_table='{alembic_table}{test_schema}'.format(
alembic_table=umd_config.ALEMBIC_TABLE,
test_schema=(f'_{umd_config.CLEAN_SCHEMA}' if os.getenv('TESTING') else ''),
),
version_table_schema=umd_config.ALEMBIC_TABLE_SCHEMA,
)
with context.begin_transaction():
context.run_migrations()

31
migrations/script.py.mako
View file

@ -1,31 +0,0 @@
"""${message}.
Revision: # ${up_revision} at ${create_date}
Revises: # ${down_revision | comma,n}
"""
import os
import sqlalchemy as sa
from alembic import op
${imports if imports else ""}
from urban_meal_delivery import configuration
revision = ${repr(up_revision)}
down_revision = ${repr(down_revision)}
branch_labels = ${repr(branch_labels)}
depends_on = ${repr(depends_on)}
config = configuration.make_config('testing' if os.getenv('TESTING') else 'production')
def upgrade():
"""Upgrade to revision ${up_revision}."""
${upgrades if upgrades else "pass"}
def downgrade():
"""Downgrade to revision ${down_revision}."""
${downgrades if downgrades else "pass"}

802
migrations/versions/rev_20200806_23_f11cd76d2f45_create_the_database.py
View file

@ -1,802 +0,0 @@
"""Create the database from scratch.
Revision: #f11cd76d2f45 at 2020-08-06 23:24:32
"""
import os
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
from urban_meal_delivery import configuration
revision = 'f11cd76d2f45'
down_revision = None
branch_labels = None
depends_on = None
config = configuration.make_config('testing' if os.getenv('TESTING') else 'production')
def upgrade():
"""Upgrade to revision f11cd76d2f45."""
op.execute(f'CREATE SCHEMA {config.CLEAN_SCHEMA};')
op.create_table( # noqa:ECE001
'cities',
sa.Column('id', sa.SmallInteger(), autoincrement=False, nullable=False),
sa.Column('name', sa.Unicode(length=10), nullable=False),
sa.Column('kml', sa.UnicodeText(), nullable=False),
sa.Column('center_latitude', postgresql.DOUBLE_PRECISION(), nullable=False),
sa.Column('center_longitude', postgresql.DOUBLE_PRECISION(), nullable=False),
sa.Column('northeast_latitude', postgresql.DOUBLE_PRECISION(), nullable=False),
sa.Column('northeast_longitude', postgresql.DOUBLE_PRECISION(), nullable=False),
sa.Column('southwest_latitude', postgresql.DOUBLE_PRECISION(), nullable=False),
sa.Column('southwest_longitude', postgresql.DOUBLE_PRECISION(), nullable=False),
sa.Column('initial_zoom', sa.SmallInteger(), nullable=False),
sa.PrimaryKeyConstraint('id', name=op.f('pk_cities')),
*(
[ # noqa:WPS504
sa.ForeignKeyConstraint(
['id'],
[f'{config.ORIGINAL_SCHEMA}.cities.id'],
name=op.f('pk_cities_sanity'),
onupdate='RESTRICT',
ondelete='RESTRICT',
),
]
if not config.TESTING
else []
),
schema=config.CLEAN_SCHEMA,
)
op.create_table( # noqa:ECE001
'couriers',
sa.Column('id', sa.Integer(), autoincrement=False, nullable=False),
sa.Column('created_at', sa.DateTime(), nullable=False),
sa.Column('vehicle', sa.Unicode(length=10), nullable=False),
sa.Column('speed', postgresql.DOUBLE_PRECISION(), nullable=False),
sa.Column('capacity', sa.SmallInteger(), nullable=False),
sa.Column('pay_per_hour', sa.SmallInteger(), nullable=False),
sa.Column('pay_per_order', sa.SmallInteger(), nullable=False),
sa.CheckConstraint(
"vehicle IN ('bicycle', 'motorcycle')",
name=op.f('ck_couriers_on_available_vehicle_types'),
),
sa.CheckConstraint(
'0 <= capacity AND capacity <= 200',
name=op.f('ck_couriers_on_capacity_under_200_liters'),
),
sa.CheckConstraint(
'0 <= pay_per_hour AND pay_per_hour <= 1500',
name=op.f('ck_couriers_on_realistic_pay_per_hour'),
),
sa.CheckConstraint(
'0 <= pay_per_order AND pay_per_order <= 650',
name=op.f('ck_couriers_on_realistic_pay_per_order'),
),
sa.CheckConstraint(
'0 <= speed AND speed <= 30', name=op.f('ck_couriers_on_realistic_speed'),
),
sa.PrimaryKeyConstraint('id', name=op.f('pk_couriers')),
*(
[ # noqa:WPS504
sa.ForeignKeyConstraint(
['id'],
[f'{config.ORIGINAL_SCHEMA}.couriers.id'],
name=op.f('pk_couriers_sanity'),
onupdate='RESTRICT',
ondelete='RESTRICT',
),
]
if not config.TESTING
else []
),
schema=config.CLEAN_SCHEMA,
)
op.create_table(
'customers',
sa.Column('id', sa.Integer(), autoincrement=False, nullable=False),
sa.PrimaryKeyConstraint('id', name=op.f('pk_customers')),
schema=config.CLEAN_SCHEMA,
)
op.create_table( # noqa:ECE001
'addresses',
sa.Column('id', sa.Integer(), autoincrement=False, nullable=False),
sa.Column('primary_id', sa.Integer(), nullable=False),
sa.Column('created_at', sa.DateTime(), nullable=False),
sa.Column('place_id', sa.Unicode(length=120), nullable=False),
sa.Column('latitude', postgresql.DOUBLE_PRECISION(), nullable=False),
sa.Column('longitude', postgresql.DOUBLE_PRECISION(), nullable=False),
sa.Column('city_id', sa.SmallInteger(), nullable=False),
sa.Column('city', sa.Unicode(length=25), nullable=False),
sa.Column('zip_code', sa.Integer(), nullable=False),
sa.Column('street', sa.Unicode(length=80), nullable=False),
sa.Column('floor', sa.SmallInteger(), nullable=True),
sa.CheckConstraint(
'-180 <= longitude AND longitude <= 180',
name=op.f('ck_addresses_on_longitude_between_180_degrees'),
),
sa.CheckConstraint(
'-90 <= latitude AND latitude <= 90',
name=op.f('ck_addresses_on_latitude_between_90_degrees'),
),
sa.CheckConstraint(
'0 <= floor AND floor <= 40', name=op.f('ck_addresses_on_realistic_floor'),
),
sa.CheckConstraint(
'30000 <= zip_code AND zip_code <= 99999',
name=op.f('ck_addresses_on_valid_zip_code'),
),
sa.ForeignKeyConstraint(
['city_id'],
[f'{config.CLEAN_SCHEMA}.cities.id'],
name=op.f('fk_addresses_to_cities_via_city_id'),
onupdate='RESTRICT',
ondelete='RESTRICT',
),
sa.ForeignKeyConstraint(
['primary_id'],
[f'{config.CLEAN_SCHEMA}.addresses.id'],
name=op.f('fk_addresses_to_addresses_via_primary_id'),
onupdate='RESTRICT',
ondelete='RESTRICT',
),
sa.PrimaryKeyConstraint('id', name=op.f('pk_addresses')),
*(
[ # noqa:WPS504
sa.ForeignKeyConstraint(
['id'],
[f'{config.ORIGINAL_SCHEMA}.addresses.id'],
name=op.f('pk_addresses_sanity'),
onupdate='RESTRICT',
ondelete='RESTRICT',
),
]
if not config.TESTING
else []
),
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_addresses_on_city_id'),
'addresses',
['city_id'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_addresses_on_place_id'),
'addresses',
['place_id'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_addresses_on_primary_id'),
'addresses',
['primary_id'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_addresses_on_zip_code'),
'addresses',
['zip_code'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_table( # noqa:ECE001
'restaurants',
sa.Column('id', sa.SmallInteger(), autoincrement=False, nullable=False),
sa.Column('created_at', sa.DateTime(), nullable=False),
sa.Column('name', sa.Unicode(length=45), nullable=False),
sa.Column('address_id', sa.Integer(), nullable=False),
sa.Column('estimated_prep_duration', sa.SmallInteger(), nullable=False),
sa.CheckConstraint(
'0 <= estimated_prep_duration AND estimated_prep_duration <= 2400',
name=op.f('ck_restaurants_on_realistic_estimated_prep_duration'),
),
sa.ForeignKeyConstraint(
['address_id'],
[f'{config.CLEAN_SCHEMA}.addresses.id'],
name=op.f('fk_restaurants_to_addresses_via_address_id'),
onupdate='RESTRICT',
ondelete='RESTRICT',
),
sa.PrimaryKeyConstraint('id', name=op.f('pk_restaurants')),
*(
[ # noqa:WPS504
sa.ForeignKeyConstraint(
['id'],
[f'{config.ORIGINAL_SCHEMA}.businesses.id'],
name=op.f('pk_restaurants_sanity'),
onupdate='RESTRICT',
ondelete='RESTRICT',
),
]
if not config.TESTING
else []
),
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_restaurants_on_address_id'),
'restaurants',
['address_id'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_table( # noqa:ECE001
'orders',
sa.Column('id', sa.Integer(), autoincrement=False, nullable=False),
sa.Column('delivery_id', sa.Integer(), nullable=True),
sa.Column('customer_id', sa.Integer(), nullable=False),
sa.Column('placed_at', sa.DateTime(), nullable=False),
sa.Column('ad_hoc', sa.Boolean(), nullable=False),
sa.Column('scheduled_delivery_at', sa.DateTime(), nullable=True),
sa.Column('scheduled_delivery_at_corrected', sa.Boolean(), nullable=True),
sa.Column('first_estimated_delivery_at', sa.DateTime(), nullable=True),
sa.Column('cancelled', sa.Boolean(), nullable=False),
sa.Column('cancelled_at', sa.DateTime(), nullable=True),
sa.Column('cancelled_at_corrected', sa.Boolean(), nullable=True),
sa.Column('sub_total', sa.Integer(), nullable=False),
sa.Column('delivery_fee', sa.SmallInteger(), nullable=False),
sa.Column('total', sa.Integer(), nullable=False),
sa.Column('restaurant_id', sa.SmallInteger(), nullable=False),
sa.Column('restaurant_notified_at', sa.DateTime(), nullable=True),
sa.Column('restaurant_notified_at_corrected', sa.Boolean(), nullable=True),
sa.Column('restaurant_confirmed_at', sa.DateTime(), nullable=True),
sa.Column('restaurant_confirmed_at_corrected', sa.Boolean(), nullable=True),
sa.Column('estimated_prep_duration', sa.Integer(), nullable=True),
sa.Column('estimated_prep_duration_corrected', sa.Boolean(), nullable=True),
sa.Column('estimated_prep_buffer', sa.Integer(), nullable=False),
sa.Column('courier_id', sa.Integer(), nullable=True),
sa.Column('dispatch_at', sa.DateTime(), nullable=True),
sa.Column('dispatch_at_corrected', sa.Boolean(), nullable=True),
sa.Column('courier_notified_at', sa.DateTime(), nullable=True),
sa.Column('courier_notified_at_corrected', sa.Boolean(), nullable=True),
sa.Column('courier_accepted_at', sa.DateTime(), nullable=True),
sa.Column('courier_accepted_at_corrected', sa.Boolean(), nullable=True),
sa.Column('utilization', sa.SmallInteger(), nullable=False),
sa.Column('pickup_address_id', sa.Integer(), nullable=False),
sa.Column('reached_pickup_at', sa.DateTime(), nullable=True),
sa.Column('pickup_at', sa.DateTime(), nullable=True),
sa.Column('pickup_at_corrected', sa.Boolean(), nullable=True),
sa.Column('pickup_not_confirmed', sa.Boolean(), nullable=True),
sa.Column('left_pickup_at', sa.DateTime(), nullable=True),
sa.Column('left_pickup_at_corrected', sa.Boolean(), nullable=True),
sa.Column('delivery_address_id', sa.Integer(), nullable=False),
sa.Column('reached_delivery_at', sa.DateTime(), nullable=True),
sa.Column('delivery_at', sa.DateTime(), nullable=True),
sa.Column('delivery_at_corrected', sa.Boolean(), nullable=True),
sa.Column('delivery_not_confirmed', sa.Boolean(), nullable=True),
sa.Column('courier_waited_at_delivery', sa.Boolean(), nullable=True),
sa.Column('logged_delivery_distance', sa.SmallInteger(), nullable=True),
sa.Column('logged_avg_speed', postgresql.DOUBLE_PRECISION(), nullable=True),
sa.Column('logged_avg_speed_distance', sa.SmallInteger(), nullable=True),
sa.CheckConstraint(
'0 <= estimated_prep_buffer AND estimated_prep_buffer <= 900',
name=op.f('ck_orders_on_estimated_prep_buffer_between_0_and_900'),
),
sa.CheckConstraint(
'0 <= estimated_prep_duration AND estimated_prep_duration <= 2700',
name=op.f('ck_orders_on_estimated_prep_duration_between_0_and_2700'),
),
sa.CheckConstraint(
'0 <= utilization AND utilization <= 100',
name=op.f('ck_orders_on_utilization_between_0_and_100'),
),
sa.CheckConstraint(
'(cancelled_at IS NULL AND cancelled_at_corrected IS NULL) OR (cancelled_at IS NULL AND cancelled_at_corrected IS TRUE) OR (cancelled_at IS NOT NULL AND cancelled_at_corrected IS NOT NULL)', # noqa:E501
name=op.f('ck_orders_on_corrections_only_for_set_value_1'),
),
sa.CheckConstraint(
'(courier_accepted_at IS NULL AND courier_accepted_at_corrected IS NULL) OR (courier_accepted_at IS NULL AND courier_accepted_at_corrected IS TRUE) OR (courier_accepted_at IS NOT NULL AND courier_accepted_at_corrected IS NOT NULL)', # noqa:E501
name=op.f('ck_orders_on_corrections_only_for_set_value_7'),
),
sa.CheckConstraint(
'(courier_notified_at IS NULL AND courier_notified_at_corrected IS NULL) OR (courier_notified_at IS NULL AND courier_notified_at_corrected IS TRUE) OR (courier_notified_at IS NOT NULL AND courier_notified_at_corrected IS NOT NULL)', # noqa:E501
name=op.f('ck_orders_on_corrections_only_for_set_value_6'),
),
sa.CheckConstraint(
'(delivery_at IS NULL AND delivery_at_corrected IS NULL) OR (delivery_at IS NULL AND delivery_at_corrected IS TRUE) OR (delivery_at IS NOT NULL AND delivery_at_corrected IS NOT NULL)', # noqa:E501
name=op.f('ck_orders_on_corrections_only_for_set_value_10'),
),
sa.CheckConstraint(
'(dispatch_at IS NULL AND dispatch_at_corrected IS NULL) OR (dispatch_at IS NULL AND dispatch_at_corrected IS TRUE) OR (dispatch_at IS NOT NULL AND dispatch_at_corrected IS NOT NULL)', # noqa:E501
name=op.f('ck_orders_on_corrections_only_for_set_value_5'),
),
sa.CheckConstraint(
'(estimated_prep_duration IS NULL AND estimated_prep_duration_corrected IS NULL) OR (estimated_prep_duration IS NULL AND estimated_prep_duration_corrected IS TRUE) OR (estimated_prep_duration IS NOT NULL AND estimated_prep_duration_corrected IS NOT NULL)', # noqa:E501
name=op.f('ck_orders_on_corrections_only_for_set_value_4'),
),
sa.CheckConstraint(
'(left_pickup_at IS NULL AND left_pickup_at_corrected IS NULL) OR (left_pickup_at IS NULL AND left_pickup_at_corrected IS TRUE) OR (left_pickup_at IS NOT NULL AND left_pickup_at_corrected IS NOT NULL)', # noqa:E501
name=op.f('ck_orders_on_corrections_only_for_set_value_9'),
),
sa.CheckConstraint(
'(pickup_at IS NULL AND pickup_at_corrected IS NULL) OR (pickup_at IS NULL AND pickup_at_corrected IS TRUE) OR (pickup_at IS NOT NULL AND pickup_at_corrected IS NOT NULL)', # noqa:E501
name=op.f('ck_orders_on_corrections_only_for_set_value_8'),
),
sa.CheckConstraint(
'(restaurant_confirmed_at IS NULL AND restaurant_confirmed_at_corrected IS NULL) OR (restaurant_confirmed_at IS NULL AND restaurant_confirmed_at_corrected IS TRUE) OR (restaurant_confirmed_at IS NOT NULL AND restaurant_confirmed_at_corrected IS NOT NULL)', # noqa:E501
name=op.f('ck_orders_on_corrections_only_for_set_value_3'),
),
sa.CheckConstraint(
'(restaurant_notified_at IS NULL AND restaurant_notified_at_corrected IS NULL) OR (restaurant_notified_at IS NULL AND restaurant_notified_at_corrected IS TRUE) OR (restaurant_notified_at IS NOT NULL AND restaurant_notified_at_corrected IS NOT NULL)', # noqa:E501
name=op.f('ck_orders_on_corrections_only_for_set_value_2'),
),
sa.CheckConstraint(
'(scheduled_delivery_at IS NULL AND scheduled_delivery_at_corrected IS NULL) OR (scheduled_delivery_at IS NULL AND scheduled_delivery_at_corrected IS TRUE) OR (scheduled_delivery_at IS NOT NULL AND scheduled_delivery_at_corrected IS NOT NULL)', # noqa:E501
name=op.f('ck_orders_on_corrections_only_for_set_value_0'),
),
sa.CheckConstraint(
'(ad_hoc IS TRUE AND scheduled_delivery_at IS NULL) OR (ad_hoc IS FALSE AND scheduled_delivery_at IS NOT NULL)', # noqa:E501
name=op.f('ck_orders_on_either_ad_hoc_or_scheduled_order'),
),
sa.CheckConstraint(
'NOT (EXTRACT(EPOCH FROM scheduled_delivery_at - placed_at) < 1800)',
name=op.f('ck_orders_on_scheduled_orders_not_within_30_minutes'),
),
sa.CheckConstraint(
'NOT (ad_hoc IS FALSE AND ((EXTRACT(HOUR FROM scheduled_delivery_at) <= 11 AND NOT (EXTRACT(HOUR FROM scheduled_delivery_at) = 11 AND EXTRACT(MINUTE FROM scheduled_delivery_at) = 45)) OR EXTRACT(HOUR FROM scheduled_delivery_at) > 22))', # noqa:E501
name=op.f('ck_orders_on_scheduled_orders_within_business_hours'),
),
sa.CheckConstraint(
'NOT (ad_hoc IS TRUE AND (EXTRACT(HOUR FROM placed_at) < 11 OR EXTRACT(HOUR FROM placed_at) > 22))', # noqa:E501
name=op.f('ck_orders_on_ad_hoc_orders_within_business_hours'),
),
sa.CheckConstraint(
'NOT (cancelled IS FALSE AND cancelled_at IS NOT NULL)',
name=op.f('ck_orders_on_only_cancelled_orders_may_have_cancelled_at'),
),
sa.CheckConstraint(
'NOT (cancelled IS TRUE AND delivery_at IS NOT NULL)',
name=op.f('ck_orders_on_cancelled_orders_must_not_be_delivered'),
),
sa.CheckConstraint(
'cancelled_at > courier_accepted_at',
name=op.f('ck_orders_on_ordered_timestamps_16'),
),
sa.CheckConstraint(
'cancelled_at > courier_notified_at',
name=op.f('ck_orders_on_ordered_timestamps_15'),
),
sa.CheckConstraint(
'cancelled_at > delivery_at',
name=op.f('ck_orders_on_ordered_timestamps_21'),
),
sa.CheckConstraint(
'cancelled_at > dispatch_at',
name=op.f('ck_orders_on_ordered_timestamps_14'),
),
sa.CheckConstraint(
'cancelled_at > left_pickup_at',
name=op.f('ck_orders_on_ordered_timestamps_19'),
),
sa.CheckConstraint(
'cancelled_at > pickup_at', name=op.f('ck_orders_on_ordered_timestamps_18'),
),
sa.CheckConstraint(
'cancelled_at > reached_delivery_at',
name=op.f('ck_orders_on_ordered_timestamps_20'),
),
sa.CheckConstraint(
'cancelled_at > reached_pickup_at',
name=op.f('ck_orders_on_ordered_timestamps_17'),
),
sa.CheckConstraint(
'cancelled_at > restaurant_confirmed_at',
name=op.f('ck_orders_on_ordered_timestamps_13'),
),
sa.CheckConstraint(
'cancelled_at > restaurant_notified_at',
name=op.f('ck_orders_on_ordered_timestamps_12'),
),
sa.CheckConstraint(
'courier_accepted_at < delivery_at',
name=op.f('ck_orders_on_ordered_timestamps_42'),
),
sa.CheckConstraint(
'courier_accepted_at < left_pickup_at',
name=op.f('ck_orders_on_ordered_timestamps_40'),
),
sa.CheckConstraint(
'courier_accepted_at < pickup_at',
name=op.f('ck_orders_on_ordered_timestamps_39'),
),
sa.CheckConstraint(
'courier_accepted_at < reached_delivery_at',
name=op.f('ck_orders_on_ordered_timestamps_41'),
),
sa.CheckConstraint(
'courier_accepted_at < reached_pickup_at',
name=op.f('ck_orders_on_ordered_timestamps_38'),
),
sa.CheckConstraint(
'courier_notified_at < courier_accepted_at',
name=op.f('ck_orders_on_ordered_timestamps_32'),
),
sa.CheckConstraint(
'courier_notified_at < delivery_at',
name=op.f('ck_orders_on_ordered_timestamps_37'),
),
sa.CheckConstraint(
'courier_notified_at < left_pickup_at',
name=op.f('ck_orders_on_ordered_timestamps_35'),
),
sa.CheckConstraint(
'courier_notified_at < pickup_at',
name=op.f('ck_orders_on_ordered_timestamps_34'),
),
sa.CheckConstraint(
'courier_notified_at < reached_delivery_at',
name=op.f('ck_orders_on_ordered_timestamps_36'),
),
sa.CheckConstraint(
'courier_notified_at < reached_pickup_at',
name=op.f('ck_orders_on_ordered_timestamps_33'),
),
sa.CheckConstraint(
'dispatch_at < courier_accepted_at',
name=op.f('ck_orders_on_ordered_timestamps_26'),
),
sa.CheckConstraint(
'dispatch_at < courier_notified_at',
name=op.f('ck_orders_on_ordered_timestamps_25'),
),
sa.CheckConstraint(
'dispatch_at < delivery_at',
name=op.f('ck_orders_on_ordered_timestamps_31'),
),
sa.CheckConstraint(
'dispatch_at < left_pickup_at',
name=op.f('ck_orders_on_ordered_timestamps_29'),
),
sa.CheckConstraint(
'dispatch_at < pickup_at', name=op.f('ck_orders_on_ordered_timestamps_28'),
),
sa.CheckConstraint(
'dispatch_at < reached_delivery_at',
name=op.f('ck_orders_on_ordered_timestamps_30'),
),
sa.CheckConstraint(
'dispatch_at < reached_pickup_at',
name=op.f('ck_orders_on_ordered_timestamps_27'),
),
sa.CheckConstraint(
'estimated_prep_buffer % 60 = 0',
name=op.f('ck_orders_on_estimated_prep_buffer_must_be_whole_minutes'),
),
sa.CheckConstraint(
'estimated_prep_duration % 60 = 0',
name=op.f('ck_orders_on_estimated_prep_duration_must_be_whole_minutes'),
),
sa.CheckConstraint(
'left_pickup_at < delivery_at',
name=op.f('ck_orders_on_ordered_timestamps_51'),
),
sa.CheckConstraint(
'left_pickup_at < reached_delivery_at',
name=op.f('ck_orders_on_ordered_timestamps_50'),
),
sa.CheckConstraint(
'pickup_at < delivery_at', name=op.f('ck_orders_on_ordered_timestamps_49'),
),
sa.CheckConstraint(
'pickup_at < left_pickup_at',
name=op.f('ck_orders_on_ordered_timestamps_47'),
),
sa.CheckConstraint(
'pickup_at < reached_delivery_at',
name=op.f('ck_orders_on_ordered_timestamps_48'),
),
sa.CheckConstraint(
'placed_at < cancelled_at', name=op.f('ck_orders_on_ordered_timestamps_2'),
),
sa.CheckConstraint(
'placed_at < courier_accepted_at',
name=op.f('ck_orders_on_ordered_timestamps_7'),
),
sa.CheckConstraint(
'placed_at < courier_notified_at',
name=op.f('ck_orders_on_ordered_timestamps_6'),
),
sa.CheckConstraint(
'placed_at < delivery_at', name=op.f('ck_orders_on_ordered_timestamps_11'),
),
sa.CheckConstraint(
'placed_at < dispatch_at', name=op.f('ck_orders_on_ordered_timestamps_5'),
),
sa.CheckConstraint(
'placed_at < first_estimated_delivery_at',
name=op.f('ck_orders_on_ordered_timestamps_1'),
),
sa.CheckConstraint(
'placed_at < left_pickup_at',
name=op.f('ck_orders_on_ordered_timestamps_9'),
),
sa.CheckConstraint(
'placed_at < reached_delivery_at',
name=op.f('ck_orders_on_ordered_timestamps_10'),
),
sa.CheckConstraint(
'placed_at < reached_pickup_at',
name=op.f('ck_orders_on_ordered_timestamps_8'),
),
sa.CheckConstraint(
'placed_at < restaurant_confirmed_at',
name=op.f('ck_orders_on_ordered_timestamps_4'),
),
sa.CheckConstraint(
'placed_at < restaurant_notified_at',
name=op.f('ck_orders_on_ordered_timestamps_3'),
),
sa.CheckConstraint(
'placed_at < scheduled_delivery_at',
name=op.f('ck_orders_on_ordered_timestamps_0'),
),
sa.CheckConstraint(
'reached_delivery_at < delivery_at',
name=op.f('ck_orders_on_ordered_timestamps_52'),
),
sa.CheckConstraint(
'reached_pickup_at < delivery_at',
name=op.f('ck_orders_on_ordered_timestamps_46'),
),
sa.CheckConstraint(
'reached_pickup_at < left_pickup_at',
name=op.f('ck_orders_on_ordered_timestamps_44'),
),
sa.CheckConstraint(
'reached_pickup_at < pickup_at',
name=op.f('ck_orders_on_ordered_timestamps_43'),
),
sa.CheckConstraint(
'reached_pickup_at < reached_delivery_at',
name=op.f('ck_orders_on_ordered_timestamps_45'),
),
sa.CheckConstraint(
'restaurant_confirmed_at < pickup_at',
name=op.f('ck_orders_on_ordered_timestamps_24'),
),
sa.CheckConstraint(
'restaurant_notified_at < pickup_at',
name=op.f('ck_orders_on_ordered_timestamps_23'),
),
sa.CheckConstraint(
'restaurant_notified_at < restaurant_confirmed_at',
name=op.f('ck_orders_on_ordered_timestamps_22'),
),
sa.CheckConstraint(
'(pickup_at IS NULL AND pickup_not_confirmed IS NULL) OR (pickup_at IS NOT NULL AND pickup_not_confirmed IS NOT NULL)', # noqa:E501
name=op.f('pickup_not_confirmed_only_if_pickup'),
),
sa.CheckConstraint(
'(delivery_at IS NULL AND delivery_not_confirmed IS NULL) OR (delivery_at IS NOT NULL AND delivery_not_confirmed IS NOT NULL)', # noqa:E501
name=op.f('delivery_not_confirmed_only_if_delivery'),
),
sa.CheckConstraint(
'(delivery_at IS NULL AND courier_waited_at_delivery IS NULL) OR (delivery_at IS NOT NULL AND courier_waited_at_delivery IS NOT NULL)', # noqa:E501
name=op.f('courier_waited_at_delivery_only_if_delivery'),
),
sa.ForeignKeyConstraint(
['courier_id'],
[f'{config.CLEAN_SCHEMA}.couriers.id'],
name=op.f('fk_orders_to_couriers_via_courier_id'),
onupdate='RESTRICT',
ondelete='RESTRICT',
),
sa.ForeignKeyConstraint(
['customer_id'],
[f'{config.CLEAN_SCHEMA}.customers.id'],
name=op.f('fk_orders_to_customers_via_customer_id'),
onupdate='RESTRICT',
ondelete='RESTRICT',
),
sa.ForeignKeyConstraint(
['delivery_address_id'],
[f'{config.CLEAN_SCHEMA}.addresses.id'],
name=op.f('fk_orders_to_addresses_via_delivery_address_id'),
onupdate='RESTRICT',
ondelete='RESTRICT',
),
sa.ForeignKeyConstraint(
['pickup_address_id'],
[f'{config.CLEAN_SCHEMA}.addresses.id'],
name=op.f('fk_orders_to_addresses_via_pickup_address_id'),
onupdate='RESTRICT',
ondelete='RESTRICT',
),
sa.ForeignKeyConstraint(
['restaurant_id'],
[f'{config.CLEAN_SCHEMA}.restaurants.id'],
name=op.f('fk_orders_to_restaurants_via_restaurant_id'),
onupdate='RESTRICT',
ondelete='RESTRICT',
),
sa.PrimaryKeyConstraint('id', name=op.f('pk_orders')),
*(
[ # noqa:WPS504
sa.ForeignKeyConstraint(
['id'],
[f'{config.ORIGINAL_SCHEMA}.orders.id'],
name=op.f('pk_orders_sanity'),
onupdate='RESTRICT',
ondelete='RESTRICT',
),
sa.ForeignKeyConstraint(
['delivery_id'],
[f'{config.ORIGINAL_SCHEMA}.deliveries.id'],
name=op.f('pk_deliveries_sanity'),
onupdate='RESTRICT',
ondelete='RESTRICT',
),
]
if not config.TESTING
else []
),
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_orders_on_cancelled'),
'orders',
['cancelled'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_orders_on_cancelled_at_corrected'),
'orders',
['cancelled_at_corrected'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_orders_on_courier_accepted_at_corrected'),
'orders',
['courier_accepted_at_corrected'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_orders_on_courier_id'),
'orders',
['courier_id'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_orders_on_courier_notified_at_corrected'),
'orders',
['courier_notified_at_corrected'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_orders_on_customer_id'),
'orders',
['customer_id'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_orders_on_delivery_address_id'),
'orders',
['delivery_address_id'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_orders_on_delivery_at_corrected'),
'orders',
['delivery_at_corrected'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_orders_on_delivery_id'),
'orders',
['delivery_id'],
unique=True,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_orders_on_dispatch_at_corrected'),
'orders',
['dispatch_at_corrected'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_orders_on_estimated_prep_buffer'),
'orders',
['estimated_prep_buffer'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_orders_on_estimated_prep_duration'),
'orders',
['estimated_prep_duration'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_orders_on_estimated_prep_duration_corrected'),
'orders',
['estimated_prep_duration_corrected'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_orders_on_left_pickup_at_corrected'),
'orders',
['left_pickup_at_corrected'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_orders_on_pickup_address_id'),
'orders',
['pickup_address_id'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_orders_on_pickup_at_corrected'),
'orders',
['pickup_at_corrected'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_orders_on_placed_at'),
'orders',
['placed_at'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_orders_on_restaurant_confirmed_at_corrected'),
'orders',
['restaurant_confirmed_at_corrected'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_orders_on_restaurant_id'),
'orders',
['restaurant_id'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_orders_on_restaurant_notified_at_corrected'),
'orders',
['restaurant_notified_at_corrected'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_orders_on_scheduled_delivery_at'),
'orders',
['scheduled_delivery_at'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_orders_on_scheduled_delivery_at_corrected'),
'orders',
['scheduled_delivery_at_corrected'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
def downgrade():
"""Downgrade to revision None."""
op.execute(f'DROP SCHEMA {config.CLEAN_SCHEMA} CASCADE;')

167
migrations/versions/rev_20210102_18_888e352d7526_add_pixel_grid.py
View file

@ -1,167 +0,0 @@
"""Add pixel grid.
Revision: #888e352d7526 at 2021-01-02 18:11:02
Revises: #f11cd76d2f45
"""
import os
import sqlalchemy as sa
from alembic import op
from urban_meal_delivery import configuration
revision = '888e352d7526'
down_revision = 'f11cd76d2f45'
branch_labels = None
depends_on = None
config = configuration.make_config('testing' if os.getenv('TESTING') else 'production')
def upgrade():
"""Upgrade to revision 888e352d7526."""
op.create_table(
'grids',
sa.Column('id', sa.SmallInteger(), autoincrement=True, nullable=False),
sa.Column('city_id', sa.SmallInteger(), nullable=False),
sa.Column('side_length', sa.SmallInteger(), nullable=True),
sa.PrimaryKeyConstraint('id', name=op.f('pk_grids')),
sa.ForeignKeyConstraint(
['city_id'],
[f'{config.CLEAN_SCHEMA}.cities.id'],
name=op.f('fk_grids_to_cities_via_city_id'),
onupdate='RESTRICT',
ondelete='RESTRICT',
),
sa.UniqueConstraint(
'city_id', 'side_length', name=op.f('uq_grids_on_city_id_side_length'),
),
# This `UniqueConstraint` is needed by the `addresses_pixels` table below.
sa.UniqueConstraint('id', 'city_id', name=op.f('uq_grids_on_id_city_id')),
schema=config.CLEAN_SCHEMA,
)
op.create_table(
'pixels',
sa.Column('id', sa.Integer(), autoincrement=True, nullable=False),
sa.Column('grid_id', sa.SmallInteger(), nullable=False),
sa.Column('n_x', sa.SmallInteger(), nullable=False),
sa.Column('n_y', sa.SmallInteger(), nullable=False),
sa.CheckConstraint('0 <= n_x', name=op.f('ck_pixels_on_n_x_is_positive')),
sa.CheckConstraint('0 <= n_y', name=op.f('ck_pixels_on_n_y_is_positive')),
sa.ForeignKeyConstraint(
['grid_id'],
[f'{config.CLEAN_SCHEMA}.grids.id'],
name=op.f('fk_pixels_to_grids_via_grid_id'),
onupdate='RESTRICT',
ondelete='RESTRICT',
),
sa.PrimaryKeyConstraint('id', name=op.f('pk_pixels')),
sa.UniqueConstraint(
'grid_id', 'n_x', 'n_y', name=op.f('uq_pixels_on_grid_id_n_x_n_y'),
),
sa.UniqueConstraint('id', 'grid_id', name=op.f('uq_pixels_on_id_grid_id')),
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_pixels_on_grid_id'),
'pixels',
['grid_id'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_pixels_on_n_x'),
'pixels',
['n_x'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_pixels_on_n_y'),
'pixels',
['n_y'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
# This `UniqueConstraint` is needed by the `addresses_pixels` table below.
op.create_unique_constraint(
'uq_addresses_on_id_city_id',
'addresses',
['id', 'city_id'],
schema=config.CLEAN_SCHEMA,
)
op.create_table(
'addresses_pixels',
sa.Column('address_id', sa.Integer(), nullable=False),
sa.Column('city_id', sa.SmallInteger(), nullable=False),
sa.Column('grid_id', sa.SmallInteger(), nullable=False),
sa.Column('pixel_id', sa.Integer(), nullable=False),
sa.ForeignKeyConstraint(
['address_id', 'city_id'],
[
f'{config.CLEAN_SCHEMA}.addresses.id',
f'{config.CLEAN_SCHEMA}.addresses.city_id',
],
name=op.f('fk_addresses_pixels_to_addresses_via_address_id_city_id'),
onupdate='RESTRICT',
ondelete='RESTRICT',
),
sa.ForeignKeyConstraint(
['grid_id', 'city_id'],
[
f'{config.CLEAN_SCHEMA}.grids.id',
f'{config.CLEAN_SCHEMA}.grids.city_id',
],
name=op.f('fk_addresses_pixels_to_grids_via_grid_id_city_id'),
onupdate='RESTRICT',
ondelete='RESTRICT',
),
sa.ForeignKeyConstraint(
['pixel_id', 'grid_id'],
[
f'{config.CLEAN_SCHEMA}.pixels.id',
f'{config.CLEAN_SCHEMA}.pixels.grid_id',
],
name=op.f('fk_addresses_pixels_to_pixels_via_pixel_id_grid_id'),
onupdate='RESTRICT',
ondelete='RESTRICT',
),
sa.PrimaryKeyConstraint(
'address_id', 'pixel_id', name=op.f('pk_addresses_pixels'),
),
sa.UniqueConstraint(
'address_id',
'grid_id',
name=op.f('uq_addresses_pixels_on_address_id_grid_id'),
),
schema=config.CLEAN_SCHEMA,
)
def downgrade():
"""Downgrade to revision f11cd76d2f45."""
op.drop_table('addresses_pixels', schema=config.CLEAN_SCHEMA)
op.drop_constraint(
'uq_addresses_on_id_city_id',
'addresses',
type_=None,
schema=config.CLEAN_SCHEMA,
)
op.drop_index(
op.f('ix_pixels_on_n_y'), table_name='pixels', schema=config.CLEAN_SCHEMA,
)
op.drop_index(
op.f('ix_pixels_on_n_x'), table_name='pixels', schema=config.CLEAN_SCHEMA,
)
op.drop_index(
op.f('ix_pixels_on_grid_id'), table_name='pixels', schema=config.CLEAN_SCHEMA,
)
op.drop_table('pixels', schema=config.CLEAN_SCHEMA)
op.drop_table('grids', schema=config.CLEAN_SCHEMA)

96
migrations/versions/rev_20210106_19_e40623e10405_add_demand_forecasting.py
View file

@ -1,96 +0,0 @@
"""Add demand forecasting.
Revision: #e40623e10405 at 2021-01-06 19:55:56
Revises: #888e352d7526
"""
import os
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
from urban_meal_delivery import configuration
revision = 'e40623e10405'
down_revision = '888e352d7526'
branch_labels = None
depends_on = None
config = configuration.make_config('testing' if os.getenv('TESTING') else 'production')
def upgrade():
"""Upgrade to revision e40623e10405."""
op.create_table(
'forecasts',
sa.Column('id', sa.Integer(), autoincrement=True, nullable=False),
sa.Column('pixel_id', sa.Integer(), nullable=False),
sa.Column('start_at', sa.DateTime(), nullable=False),
sa.Column('time_step', sa.SmallInteger(), nullable=False),
sa.Column('training_horizon', sa.SmallInteger(), nullable=False),
sa.Column('method', sa.Unicode(length=20), nullable=False),
sa.Column('prediction', postgresql.DOUBLE_PRECISION(), nullable=False),
sa.PrimaryKeyConstraint('id', name=op.f('pk_forecasts')),
sa.ForeignKeyConstraint(
['pixel_id'],
[f'{config.CLEAN_SCHEMA}.pixels.id'],
name=op.f('fk_forecasts_to_pixels_via_pixel_id'),
onupdate='RESTRICT',
ondelete='RESTRICT',
),
sa.CheckConstraint(
"""
NOT (
EXTRACT(HOUR FROM start_at) < 11
OR
EXTRACT(HOUR FROM start_at) > 22
)
""",
name=op.f('ck_forecasts_on_start_at_must_be_within_operating_hours'),
),
sa.CheckConstraint(
'CAST(EXTRACT(MINUTES FROM start_at) AS INTEGER) % 15 = 0',
name=op.f('ck_forecasts_on_start_at_minutes_must_be_quarters_of_the_hour'),
),
sa.CheckConstraint(
'CAST(EXTRACT(MICROSECONDS FROM start_at) AS INTEGER) % 1000000 = 0',
name=op.f('ck_forecasts_on_start_at_allows_no_microseconds'),
),
sa.CheckConstraint(
'EXTRACT(SECONDS FROM start_at) = 0',
name=op.f('ck_forecasts_on_start_at_allows_no_seconds'),
),
sa.CheckConstraint(
'time_step > 0', name=op.f('ck_forecasts_on_time_step_must_be_positive'),
),
sa.CheckConstraint(
'training_horizon > 0',
name=op.f('ck_forecasts_on_training_horizon_must_be_positive'),
),
sa.UniqueConstraint(
'pixel_id',
'start_at',
'time_step',
'training_horizon',
'method',
name=op.f(
'uq_forecasts_on_pixel_id_start_at_time_step_training_horizon_method',
),
),
schema=config.CLEAN_SCHEMA,
)
op.create_index(
op.f('ix_forecasts_on_pixel_id'),
'forecasts',
['pixel_id'],
unique=False,
schema=config.CLEAN_SCHEMA,
)
def downgrade():
"""Downgrade to revision 888e352d7526."""
op.drop_table('forecasts', schema=config.CLEAN_SCHEMA)

124
migrations/versions/rev_20210120_16_26711cd3f9b9_add_confidence_intervals_to_forecasts.py
View file

@ -1,124 +0,0 @@
"""Add confidence intervals to forecasts.
Revision: #26711cd3f9b9 at 2021-01-20 16:08:21
Revises: #e40623e10405
"""
import os
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
from urban_meal_delivery import configuration
revision = '26711cd3f9b9'
down_revision = 'e40623e10405'
branch_labels = None
depends_on = None
config = configuration.make_config('testing' if os.getenv('TESTING') else 'production')
def upgrade():
"""Upgrade to revision 26711cd3f9b9."""
op.alter_column(
'forecasts', 'method', new_column_name='model', schema=config.CLEAN_SCHEMA,
)
op.add_column(
'forecasts',
sa.Column('low80', postgresql.DOUBLE_PRECISION(), nullable=True),
schema=config.CLEAN_SCHEMA,
)
op.add_column(
'forecasts',
sa.Column('high80', postgresql.DOUBLE_PRECISION(), nullable=True),
schema=config.CLEAN_SCHEMA,
)
op.add_column(
'forecasts',
sa.Column('low95', postgresql.DOUBLE_PRECISION(), nullable=True),
schema=config.CLEAN_SCHEMA,
)
op.add_column(
'forecasts',
sa.Column('high95', postgresql.DOUBLE_PRECISION(), nullable=True),
schema=config.CLEAN_SCHEMA,
)
op.create_check_constraint(
op.f('ck_forecasts_on_ci_upper_and_lower_bounds'),
'forecasts',
"""
NOT (
low80 IS NULL AND high80 IS NOT NULL
OR
low80 IS NOT NULL AND high80 IS NULL
OR
low95 IS NULL AND high95 IS NOT NULL
OR
low95 IS NOT NULL AND high95 IS NULL
)
""",
schema=config.CLEAN_SCHEMA,
)
op.create_check_constraint(
op.f('prediction_must_be_within_ci'),
'forecasts',
"""
NOT (
prediction < low80
OR
prediction < low95
OR
prediction > high80
OR
prediction > high95
)
""",
schema=config.CLEAN_SCHEMA,
)
op.create_check_constraint(
op.f('ci_upper_bound_greater_than_lower_bound'),
'forecasts',
"""
NOT (
low80 > high80
OR
low95 > high95
)
""",
schema=config.CLEAN_SCHEMA,
)
op.create_check_constraint(
op.f('ci95_must_be_wider_than_ci80'),
'forecasts',
"""
NOT (
low80 < low95
OR
high80 > high95
)
""",
schema=config.CLEAN_SCHEMA,
)
def downgrade():
"""Downgrade to revision e40623e10405."""
op.alter_column(
'forecasts', 'model', new_column_name='method', schema=config.CLEAN_SCHEMA,
)
op.drop_column(
'forecasts', 'low80', schema=config.CLEAN_SCHEMA,
)
op.drop_column(
'forecasts', 'high80', schema=config.CLEAN_SCHEMA,
)
op.drop_column(
'forecasts', 'low95', schema=config.CLEAN_SCHEMA,
)
op.drop_column(
'forecasts', 'high95', schema=config.CLEAN_SCHEMA,
)

398
migrations/versions/rev_20210123_15_e86290e7305e_remove_orders_from_restaurants_with.py
View file

@ -1,398 +0,0 @@
"""Remove orders from restaurants with invalid location ...
... and also de-duplicate a couple of redundant addresses.
Revision: #e86290e7305e at 2021-01-23 15:56:59
Revises: #26711cd3f9b9
1) Remove orders
Some restaurants have orders to be picked up at an address that
not their primary address. That is ok if that address is the location
of a second franchise. However, for a small number of restaurants
there is only exactly one order at that other address that often is
located far away from the restaurant's primary location. It looks
like a restaurant signed up with some invalid location that was then
corrected into the primary one.
Use the following SQL statement to obtain a list of these locations
before this migration is run:
SELECT
orders.pickup_address_id,
COUNT(*) AS n_orders,
MIN(placed_at) as first_order_at,
MAX(placed_at) as last_order_at
FROM
{config.CLEAN_SCHEMA}.orders
LEFT OUTER JOIN
{config.CLEAN_SCHEMA}.restaurants
ON orders.restaurant_id = restaurants.id
WHERE
orders.pickup_address_id <> restaurants.address_id
GROUP BY
pickup_address_id;
50 orders with such weird pickup addresses are removed with this migration.
2) De-duplicate addresses
Five restaurants have two pickup addresses that are actually the same location.
The following SQL statement shows them before this migration is run:
SELECT
orders.restaurant_id,
restaurants.name,
restaurants.address_id AS primary_address_id,
addresses.id AS address_id,
addresses.street,
COUNT(*) AS n_orders
FROM
{config.CLEAN_SCHEMA}.orders
LEFT OUTER JOIN
{config.CLEAN_SCHEMA}.addresses ON orders.pickup_address_id = addresses.id
LEFT OUTER JOIN
{config.CLEAN_SCHEMA}.restaurants ON orders.restaurant_id = restaurants.id
WHERE
orders.restaurant_id IN (
SELECT
restaurant_id
FROM (
SELECT DISTINCT
restaurant_id,
pickup_address_id
FROM
{config.CLEAN_SCHEMA}.orders
) AS restaurant_locations
GROUP BY
restaurant_id
HAVING
COUNT(pickup_address_id) > 1
)
GROUP BY
orders.restaurant_id,
restaurants.name,
restaurants.address_id,
addresses.id,
addresses.street
ORDER BY
orders.restaurant_id,
restaurants.name,
restaurants.address_id,
addresses.id,
addresses.street;
3) Remove addresses without any association
After steps 1) and 2) some addresses are not associated with a restaurant any more.
The following SQL statement lists them before this migration is run:
SELECT
id,
street,
zip_code,
city
FROM
{config.CLEAN_SCHEMA}.addresses
WHERE
id NOT IN (
SELECT DISTINCT
pickup_address_id AS id
FROM
{config.CLEAN_SCHEMA}.orders
UNION
SELECT DISTINCT
delivery_address_id AS id
FROM
{config.CLEAN_SCHEMA}.orders
UNION
SELECT DISTINCT
address_id AS id
FROM
{config.CLEAN_SCHEMA}.restaurants
);
4) Ensure every `Restaurant` has exactly one `Address`.
Replace the current `ForeignKeyConstraint` to from `Order` to `Restaurant`
with one that also includes the `Order.pickup_address_id`.
"""
import os
from alembic import op
from urban_meal_delivery import configuration
revision = 'e86290e7305e'
down_revision = '26711cd3f9b9'
branch_labels = None
depends_on = None
config = configuration.make_config('testing' if os.getenv('TESTING') else 'production')
def upgrade():
"""Upgrade to revision e86290e7305e."""
# 1) Remove orders
op.execute(
f"""
DELETE
FROM
{config.CLEAN_SCHEMA}.orders
WHERE pickup_address_id IN (
SELECT
orders.pickup_address_id
FROM
{config.CLEAN_SCHEMA}.orders
LEFT OUTER JOIN
{config.CLEAN_SCHEMA}.restaurants
ON orders.restaurant_id = restaurants.id
WHERE
orders.pickup_address_id <> restaurants.address_id
GROUP BY
orders.pickup_address_id
HAVING
COUNT(*) = 1
);
""",
)
# 2) De-duplicate addresses
op.execute(
f"""
UPDATE
{config.CLEAN_SCHEMA}.orders
SET
pickup_address_id = 353
WHERE
pickup_address_id = 548916;
""",
)
op.execute(
f"""
UPDATE
{config.CLEAN_SCHEMA}.orders
SET
pickup_address_id = 4850
WHERE
pickup_address_id = 6415;
""",
)
op.execute(
f"""
UPDATE
{config.CLEAN_SCHEMA}.orders
SET
pickup_address_id = 16227
WHERE
pickup_address_id = 44627;
""",
)
op.execute(
f"""
UPDATE
{config.CLEAN_SCHEMA}.orders
SET
pickup_address_id = 44458
WHERE
pickup_address_id = 534543;
""",
)
op.execute(
f"""
UPDATE
{config.CLEAN_SCHEMA}.orders
SET
pickup_address_id = 289997
WHERE
pickup_address_id = 309525;
""",
)
# 3) Remove addresses
op.execute(
f"""
DELETE
FROM
{config.CLEAN_SCHEMA}.addresses_pixels
WHERE
address_id NOT IN (
SELECT DISTINCT
pickup_address_id AS id
FROM
{config.CLEAN_SCHEMA}.orders
UNION
SELECT DISTINCT
delivery_address_id AS id
FROM
{config.CLEAN_SCHEMA}.orders
UNION
SELECT DISTINCT
address_id AS id
FROM
{config.CLEAN_SCHEMA}.restaurants
);
""",
)
op.execute(
f"""
UPDATE
{config.CLEAN_SCHEMA}.addresses
SET
primary_id = 302883
WHERE
primary_id = 43526;
""",
)
op.execute(
f"""
UPDATE
{config.CLEAN_SCHEMA}.addresses
SET
primary_id = 47597
WHERE
primary_id = 43728;
""",
)
op.execute(
f"""
UPDATE
{config.CLEAN_SCHEMA}.addresses
SET
primary_id = 159631
WHERE
primary_id = 43942;
""",
)
op.execute(
f"""
UPDATE
{config.CLEAN_SCHEMA}.addresses
SET
primary_id = 275651
WHERE
primary_id = 44759;
""",
)
op.execute(
f"""
UPDATE
{config.CLEAN_SCHEMA}.addresses
SET
primary_id = 156685
WHERE
primary_id = 50599;
""",
)
op.execute(
f"""
UPDATE
{config.CLEAN_SCHEMA}.addresses
SET
primary_id = 480206
WHERE
primary_id = 51774;
""",
)
op.execute(
f"""
DELETE
FROM
{config.CLEAN_SCHEMA}.addresses
WHERE
id NOT IN (
SELECT DISTINCT
pickup_address_id AS id
FROM
{config.CLEAN_SCHEMA}.orders
UNION
SELECT DISTINCT
delivery_address_id AS id
FROM
{config.CLEAN_SCHEMA}.orders
UNION
SELECT DISTINCT
address_id AS id
FROM
{config.CLEAN_SCHEMA}.restaurants
);
""",
)
# 4) Ensure every `Restaurant` has only one `Order.pickup_address`.
op.execute(
f"""
UPDATE
{config.CLEAN_SCHEMA}.orders
SET
pickup_address_id = 53733
WHERE
pickup_address_id = 54892;
""",
)
op.execute(
f"""
DELETE
FROM
{config.CLEAN_SCHEMA}.addresses
WHERE
id = 54892;
""",
)
op.create_unique_constraint(
'uq_restaurants_on_id_address_id',
'restaurants',
['id', 'address_id'],
schema=config.CLEAN_SCHEMA,
)
op.create_foreign_key(
op.f('fk_orders_to_restaurants_via_restaurant_id_pickup_address_id'),
'orders',
'restaurants',
['restaurant_id', 'pickup_address_id'],
['id', 'address_id'],
source_schema=config.CLEAN_SCHEMA,
referent_schema=config.CLEAN_SCHEMA,
onupdate='RESTRICT',
ondelete='RESTRICT',
)
op.drop_constraint(
'fk_orders_to_restaurants_via_restaurant_id',
'orders',
type_='foreignkey',
schema=config.CLEAN_SCHEMA,
)
def downgrade():
"""Downgrade to revision 26711cd3f9b9."""
op.create_foreign_key(
op.f('fk_orders_to_restaurants_via_restaurant_id'),
'orders',
'restaurants',
['restaurant_id'],
['id'],
source_schema=config.CLEAN_SCHEMA,
referent_schema=config.CLEAN_SCHEMA,
onupdate='RESTRICT',
ondelete='RESTRICT',
)
op.drop_constraint(
'fk_orders_to_restaurants_via_restaurant_id_pickup_address_id',
'orders',
type_='foreignkey',
schema=config.CLEAN_SCHEMA,
)
op.drop_constraint(
'uq_restaurants_on_id_address_id',
'restaurants',
type_='unique',
schema=config.CLEAN_SCHEMA,
)

41
migrations/versions/rev_20210129_11_c2af85bada01_store_actuals_with_forecast.py
View file

@ -1,41 +0,0 @@
"""Store actuals with forecast.
Revision: #c2af85bada01 at 2021-01-29 11:13:15
Revises: #e86290e7305e
"""
import os
import sqlalchemy as sa
from alembic import op
from urban_meal_delivery import configuration
revision = 'c2af85bada01'
down_revision = 'e86290e7305e'
branch_labels = None
depends_on = None
config = configuration.make_config('testing' if os.getenv('TESTING') else 'production')
def upgrade():
"""Upgrade to revision c2af85bada01."""
op.add_column(
'forecasts',
sa.Column('actual', sa.SmallInteger(), nullable=False),
schema=config.CLEAN_SCHEMA,
)
op.create_check_constraint(
op.f('ck_forecasts_on_actuals_must_be_non_negative'),
'forecasts',
'actual >= 0',
schema=config.CLEAN_SCHEMA,
)
def downgrade():
"""Downgrade to revision e86290e7305e."""
op.drop_column('forecasts', 'actual', schema=config.CLEAN_SCHEMA)

48
migrations/versions/rev_20210202_12_8bfb928a31f8_rename_training_horizon_into_train_horizon.py
View file

@ -1,48 +0,0 @@
"""Rename `Forecast.training_horizon` into `.train_horizon`.
Revision: #8bfb928a31f8 at 2021-02-02 12:55:09
Revises: #c2af85bada01
"""
import os
from alembic import op
from urban_meal_delivery import configuration
revision = '8bfb928a31f8'
down_revision = 'c2af85bada01'
branch_labels = None
depends_on = None
config = configuration.make_config('testing' if os.getenv('TESTING') else 'production')
def upgrade():
"""Upgrade to revision 8bfb928a31f8."""
op.execute(
f"""
ALTER TABLE
{config.CLEAN_SCHEMA}.forecasts
RENAME COLUMN
training_horizon
TO
train_horizon;
""",
) # noqa:WPS355
def downgrade():
"""Downgrade to revision c2af85bada01."""
op.execute(
f"""
ALTER TABLE
{config.CLEAN_SCHEMA}.forecasts
RENAME COLUMN
train_horizon
TO
training_horizon;
""",
) # noqa:WPS355

96
migrations/versions/rev_20210301_16_b4dd0b8903a5_add_distance_matrix.py
View file

@ -1,96 +0,0 @@
"""Add distance matrix.
Revision: #b4dd0b8903a5 at 2021-03-01 16:14:06
Revises: #8bfb928a31f8
"""
import os
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
from urban_meal_delivery import configuration
revision = 'b4dd0b8903a5'
down_revision = '8bfb928a31f8'
branch_labels = None
depends_on = None
config = configuration.make_config('testing' if os.getenv('TESTING') else 'production')
def upgrade():
"""Upgrade to revision b4dd0b8903a5."""
op.create_table(
'addresses_addresses',
sa.Column('first_address_id', sa.Integer(), nullable=False),
sa.Column('second_address_id', sa.Integer(), nullable=False),
sa.Column('city_id', sa.SmallInteger(), nullable=False),
sa.Column('air_distance', sa.Integer(), nullable=False),
sa.Column('bicycle_distance', sa.Integer(), nullable=True),
sa.Column('bicycle_duration', sa.Integer(), nullable=True),
sa.Column('directions', postgresql.JSON(), nullable=True),
sa.PrimaryKeyConstraint(
'first_address_id',
'second_address_id',
name=op.f('pk_addresses_addresses'),
),
sa.ForeignKeyConstraint(
['first_address_id', 'city_id'],
[
f'{config.CLEAN_SCHEMA}.addresses.id',
f'{config.CLEAN_SCHEMA}.addresses.city_id',
],
name=op.f(
'fk_addresses_addresses_to_addresses_via_first_address_id_city_id',
),
onupdate='RESTRICT',
ondelete='RESTRICT',
),
sa.ForeignKeyConstraint(
['second_address_id', 'city_id'],
[
f'{config.CLEAN_SCHEMA}.addresses.id',
f'{config.CLEAN_SCHEMA}.addresses.city_id',
],
name=op.f(
'fk_addresses_addresses_to_addresses_via_second_address_id_city_id',
),
onupdate='RESTRICT',
ondelete='RESTRICT',
),
sa.UniqueConstraint(
'first_address_id',
'second_address_id',
name=op.f('uq_addresses_addresses_on_first_address_id_second_address_id'),
),
sa.CheckConstraint(
'first_address_id < second_address_id',
name=op.f('ck_addresses_addresses_on_distances_are_symmetric_for_bicycles'),
),
sa.CheckConstraint(
'0 <= air_distance AND air_distance < 20000',
name=op.f('ck_addresses_addresses_on_realistic_air_distance'),
),
sa.CheckConstraint(
'bicycle_distance < 25000',
name=op.f('ck_addresses_addresses_on_realistic_bicycle_distance'),
),
sa.CheckConstraint(
'air_distance <= bicycle_distance',
name=op.f('ck_addresses_addresses_on_air_distance_is_shortest'),
),
sa.CheckConstraint(
'0 <= bicycle_duration AND bicycle_duration <= 3600',
name=op.f('ck_addresses_addresses_on_realistic_bicycle_travel_time'),
),
schema=config.CLEAN_SCHEMA,
)
def downgrade():
"""Downgrade to revision 8bfb928a31f8."""
op.drop_table('addresses_addresses', schema=config.CLEAN_SCHEMA)

358
noxfile.py
View file

@ -17,7 +17,7 @@ as unified tasks to assure the quality of the source code:
that are then interpreted as the paths the formatters and linters work that are then interpreted as the paths the formatters and linters work
on recursively on recursively
- "lint" (flake8, mypy): same as "format" - "lint" (flake8, mypy, pylint): same as "format"
- "test" (pytest, xdoctest): - "test" (pytest, xdoctest):
@ -25,22 +25,41 @@ as unified tasks to assure the quality of the source code:
+ accepts extra arguments, e.g., `poetry run nox -s test -- --no-cov`, + accepts extra arguments, e.g., `poetry run nox -s test -- --no-cov`,
that are passed on to `pytest` and `xdoctest` with no changes that are passed on to `pytest` and `xdoctest` with no changes
=> may be paths or options => may be paths or options
GitHub Actions implements a CI workflow:
- "format", "lint", and "test" as above
- "safety": check if dependencies contain known security vulnerabilites
- "docs": build the documentation with sphinx
The pre-commit framework invokes the "pre-commit" and "pre-merge" sessions:
- "pre-commit" before all commits:
+ triggers "format" and "lint" on staged source files
+ => test coverage may be < 100%
- "pre-merge" before all merges and pushes:
+ same as "pre-commit"
+ plus: triggers "test", "safety", and "docs" (that ignore extra arguments)
+ => test coverage is enforced to be 100%
""" """
import contextlib import contextlib
import glob import glob
import os import os
import re
import shutil
import subprocess # noqa:S404
import tempfile import tempfile
from typing import Generator, IO, Tuple
import nox import nox
from nox.sessions import Session from nox.sessions import Session
GITHUB_REPOSITORY = 'webartifex/urban-meal-delivery'
PACKAGE_IMPORT_NAME = 'urban_meal_delivery' PACKAGE_IMPORT_NAME = 'urban_meal_delivery'
# Docs/sphinx locations. # Docs/sphinx locations.
@ -55,9 +74,7 @@ PYTEST_LOCATION = 'tests/'
# Paths with all *.py files. # Paths with all *.py files.
SRC_LOCATIONS = ( SRC_LOCATIONS = (
f'{DOCS_SRC}conf.py', f'{DOCS_SRC}/conf.py',
'migrations/env.py',
'migrations/versions/',
'noxfile.py', 'noxfile.py',
PACKAGE_SOURCE_LOCATION, PACKAGE_SOURCE_LOCATION,
PYTEST_LOCATION, PYTEST_LOCATION,
@ -72,7 +89,7 @@ nox.options.envdir = '.cache/nox'
# Avoid accidental successes if the environment is not set up properly. # Avoid accidental successes if the environment is not set up properly.
nox.options.error_on_external_run = True nox.options.error_on_external_run = True
# Run only local checks by default. # Run only CI related checks by default.
nox.options.sessions = ( nox.options.sessions = (
'format', 'format',
'lint', 'lint',
@ -121,7 +138,7 @@ def format_(session):
@nox.session(python=PYTHON) @nox.session(python=PYTHON)
def lint(session): def lint(session):
"""Lint source files with flake8 and mypy. """Lint source files with flake8, mypy, and pylint.
If no extra arguments are provided, all source files are linted. If no extra arguments are provided, all source files are linted.
Otherwise, they are interpreted as paths the linters work on recursively. Otherwise, they are interpreted as paths the linters work on recursively.
@ -135,8 +152,10 @@ def lint(session):
'flake8', 'flake8',
'flake8-annotations', 'flake8-annotations',
'flake8-black', 'flake8-black',
'flake8-expression-complexity',
'flake8-pytest-style', 'flake8-pytest-style',
'mypy', 'mypy',
'pylint',
'wemake-python-styleguide', 'wemake-python-styleguide',
) )
@ -160,6 +179,35 @@ def lint(session):
else: else:
session.log('No paths to be checked with mypy') session.log('No paths to be checked with mypy')
# Ignore errors where pylint cannot import a third-party package due its
# being run in an isolated environment. For the same reason, pylint is
# also not able to determine the correct order of imports.
# One way to fix this is to install all develop dependencies in this nox
# session, which we do not do. The whole point of static linting tools is
# to not rely on any package be importable at runtime. Instead, these
# imports are validated implicitly when the test suite is run.
session.run('pylint', '--version')
session.run(
'pylint', '--disable=import-error', '--disable=wrong-import-order', *locations,
)
@nox.session(name='pre-commit', python=PYTHON, venv_backend='none')
def pre_commit(session):
"""Run the format and lint sessions.
Source files must be well-formed before they enter git.
Intended to be run as a pre-commit hook.
Passed in extra arguments are forwarded. So, if it is run as a pre-commit
hook, only the currently staged source files are formatted and linted.
"""
# "format" and "lint" are run in sessions on their own as
# session.notify() creates new Session objects.
session.notify('format')
session.notify('lint')
@nox.session(python=PYTHON) @nox.session(python=PYTHON)
def test(session): def test(session):
@ -187,69 +235,27 @@ def test(session):
# non-develop dependencies be installed in the virtual environment. # non-develop dependencies be installed in the virtual environment.
session.run('poetry', 'install', '--no-dev', external=True) session.run('poetry', 'install', '--no-dev', external=True)
_install_packages( _install_packages(
session, session, 'packaging', 'pytest', 'pytest-cov', 'xdoctest[optional]',
'Faker',
'factory-boy',
'geopy',
'packaging',
'pytest',
'pytest-cov',
'pytest-env',
'pytest-mock',
'xdoctest[optional]',
) )
session.run('pytest', '--version')
# When the CI server runs the slow tests, we only execute the R related
# test cases that require the slow installation of R and some packages.
if session.env.get('_slow_ci_tests'):
session.run(
'pytest', '-m', 'r and not db', PYTEST_LOCATION,
)
# In the "ci-tests-slow" session, we do not run any test tool
# other than pytest. So, xdoctest, for example, is only run
# locally or in the "ci-tests-fast" session.
return
# When the CI server executes pytest, no database is available.
# Therefore, the CI server does not measure coverage.
elif session.env.get('_fast_ci_tests'):
pytest_args = (
'-m',
'not (db or r)',
PYTEST_LOCATION,
)
# When pytest is executed in the local develop environment,
# both R and a database are available.
# Therefore, we require 100% coverage.
else:
pytest_args = (
'--cov',
'--no-cov-on-fail',
'--cov-branch',
'--cov-fail-under=100',
'--cov-report=term-missing:skip-covered',
PYTEST_LOCATION,
)
# Interpret extra arguments as options for pytest. # Interpret extra arguments as options for pytest.
# They are "dropped" by the hack in the test_suite() function # They are "dropped" by the hack in the pre_merge() function
# if this function is run within the "test-suite" session. # if this function is run within the "pre-merge" session.
posargs = () if session.env.get('_drop_posargs') else session.posargs posargs = () if session.env.get('_drop_posargs') else session.posargs
session.run('pytest', *(posargs or pytest_args)) args = posargs or (
'--cov',
'--no-cov-on-fail',
'--cov-branch',
'--cov-fail-under=100',
'--cov-report=term-missing:skip-covered',
PYTEST_LOCATION,
)
session.run('pytest', '--version')
session.run('pytest', *args)
# For xdoctest, the default arguments are different from pytest. # For xdoctest, the default arguments are different from pytest.
args = posargs or [PACKAGE_IMPORT_NAME] args = posargs or [PACKAGE_IMPORT_NAME]
# The "TESTING" environment variable forces the global `engine`, `connection`,
# and `session` objects to be set to `None` and avoid any database connection.
# For pytest above this is not necessary as pytest sets this variable itself.
session.env['TESTING'] = 'true'
session.run('xdoctest', '--version') session.run('xdoctest', '--version')
session.run('xdoctest', '--quiet', *args) # --quiet => less verbose output session.run('xdoctest', '--quiet', *args) # --quiet => less verbose output
@ -293,10 +299,6 @@ def docs(session):
session.run('poetry', 'install', '--no-dev', external=True) session.run('poetry', 'install', '--no-dev', external=True)
_install_packages(session, 'sphinx', 'sphinx-autodoc-typehints') _install_packages(session, 'sphinx', 'sphinx-autodoc-typehints')
# The "TESTING" environment variable forces the global `engine`, `connection`,
# and `session` objects to be set to `None` and avoid any database connection.
session.env['TESTING'] = 'true'
session.run('sphinx-build', DOCS_SRC, DOCS_BUILD) session.run('sphinx-build', DOCS_SRC, DOCS_BUILD)
# Verify all external links return 200 OK. # Verify all external links return 200 OK.
session.run('sphinx-build', '-b', 'linkcheck', DOCS_SRC, DOCS_BUILD) session.run('sphinx-build', '-b', 'linkcheck', DOCS_SRC, DOCS_BUILD)
@ -304,74 +306,28 @@ def docs(session):
print(f'Docs are available at {os.getcwd()}/{DOCS_BUILD}index.html') # noqa:WPS421 print(f'Docs are available at {os.getcwd()}/{DOCS_BUILD}index.html') # noqa:WPS421
@nox.session(name='ci-tests-fast', python=PYTHON) @nox.session(name='pre-merge', python=PYTHON)
def fast_ci_tests(session): def pre_merge(session):
"""Fast tests run by the GitHub Actions CI server. """Run the format, lint, test, safety, and docs sessions.
These regards all test cases NOT involving R via `rpy2`. Intended to be run either as a pre-merge or pre-push hook.
Also, coverage is not measured as full coverage can only be
achieved by running the tests in the local develop environment
that has access to a database.
"""
# Re-using an old environment is not so easy here as the "test" session
# runs `poetry install --no-dev`, which removes previously installed packages.
if session.virtualenv.reuse_existing:
raise RuntimeError(
'The "ci-tests-fast" session must be run without the "-r" option',
)
# Little hack to pass arguments to the "test" session.
session.env['_fast_ci_tests'] = 'true'
# Cannot use session.notify() to trigger the "test" session
# as that would create a new Session object without the flag
# in the env(ironment).
test(session)
@nox.session(name='ci-tests-slow', python=PYTHON)
def slow_ci_tests(session):
"""Slow tests run by the GitHub Actions CI server.
These regards all test cases involving R via `rpy2`.
They are slow as the CI server needs to install R and some packages
first, which takes a couple of minutes.
Also, coverage is not measured as full coverage can only be
achieved by running the tests in the local develop environment
that has access to a database.
"""
# Re-using an old environment is not so easy here as the "test" session
# runs `poetry install --no-dev`, which removes previously installed packages.
if session.virtualenv.reuse_existing:
raise RuntimeError(
'The "ci-tests-slow" session must be run without the "-r" option',
)
# Little hack to pass arguments to the "test" session.
session.env['_slow_ci_tests'] = 'true'
# Cannot use session.notify() to trigger the "test" session
# as that would create a new Session object without the flag
# in the env(ironment).
test(session)
@nox.session(name='test-suite', python=PYTHON)
def test_suite(session):
"""Run the entire test suite as a pre-commit hook.
Ignores the paths passed in by the pre-commit framework Ignores the paths passed in by the pre-commit framework
and runs the entire test suite. for the test, safety, and docs sessions so that the
entire test suite is executed.
""" """
# Re-using an old environment is not so easy here as the "test" session # Re-using an old environment is not so easy here as the "test" session
# runs `poetry install --no-dev`, which removes previously installed packages. # runs `poetry install --no-dev`, which removes previously installed packages.
if session.virtualenv.reuse_existing: if session.virtualenv.reuse_existing:
raise RuntimeError( raise RuntimeError(
'The "test-suite" session must be run without the "-r" option', 'The "pre-merge" session must be run without the "-r" option',
) )
session.notify('format')
session.notify('lint')
session.notify('safety')
session.notify('docs')
# Little hack to not work with the extra arguments provided # Little hack to not work with the extra arguments provided
# by the pre-commit framework. Create a flag in the # by the pre-commit framework. Create a flag in the
# env(ironment) that must contain only `str`-like objects. # env(ironment) that must contain only `str`-like objects.
@ -379,130 +335,11 @@ def test_suite(session):
# Cannot use session.notify() to trigger the "test" session # Cannot use session.notify() to trigger the "test" session
# as that would create a new Session object without the flag # as that would create a new Session object without the flag
# in the env(ironment). # in the env(ironment). Instead, run the test() function within
# the "pre-merge" session.
test(session) test(session)
@nox.session(name='fix-branch-references', python=PYTHON, venv_backend='none')
def fix_branch_references(session): # noqa:WPS210,WPS231
"""Replace branch references with the current branch.
Intended to be run as a pre-commit hook.
Many files in the project (e.g., README.md) contain links to resources
on github.com or nbviewer.jupyter.org that contain branch labels.
This task rewrites these links such that they contain branch references
that make sense given the context:
- If the branch is only a temporary one that is to be merged into
the 'main' branch, all references are adjusted to 'main' as well.
- If the branch is not named after a default branch in the GitFlow
model, it is interpreted as a feature branch and the references
are adjusted into 'develop'.
This task may be called with one positional argument that is interpreted
as the branch to which all references are changed into.
The format must be "--branch=BRANCH_NAME".
"""
# Adjust this to add/remove glob patterns
# whose links are re-written.
paths = ['*.md', '**/*.md', '**/*.ipynb']
# Get the branch git is currently on.
# This is the branch to which all references are changed into
# if none of the two exceptions below apply.
branch = (
subprocess.check_output( # noqa:S603
('git', 'rev-parse', '--abbrev-ref', 'HEAD'),
)
.decode()
.strip()
)
# If the current branch is only a temporary one that is to be merged
# into 'main', we adjust all branch references to 'main' as well.
if branch.startswith('release') or branch.startswith('research'):
branch = 'main'
# If the current branch appears to be a feature branch, we adjust
# all branch references to 'develop'.
elif branch != 'main':
branch = 'develop'
# If a "--branch=BRANCH_NAME" argument is passed in
# as the only positional argument, we use BRANCH_NAME.
# Note: The --branch is required as session.posargs contains
# the staged files passed in by pre-commit in most cases.
if session.posargs and len(session.posargs) == 1:
match = re.match(
pattern=r'^--branch=([\w\.-]+)$', string=session.posargs[0].strip(),
)
if match:
branch = match.groups()[0]
rewrites = [
{
'name': 'github',
'pattern': re.compile(
fr'((((http)|(https))://github\.com/{GITHUB_REPOSITORY}/((blob)|(tree))/)([\w\.-]+)/)', # noqa:E501
),
'replacement': fr'\2{branch}/',
},
{
'name': 'nbviewer',
'pattern': re.compile(
fr'((((http)|(https))://nbviewer\.jupyter\.org/github/{GITHUB_REPOSITORY}/((blob)|(tree))/)([\w\.-]+)/)', # noqa:E501
),
'replacement': fr'\2{branch}/',
},
]
for expanded in _expand(*paths):
with _line_by_line_replace(expanded) as (old_file, new_file):
for line in old_file:
for rewrite in rewrites:
line = re.sub(rewrite['pattern'], rewrite['replacement'], line)
new_file.write(line)
def _expand(*patterns: str) -> Generator[str, None, None]:
"""Expand glob patterns into paths.
Args:
*patterns: the patterns to be expanded
Yields:
expanded: a single expanded path
""" # noqa:RST213
for pattern in patterns:
yield from glob.glob(pattern.strip())
@contextlib.contextmanager
def _line_by_line_replace(path: str) -> Generator[Tuple[IO, IO], None, None]:
"""Replace/change the lines in a file one by one.
This generator function yields two file handles, one to the current file
(i.e., `old_file`) and one to its replacement (i.e., `new_file`).
Usage: loop over the lines in `old_file` and write the files to be kept
to `new_file`. Files not written to `new_file` are removed!
Args:
path: the file whose lines are to be replaced
Yields:
old_file, new_file: handles to a file and its replacement
"""
file_handle, new_file_path = tempfile.mkstemp()
with os.fdopen(file_handle, 'w') as new_file:
with open(path) as old_file:
yield old_file, new_file
shutil.copymode(path, new_file_path)
os.remove(path)
shutil.move(new_file_path, path)
@nox.session(name='init-project', python=PYTHON, venv_backend='none') @nox.session(name='init-project', python=PYTHON, venv_backend='none')
def init_project(session): def init_project(session):
"""Install the pre-commit hooks.""" """Install the pre-commit hooks."""
@ -511,27 +348,25 @@ def init_project(session):
@nox.session(name='clean-pwd', python=PYTHON, venv_backend='none') @nox.session(name='clean-pwd', python=PYTHON, venv_backend='none')
def clean_pwd(session): # noqa:WPS231 def clean_pwd(session):
"""Remove (almost) all glob patterns listed in .gitignore. """Remove (almost) all glob patterns listed in .gitignore.
The difference compared to `git clean -X` is that this task The difference compared to `git clean -X` is that this task
does not remove pyenv's .python-version file and poetry's does not remove pyenv's .python-version file and poetry's
virtual environment. virtual environment.
""" """
exclude = frozenset(('.env', '.python-version', '.venv/', 'venv/')) exclude = frozenset(('.python-version', '.venv', 'venv'))
with open('.gitignore') as file_handle: with open('.gitignore') as file_handle:
paths = file_handle.readlines() paths = file_handle.readlines()
for path in _expand(*paths): for path in paths:
if path.startswith('#'): path = path.strip()
if path.startswith('#') or path in exclude:
continue continue
for excluded in exclude: for expanded in glob.glob(path):
if path.startswith(excluded): session.run(f'rm -rf {expanded}')
break
else:
session.run('rm', '-rf', path)
def _begin(session): def _begin(session):
@ -585,7 +420,6 @@ def _install_packages(session: Session, *packages_or_pip_args: str, **kwargs) ->
'--dev', '--dev',
'--format=requirements.txt', '--format=requirements.txt',
f'--output={requirements_txt.name}', f'--output={requirements_txt.name}',
'--without-hashes',
external=True, external=True,
) )
session.install( session.install(
@ -594,11 +428,11 @@ def _install_packages(session: Session, *packages_or_pip_args: str, **kwargs) ->
# TODO (isort): Remove this fix after # TODO (isort): Remove this fix after
# upgrading to isort ^5.5.4 in pyproject.toml. # upgrading to isort ^5.3.0 in pyproject.toml.
@contextlib.contextmanager @contextlib.contextmanager
def _isort_fix(session): def _isort_fix(session):
"""Temporarily upgrade to isort 5.5.4.""" """Temporarily upgrade to isort 5.3.0."""
session.install('isort==5.5.4') session.install('isort==5.3.0')
try: try:
yield yield
finally: finally:

3319
poetry.lock generated
View file

File diff suppressed because it is too large Load diff

42
pyproject.toml
View file

@ -9,7 +9,7 @@ target-version = ["py38"]
[tool.poetry] [tool.poetry]
name = "urban-meal-delivery" name = "urban-meal-delivery"
version = "0.4.0" version = "0.1.0"
authors = ["Alexander Hess <alexander@webartifex.biz>"] authors = ["Alexander Hess <alexander@webartifex.biz>"]
description = "Optimizing an urban meal delivery platform" description = "Optimizing an urban meal delivery platform"
@ -27,36 +27,7 @@ repository = "https://github.com/webartifex/urban-meal-delivery"
[tool.poetry.dependencies] [tool.poetry.dependencies]
python = "^3.8" python = "^3.8"
# Package => code developed in *.py files and packaged under src/urban_meal_delivery
Shapely = "^1.7.1"
alembic = "^1.4.2"
click = "^7.1.2" click = "^7.1.2"
folium = "^0.12.1"
geopy = "^2.1.0"
googlemaps = "^4.4.2"
matplotlib = "^3.3.3"
ordered-set = "^4.0.2"
pandas = "^1.1.0"
psycopg2 = "^2.8.5" # adapter for PostgreSQL
rpy2 = "^3.4.1"
sqlalchemy = "^1.3.18"
statsmodels = "^0.12.1"
utm = "^0.7.0"
# Jupyter Lab => notebooks with analyses using the developed package
# IMPORTANT: must be kept in sync with the "research" extra below
jupyterlab = { version="^2.2.2", optional=true }
nb_black = { version="^1.0.7", optional=true }
numpy = { version="^1.19.1", optional=true }
pytz = { version="^2020.1", optional=true }
[tool.poetry.extras]
research = [
"jupyterlab",
"nb_black",
"numpy",
"pytz",
]
[tool.poetry.dev-dependencies] [tool.poetry.dev-dependencies]
# Task Runners # Task Runners
@ -66,25 +37,22 @@ pre-commit = "^2.6.0"
# Code Formatters # Code Formatters
autoflake = "^1.3.1" autoflake = "^1.3.1"
black = "^19.10b0" black = "^19.10b0"
isort = "^4.3.21" # TODO (isort): not ^5.5.4 due to wemake-python-styleguide isort = "^4.3.21" # TODO (isort): not ^5.2.2 due to pylint and wemake-python-styleguide
# (Static) Code Analyzers # (Static) Code Analyzers
flake8 = "^3.8.3" flake8 = "^3.8.3"
flake8-annotations = "^2.3.0" flake8-annotations = "^2.3.0"
flake8-black = "^0.2.1" flake8-black = "^0.2.1"
flake8-expression-complexity = "^0.0.8"
flake8-pytest-style = "^1.2.2" flake8-pytest-style = "^1.2.2"
mypy = "^0.782" mypy = "^0.782"
pylint = "^2.5.3"
wemake-python-styleguide = "^0.14.1" # flake8 plug-in wemake-python-styleguide = "^0.14.1" # flake8 plug-in
# Test Suite # Test Suite
Faker = "^5.0.1"
factory-boy = "^3.1.0"
geopy = "^2.1.0"
packaging = "^20.4" # used to test the packaged version packaging = "^20.4" # used to test the packaged version
pytest = "^6.0.1" pytest = "^6.0.1"
pytest-cov = "^2.10.0" pytest-cov = "^2.10.0"
pytest-env = "^0.6.2"
pytest-mock = "^3.5.1"
xdoctest = { version="^0.13.0", extras=["optional"] } xdoctest = { version="^0.13.0", extras=["optional"] }
# Documentation # Documentation
@ -92,4 +60,4 @@ sphinx = "^3.1.2"
sphinx-autodoc-typehints = "^1.11.0" sphinx-autodoc-typehints = "^1.11.0"
[tool.poetry.scripts] [tool.poetry.scripts]
umd = "urban_meal_delivery.console:cli" umd = "urban_meal_delivery.console:main"

1868
research/00_r_dependencies.ipynb
View file

File diff suppressed because it is too large Load diff

7655
research/01_clean_data.ipynb
View file

File diff suppressed because it is too large Load diff

168
research/02_gridification.ipynb
View file

@ -1,168 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Gridification"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This notebook runs the gridification script and creates all the pixels in the database."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[32murban-meal-delivery\u001b[0m, version \u001b[34m0.3.0\u001b[0m\n"
]
}
],
"source": [
"!umd --version"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Upgrade Database Schema"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This database migration also de-duplicates redundant addresses and removes obvious outliers."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"%cd -q .."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"INFO [alembic.runtime.migration] Context impl PostgresqlImpl.\n",
"INFO [alembic.runtime.migration] Will assume transactional DDL.\n",
"INFO [alembic.runtime.migration] Running upgrade f11cd76d2f45 -> 888e352d7526, Add pixel grid.\n",
"INFO [alembic.runtime.migration] Running upgrade 888e352d7526 -> e40623e10405, Add demand forecasting.\n",
"INFO [alembic.runtime.migration] Running upgrade e40623e10405 -> 26711cd3f9b9, Add confidence intervals to forecasts.\n",
"INFO [alembic.runtime.migration] Running upgrade 26711cd3f9b9 -> e86290e7305e, Remove orders from restaurants with invalid location ...\n"
]
}
],
"source": [
"!alembic upgrade e86290e7305e"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Create the Grids"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Put all restaurant locations in pixels."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"3 cities retrieved from the database\n",
"\n",
"Creating grids for Lyon\n",
"Creating grid with a side length of 707 meters\n",
" -> created 62 pixels\n",
"Creating grid with a side length of 1000 meters\n",
" -> created 38 pixels\n",
"Creating grid with a side length of 1414 meters\n",
" -> created 24 pixels\n",
"=> assigned 358 out of 48058 addresses in Lyon\n",
"\n",
"Creating grids for Paris\n",
"Creating grid with a side length of 707 meters\n",
" -> created 199 pixels\n",
"Creating grid with a side length of 1000 meters\n",
" -> created 111 pixels\n",
"Creating grid with a side length of 1414 meters\n",
" -> created 66 pixels\n",
"=> assigned 1133 out of 108135 addresses in Paris\n",
"\n",
"Creating grids for Bordeaux\n",
"Creating grid with a side length of 707 meters\n",
" -> created 30 pixels\n",
"Creating grid with a side length of 1000 meters\n",
" -> created 22 pixels\n",
"Creating grid with a side length of 1414 meters\n",
" -> created 15 pixels\n",
"=> assigned 123 out of 21742 addresses in Bordeaux\n"
]
}
],
"source": [
"!umd gridify"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"%cd -q research"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.6"
}
},
"nbformat": 4,
"nbformat_minor": 4
}

356
research/03_grid_visualizations.ipynb
View file

File diff suppressed because one or more lines are too long

227
research/04_visualizing_restaurants.ipynb
View file

File diff suppressed because one or more lines are too long

230
research/05_visualizing_customers.ipynb
View file

File diff suppressed because one or more lines are too long

1531
research/06_tactical_demand_forecasting.ipynb
View file

File diff suppressed because it is too large Load diff

1940
research/07_visualizing_demand_forecasting.ipynb
View file

File diff suppressed because one or more lines are too long

922
research/08_google_maps.ipynb
View file

File diff suppressed because one or more lines are too long

1
research/papers/demand-forecasting

@ -1 +0,0 @@
Subproject commit 9ee3396a24ce20c9886b4cde5cfe2665fd5a8102

82173
research/visualizations/addresses_in_bordeaux_by_zip_code.html
View file

File diff suppressed because it is too large Load diff

108381
research/visualizations/addresses_in_lyon_by_zip_code.html
View file

File diff suppressed because it is too large Load diff

332859
research/visualizations/addresses_in_paris_by_zip_code.html
View file

File diff suppressed because it is too large Load diff

3357
research/visualizations/restaurants_in_bordeaux.html
View file

File diff suppressed because it is too large Load diff

3951
research/visualizations/restaurants_in_bordeaux_on_a_grid_with_side_length_1000.html
View file

File diff suppressed because it is too large Load diff

9243
research/visualizations/restaurants_in_lyon.html
View file

File diff suppressed because it is too large Load diff

10269
research/visualizations/restaurants_in_lyon_on_a_grid_with_side_length_1000.html
View file

File diff suppressed because it is too large Load diff

30195
research/visualizations/restaurants_in_paris.html
View file

File diff suppressed because it is too large Load diff

33138
research/visualizations/restaurants_in_paris_on_a_grid_with_side_length_1000.html
View file

File diff suppressed because it is too large Load diff

178
setup.cfg
View file

@ -72,6 +72,8 @@ select =
ANN0, ANN2, ANN3, ANN0, ANN2, ANN3,
# flake8-black => complain if black would make changes # flake8-black => complain if black would make changes
BLK1, BLK9, BLK1, BLK9,
# flake8-expression-complexity => not too many expressions at once
ECE001,
# flake8-pytest-style => enforce a consistent style with pytest # flake8-pytest-style => enforce a consistent style with pytest
PT0, PT0,
@ -82,55 +84,22 @@ ignore =
# If --ignore is passed on the command # If --ignore is passed on the command
# line, still ignore the following: # line, still ignore the following:
extend-ignore = extend-ignore =
# Too long line => duplicate with E501.
B950,
# Comply with black's style. # Comply with black's style.
# Source: https://github.com/psf/black/blob/master/docs/compatible_configs.md#flake8 # Source: https://github.com/psf/black/blob/master/docs/compatible_configs.md#flake8
E203, W503, WPS348, E203, W503,
# Let's not do `@pytest.mark.no_cover()` instead of `@pytest.mark.no_cover`.
PT023,
# Google's Python Style Guide is not reStructuredText
# until after being processed by Sphinx Napoleon.
# Source: https://github.com/peterjc/flake8-rst-docstrings/issues/17
RST201,RST203,RST210,RST213,RST301,
# String constant over-use is checked visually by the programmer.
WPS226,
# Allow underscores in numbers.
WPS303,
# f-strings are ok. # f-strings are ok.
WPS305, WPS305,
# Classes should not have to specify a base class. # Classes should not have to specify a base class.
WPS306, WPS306,
# Let's be modern: The Walrus is ok.
WPS332,
# Let's not worry about the number of noqa's.
WPS402,
# Putting logic into __init__.py files may be justified. # Putting logic into __init__.py files may be justified.
WPS412, WPS412,
# Allow multiple assignment, e.g., x = y = 123 # Allow multiple assignment, e.g., x = y = 123
WPS429, WPS429,
# There are no magic numbers.
WPS432,
per-file-ignores = per-file-ignores =
# Top-levels of a sub-packages are intended to import a lot.
**/__init__.py:
F401,WPS201,
docs/conf.py: docs/conf.py:
# Allow shadowing built-ins and reading __*__ variables. # Allow shadowing built-ins and reading __*__ variables.
WPS125,WPS609, WPS125,WPS609,
migrations/env.py:
# Type annotations are not strictly enforced.
ANN0, ANN2,
migrations/versions/*.py:
# Type annotations are not strictly enforced.
ANN0, ANN2,
# Do not worry about SQL injection here.
S608,
# File names of revisions are ok.
WPS114,WPS118,
# Revisions may have too many expressions.
WPS204,WPS213,
noxfile.py: noxfile.py:
# Type annotations are not strictly enforced. # Type annotations are not strictly enforced.
ANN0, ANN2, ANN0, ANN2,
@ -138,73 +107,23 @@ per-file-ignores =
WPS202, WPS202,
# TODO (isort): Remove after simplifying the nox session "lint". # TODO (isort): Remove after simplifying the nox session "lint".
WPS213, WPS213,
src/urban_meal_delivery/configuration.py: # No overuse of string constants (e.g., '--version').
# Allow upper case class variables within classes. WPS226,
WPS115,
src/urban_meal_delivery/console/forecasts.py:
# The module is not too complex.
WPS232,
src/urban_meal_delivery/db/addresses_addresses.py:
# The module does not have too many imports.
WPS201,
src/urban_meal_delivery/db/customers.py:
# The module is not too complex.
WPS232,
src/urban_meal_delivery/db/restaurants.py:
# The module is not too complex.
WPS232,
src/urban_meal_delivery/forecasts/methods/decomposition.py:
# The module is not too complex.
WPS232,
src/urban_meal_delivery/forecasts/methods/extrapolate_season.py:
# The module is not too complex.
WPS232,
src/urban_meal_delivery/forecasts/models/tactical/horizontal.py:
# The many noqa's are ok.
WPS403,
src/urban_meal_delivery/forecasts/timify.py:
# No SQL injection as the inputs come from a safe source.
S608,
# The many noqa's are ok.
WPS403,
tests/*.py: tests/*.py:
# Type annotations are not strictly enforced. # Type annotations are not strictly enforced.
ANN0, ANN2, ANN0, ANN2,
# The `Meta` class inside the factory_boy models do not need a docstring.
D106,
# `assert` statements are ok in the test suite. # `assert` statements are ok in the test suite.
S101, S101,
# The `random` module is not used for cryptography.
S311,
# Shadowing outer scopes occurs naturally with mocks. # Shadowing outer scopes occurs naturally with mocks.
WPS442, WPS442,
# Test names may be longer than 40 characters. # No overuse of string constants (e.g., '__version__').
WPS118, WPS226,
# Modules may have many test cases.
WPS202,WPS204,WPS214,
# Do not check for Jones complexity in the test suite.
WPS221,
# "Private" methods are really just a convention for
# fixtures without a return value.
WPS338,
# We do not care about the number of "# noqa"s in the test suite.
WPS402,
# Allow closures.
WPS430,
# When testing, it is normal to use implementation details.
WPS437,
# Explicitly set mccabe's maximum complexity to 10 as recommended by # Explicitly set mccabe's maximum complexity to 10 as recommended by
# Thomas McCabe, the inventor of the McCabe complexity, and the NIST. # Thomas McCabe, the inventor of the McCabe complexity, and the NIST.
# Source: https://en.wikipedia.org/wiki/Cyclomatic_complexity#Limiting_complexity_during_development # Source: https://en.wikipedia.org/wiki/Cyclomatic_complexity#Limiting_complexity_during_development
max-complexity = 10 max-complexity = 10
# Allow more than wemake-python-styleguide's 5 local variables per function.
max-local-variables = 8
# Allow more than wemake-python-styleguide's 7 methods per class.
max-methods = 15
# Comply with black's style. # Comply with black's style.
# Source: https://github.com/psf/black/blob/master/docs/the_black_code_style.md#line-length # Source: https://github.com/psf/black/blob/master/docs/the_black_code_style.md#line-length
max-line-length = 88 max-line-length = 88
@ -216,12 +135,10 @@ show-source = true
# wemake-python-styleguide's settings # wemake-python-styleguide's settings
# =================================== # ===================================
allowed-domain-names = allowed-domain-names =
data,
obj,
param, param,
result, result,
results,
value, value,
min-name-length = 3
max-name-length = 40 max-name-length = 40
# darglint # darglint
strictness = long strictness = long
@ -269,49 +186,42 @@ single_line_exclusions = typing
[mypy] [mypy]
cache_dir = .cache/mypy cache_dir = .cache/mypy
# Check the interior of functions without type annotations. [mypy-nox.*,packaging,pytest]
check_untyped_defs = true ignore_missing_imports = true
# Disallow generic types without explicit type parameters.
disallow_any_generics = true
# Disallow functions with incomplete type annotations. [pylint.FORMAT]
disallow_incomplete_defs = true # Comply with black's style.
max-line-length = 88
# Disallow calling functions without type annotations. [pylint.MESSAGES CONTROL]
disallow_untyped_calls = true disable =
# We use TODO's to indicate locations in the source base
# that must be worked on in the near future.
fixme,
# Comply with black's style.
bad-continuation, bad-whitespace,
# =====================
# flake8 de-duplication
# Source: https://pylint.pycqa.org/en/latest/faq.html#i-am-using-another-popular-linter-alongside-pylint-which-messages-should-i-disable-to-avoid-duplicates
# =====================
# mccabe
too-many-branches,
# pep8-naming
bad-classmethod-argument, bad-mcs-classmethod-argument,
invalid-name, no-self-argument,
# pycodestyle
bad-indentation, bare-except, line-too-long, missing-final-newline,
multiple-statements, trailing-whitespace, unnecessary-semicolon, unneeded-not,
# pydocstyle
missing-class-docstring, missing-function-docstring, missing-module-docstring,
# pyflakes
undefined-variable, unused-import, unused-variable,
# wemake-python-styleguide
redefined-outer-name,
# Disallow functions without type annotations (or incomplete annotations). [pylint.REPORTS]
disallow_untyped_defs = true score = no
[mypy-folium.*]
ignore_missing_imports = true
[mypy-geopy.*]
ignore_missing_imports = true
[mypy-googlemaps.*]
ignore_missing_imports = true
[mypy-matplotlib.*]
ignore_missing_imports = true
[mypy-nox.*]
ignore_missing_imports = true
[mypy-numpy.*]
ignore_missing_imports = true
[mypy-ordered_set.*]
ignore_missing_imports = true
[mypy-packaging]
ignore_missing_imports = true
[mypy-pandas]
ignore_missing_imports = true
[mypy-pytest]
ignore_missing_imports = true
[mypy-rpy2.*]
ignore_missing_imports = true
[mypy-sqlalchemy.*]
ignore_missing_imports = true
[mypy-statsmodels.*]
ignore_missing_imports = true
[mypy-utm.*]
ignore_missing_imports = true
[tool:pytest] [tool:pytest]
@ -319,11 +229,3 @@ addopts =
--strict-markers --strict-markers
cache_dir = .cache/pytest cache_dir = .cache/pytest
console_output_style = count console_output_style = count
env =
TESTING=true
filterwarnings =
ignore:::patsy.*
markers =
db: (integration) tests touching the database
e2e: non-db and non-r integration tests
r: (integration) tests using rpy2

5
src/urban_meal_delivery/__init__.py
View file

@ -5,14 +5,9 @@ Example:
>>> umd.__version__ != '0.0.0' >>> umd.__version__ != '0.0.0'
True True
""" """
# The config object must come before all other project-internal imports.
from urban_meal_delivery.configuration import config # isort:skip
from importlib import metadata as _metadata from importlib import metadata as _metadata
from urban_meal_delivery import db
from urban_meal_delivery import forecasts
try: try:
_pkg_info = _metadata.metadata(__name__) _pkg_info = _metadata.metadata(__name__)

138
src/urban_meal_delivery/configuration.py
View file

@ -1,138 +0,0 @@
"""Provide package-wide configuration.
This module provides utils to create new `Config` objects
on the fly, mainly for testing and migrating!
Within this package, use the `config` proxy at the package's top level
to access the current configuration!
"""
import datetime
import os
import random
import string
import warnings
def random_schema_name() -> str:
"""Generate a random PostgreSQL schema name for testing."""
return 'temp_{name}'.format(
name=''.join(
(random.choice(string.ascii_lowercase) for _ in range(10)), # noqa:S311
),
)
class Config:
"""Configuration that applies in all situations."""
# Application-specific settings
# -----------------------------
# Date after which the real-life data is discarded.
CUTOFF_DAY = datetime.datetime(2017, 2, 1)
# If a scheduled pre-order is made within this
# time horizon, we treat it as an ad-hoc order.
QUASI_AD_HOC_LIMIT = datetime.timedelta(minutes=45)
# Operating hours of the platform.
SERVICE_START = 11
SERVICE_END = 23
# Side lengths (in meters) for which pixel grids are created.
# They are the basis for the aggregated demand forecasts.
GRID_SIDE_LENGTHS = [707, 1000, 1414]
# Time steps (in minutes) used to aggregate the
# individual orders into time series.
TIME_STEPS = [60]
# Training horizons (in full weeks) used to train the forecasting models.
# For now, we only use 7 and 8 weeks as that was the best performing in
# a previous study (note:4f79e8fa).
TRAIN_HORIZONS = [7, 8]
# The demand forecasting methods used in the simulations.
FORECASTING_METHODS = ['hets', 'rtarima']
# Colors for the visualizations ins `folium`.
RESTAURANT_COLOR = 'red'
CUSTOMER_COLOR = 'blue'
NEUTRAL_COLOR = 'black'
# Implementation-specific settings
# --------------------------------
DATABASE_URI = os.getenv('DATABASE_URI')
GOOGLE_MAPS_API_KEY = os.getenv('GOOGLE_MAPS_API_KEY')
# The PostgreSQL schema that holds the tables with the original data.
ORIGINAL_SCHEMA = os.getenv('ORIGINAL_SCHEMA') or 'public'
# The PostgreSQL schema that holds the tables with the cleaned data.
CLEAN_SCHEMA = os.getenv('CLEAN_SCHEMA') or 'clean'
ALEMBIC_TABLE = 'alembic_version'
ALEMBIC_TABLE_SCHEMA = 'public'
R_LIBS_PATH = os.getenv('R_LIBS')
def __repr__(self) -> str:
"""Non-literal text representation."""
return '<configuration>'
class ProductionConfig(Config):
"""Configuration for the real dataset."""
TESTING = False
class TestingConfig(Config):
"""Configuration for the test suite."""
TESTING = True
DATABASE_URI = os.getenv('DATABASE_URI_TESTING') or Config.DATABASE_URI
CLEAN_SCHEMA = os.getenv('CLEAN_SCHEMA_TESTING') or random_schema_name()
def make_config(env: str = 'production') -> Config:
"""Create a new `Config` object.
Args:
env: either 'production' or 'testing'
Returns:
config: a namespace with all configurations
Raises:
ValueError: if `env` is not as specified
""" # noqa:DAR203
config: Config # otherwise mypy is confused
if env.strip().lower() == 'production':
config = ProductionConfig()
elif env.strip().lower() == 'testing':
config = TestingConfig()
else:
raise ValueError("Must be either 'production' or 'testing'")
# Without a PostgreSQL database the package cannot work.
# As pytest sets the "TESTING" environment variable explicitly,
# the warning is only emitted if the code is not run by pytest.
# We see the bad configuration immediately as all "db" tests fail.
if config.DATABASE_URI is None and not os.getenv('TESTING'):
warnings.warn('Bad configuration: no DATABASE_URI set in the environment')
# Some functionalities require R and some packages installed.
# To ensure isolation and reproducibility, the projects keeps the R dependencies
# in a project-local folder that must be set in the environment.
if config.R_LIBS_PATH is None and not os.getenv('TESTING'):
warnings.warn('Bad configuration: no R_LIBS set in the environment')
return config
config = make_config('testing' if os.getenv('TESTING') else 'production')

10
src/urban_meal_delivery/console/main.py → src/urban_meal_delivery/console.py
View file

@ -1,14 +1,14 @@
"""The entry point for all CLI scripts in the project.""" """Provide CLI scripts for the project."""
from typing import Any from typing import Any
import click import click
from click import core as cli_core from click.core import Context
import urban_meal_delivery import urban_meal_delivery
def show_version(ctx: cli_core.Context, _param: Any, value: bool) -> None: def show_version(ctx: Context, _param: Any, value: bool) -> None:
"""Show the package's version.""" """Show the package's version."""
# If --version / -V is NOT passed in, # If --version / -V is NOT passed in,
# continue with the command. # continue with the command.
@ -24,7 +24,7 @@ def show_version(ctx: cli_core.Context, _param: Any, value: bool) -> None:
ctx.exit() ctx.exit()
@click.group() @click.command()
@click.option( @click.option(
'--version', '--version',
'-V', '-V',
@ -33,5 +33,5 @@ def show_version(ctx: cli_core.Context, _param: Any, value: bool) -> None:
is_eager=True, is_eager=True,
expose_value=False, expose_value=False,
) )
def entry_point() -> None: def main() -> None:
"""The urban-meal-delivery research project.""" """The urban-meal-delivery research project."""

11
src/urban_meal_delivery/console/__init__.py
View file

@ -1,11 +0,0 @@
"""Provide CLI scripts for the project."""
from urban_meal_delivery.console import forecasts
from urban_meal_delivery.console import gridify
from urban_meal_delivery.console import main
cli = main.entry_point
cli.add_command(forecasts.tactical_heuristic, name='tactical-forecasts')
cli.add_command(gridify.gridify)

39
src/urban_meal_delivery/console/decorators.py
View file

@ -1,39 +0,0 @@
"""Utils for the CLI scripts."""
import functools
import os
import subprocess # noqa:S404
import sys
from typing import Any, Callable
import click
def db_revision(
rev: str,
) -> Callable[..., Callable[..., Any]]: # pragma: no cover -> easy to check visually
"""A decorator ensuring the database is at a given revision."""
def decorator(func: Callable[..., Any]) -> Callable[..., Any]:
@functools.wraps(func)
def ensure(*args: Any, **kwargs: Any) -> Any: # noqa:WPS430
"""Do not execute the `func` if the revision does not match."""
if not os.getenv('TESTING'):
result = subprocess.run( # noqa:S603,S607
['alembic', 'current'],
capture_output=True,
check=False,
encoding='utf8',
)
if not result.stdout.startswith(rev):
click.echo(
click.style(f'Database is not at revision {rev}', fg='red'),
)
sys.exit(1)
return func(*args, **kwargs)
return ensure
return decorator

149
src/urban_meal_delivery/console/forecasts.py
View file

@ -1,149 +0,0 @@
"""CLI script to forecast demand.
The main purpose of this script is to pre-populate the `db.Forecast` table
with demand predictions such that they can readily be used by the
predictive routing algorithms.
"""
import datetime as dt
import sys
import click
from sqlalchemy import func
from sqlalchemy.orm import exc as orm_exc
from urban_meal_delivery import config
from urban_meal_delivery import db
from urban_meal_delivery.console import decorators
from urban_meal_delivery.forecasts import timify
@click.command()
@click.argument('city', default='Paris', type=str)
@click.argument('side_length', default=1000, type=int)
@click.argument('time_step', default=60, type=int)
@click.argument('train_horizon', default=8, type=int)
@decorators.db_revision('8bfb928a31f8')
def tactical_heuristic( # noqa:C901,WPS213,WPS216,WPS231
city: str, side_length: int, time_step: int, train_horizon: int,
) -> None: # pragma: no cover
"""Predict demand for all pixels and days in a city.
This command makes demand `Forecast`s for all `Pixel`s and days
for tactical purposes with the heuristic specified in
`urban_meal_delivery.forecasts.timify.OrderHistory.choose_tactical_model()`.
According to this heuristic, there is exactly one `Forecast` per
`Pixel` and time step (e.g., hour of the day with 60-minute time steps)
given the lengths of the training horizon and a time step. That is so
as the heuristic chooses the most promising forecasting `*Model`.
All `Forecast`s are persisted to the database so that they can be readily
used by the predictive routing algorithms.
This command first checks, which `Forecast`s still need to be made
and then does its work. So, it can be interrupted at any point in
time and then simply continues where it left off the next time it
is executed.
Important: In a future revision, this command may need to be adapted such
that is does not simply obtain the last time step for which a `Forecast`
was made and continues from there. The reason is that another future command
may make predictions using all available forecasting `*Model`s per `Pixel`
and time step.
Arguments:
CITY: one of "Bordeaux", "Lyon", or "Paris" (=default)
SIDE_LENGTH: of a pixel in the grid; defaults to `1000`
TIME_STEP: length of one time step in minutes; defaults to `60`
TRAIN_HORIZON: length of the training horizon; defaults to `8`
""" # noqa:D412,D417,RST215
# Input validation.
try:
city_obj = (
db.session.query(db.City).filter_by(name=city.title()).one() # noqa:WPS221
)
except orm_exc.NoResultFound:
click.echo('NAME must be one of "Paris", "Lyon", or "Bordeaux"')
sys.exit(1)
for grid in city_obj.grids:
if grid.side_length == side_length:
break
else:
click.echo(f'SIDE_LENGTH must be in {config.GRID_SIDE_LENGTHS}')
sys.exit(1)
if time_step not in config.TIME_STEPS:
click.echo(f'TIME_STEP must be in {config.TIME_STEPS}')
sys.exit(1)
if train_horizon not in config.TRAIN_HORIZONS:
click.echo(f'TRAIN_HORIZON must be in {config.TRAIN_HORIZONS}')
sys.exit(1)
click.echo(
'Parameters: '
+ f'city="{city}", grid.side_length={side_length}, '
+ f'time_step={time_step}, train_horizon={train_horizon}',
)
# Load the historic order data.
order_history = timify.OrderHistory(grid=grid, time_step=time_step) # noqa:WPS441
order_history.aggregate_orders()
# Run the tactical heuristic.
for pixel in grid.pixels: # noqa:WPS441
# Important: this check may need to be adapted once further
# commands are added the make `Forecast`s without the heuristic!
# Continue with forecasting on the day the last prediction was made ...
last_predict_at = (
db.session.query(func.max(db.Forecast.start_at)) # noqa:WPS221
.join(db.Pixel, db.Forecast.pixel_id == db.Pixel.id)
.join(db.Grid, db.Pixel.grid_id == db.Grid.id)
.filter(db.Forecast.pixel == pixel)
.filter(db.Grid.side_length == side_length)
.filter(db.Forecast.time_step == time_step)
.filter(db.Forecast.train_horizon == train_horizon)
.first()
)[0]
# ... or start `train_horizon` weeks after the first `Order`
# if no `Forecast`s are in the database yet.
if last_predict_at is None:
predict_day = order_history.first_order_at(pixel_id=pixel.id).date()
predict_day += dt.timedelta(weeks=train_horizon)
else:
predict_day = last_predict_at.date()
# Go over all days in chronological order ...
while predict_day <= order_history.last_order_at(pixel_id=pixel.id).date():
# ... and choose the most promising `*Model` for that day.
model = order_history.choose_tactical_model(
pixel_id=pixel.id, predict_day=predict_day, train_horizon=train_horizon,
)
click.echo(
f'Predicting pixel #{pixel.id} in {city} '
+ f'for {predict_day} with {model.name}',
)
# Only loop over the time steps corresponding to working hours.
predict_at = dt.datetime(
predict_day.year,
predict_day.month,
predict_day.day,
config.SERVICE_START,
)
while predict_at.hour < config.SERVICE_END:
model.make_forecast(
pixel=pixel, predict_at=predict_at, train_horizon=train_horizon,
)
predict_at += dt.timedelta(minutes=time_step)
predict_day += dt.timedelta(days=1)

49
src/urban_meal_delivery/console/gridify.py
View file

@ -1,49 +0,0 @@
"""CLI script to create pixel grids."""
import click
from urban_meal_delivery import config
from urban_meal_delivery import db
from urban_meal_delivery.console import decorators
@click.command()
@decorators.db_revision('e86290e7305e')
def gridify() -> None: # pragma: no cover note:b1f68d24
"""Create grids for all cities.
This command creates grids with pixels of various
side lengths (specified in `urban_meal_delivery.config`).
Pixels are only generated if they contain at least one
(pickup or delivery) address.
All data are persisted to the database.
"""
cities = db.session.query(db.City).all()
click.echo(f'{len(cities)} cities retrieved from the database')
for city in cities:
click.echo(f'\nCreating grids for {city.name}')
for side_length in config.GRID_SIDE_LENGTHS:
click.echo(f'Creating grid with a side length of {side_length} meters')
grid = db.Grid.gridify(city=city, side_length=side_length)
db.session.add(grid)
click.echo(f' -> created {len(grid.pixels)} pixels')
# Because the number of assigned addresses is the same across
# different `side_length`s, we can take any `grid` from the `city`.
grid = db.session.query(db.Grid).filter_by(city=city).first()
n_assigned = (
db.session.query(db.AddressPixelAssociation)
.filter(db.AddressPixelAssociation.grid_id == grid.id)
.count()
)
click.echo(
f'=> assigned {n_assigned} out of {len(city.addresses)} addresses in {city.name}', # noqa:E501
)
db.session.commit()

17
src/urban_meal_delivery/db/__init__.py
View file

@ -1,17 +0,0 @@
"""Provide the ORM models and a connection to the database."""
from urban_meal_delivery.db.addresses import Address
from urban_meal_delivery.db.addresses_addresses import Path
from urban_meal_delivery.db.addresses_pixels import AddressPixelAssociation
from urban_meal_delivery.db.cities import City
from urban_meal_delivery.db.connection import connection
from urban_meal_delivery.db.connection import engine
from urban_meal_delivery.db.connection import session
from urban_meal_delivery.db.couriers import Courier
from urban_meal_delivery.db.customers import Customer
from urban_meal_delivery.db.forecasts import Forecast
from urban_meal_delivery.db.grids import Grid
from urban_meal_delivery.db.meta import Base
from urban_meal_delivery.db.orders import Order
from urban_meal_delivery.db.pixels import Pixel
from urban_meal_delivery.db.restaurants import Restaurant

163
src/urban_meal_delivery/db/addresses.py
View file

@ -1,163 +0,0 @@
"""Provide the ORM's `Address` model."""
from __future__ import annotations
import functools
from typing import Any
import folium
import sqlalchemy as sa
from sqlalchemy import orm
from sqlalchemy.dialects import postgresql
from sqlalchemy.ext import hybrid
from urban_meal_delivery import config
from urban_meal_delivery.db import meta
from urban_meal_delivery.db import utils
class Address(meta.Base):
"""An address of a `Customer` or a `Restaurant` on the UDP."""
__tablename__ = 'addresses'
# Columns
id = sa.Column(sa.Integer, primary_key=True, autoincrement=False) # noqa:WPS125
primary_id = sa.Column(sa.Integer, nullable=False, index=True)
created_at = sa.Column(sa.DateTime, nullable=False)
place_id = sa.Column(sa.Unicode(length=120), nullable=False, index=True)
latitude = sa.Column(postgresql.DOUBLE_PRECISION, nullable=False)
longitude = sa.Column(postgresql.DOUBLE_PRECISION, nullable=False)
city_id = sa.Column(sa.SmallInteger, nullable=False, index=True)
city_name = sa.Column('city', sa.Unicode(length=25), nullable=False)
zip_code = sa.Column(sa.Integer, nullable=False, index=True)
street = sa.Column(sa.Unicode(length=80), nullable=False)
floor = sa.Column(sa.SmallInteger)
# Constraints
__table_args__ = (
sa.ForeignKeyConstraint(
['primary_id'], ['addresses.id'], onupdate='RESTRICT', ondelete='RESTRICT',
),
sa.ForeignKeyConstraint(
['city_id'], ['cities.id'], onupdate='RESTRICT', ondelete='RESTRICT',
),
sa.CheckConstraint(
'-90 <= latitude AND latitude <= 90', name='latitude_between_90_degrees',
),
sa.CheckConstraint(
'-180 <= longitude AND longitude <= 180',
name='longitude_between_180_degrees',
),
# Needed by a `ForeignKeyConstraint` in `AddressPixelAssociation`.
sa.UniqueConstraint('id', 'city_id'),
sa.CheckConstraint(
'30000 <= zip_code AND zip_code <= 99999', name='valid_zip_code',
),
sa.CheckConstraint('0 <= floor AND floor <= 40', name='realistic_floor'),
)
# Relationships
city = orm.relationship('City', back_populates='addresses')
restaurants = orm.relationship('Restaurant', back_populates='address')
orders_picked_up = orm.relationship(
'Order',
back_populates='pickup_address',
foreign_keys='[Order.pickup_address_id]',
)
orders_delivered = orm.relationship(
'Order',
back_populates='delivery_address',
foreign_keys='[Order.delivery_address_id]',
)
pixels = orm.relationship('AddressPixelAssociation', back_populates='address')
def __repr__(self) -> str:
"""Non-literal text representation."""
return '<{cls}({street} in {city})>'.format(
cls=self.__class__.__name__, street=self.street, city=self.city_name,
)
@hybrid.hybrid_property
def is_primary(self) -> bool:
"""If an `Address` object is the earliest one entered at its location.
Street addresses may have been entered several times with different
versions/spellings of the street name and/or different floors.
`.is_primary` indicates the first in a group of `Address` objects.
"""
return self.id == self.primary_id
@functools.cached_property
def location(self) -> utils.Location:
"""The location of the address.
The returned `Location` object relates to `.city.southwest`.
See also the `.x` and `.y` properties that are shortcuts for
`.location.x` and `.location.y`.
Implementation detail: This property is cached as none of the
underlying attributes to calculate the value are to be changed.
"""
location = utils.Location(self.latitude, self.longitude)
location.relate_to(self.city.southwest)
return location
@property
def x(self) -> int: # noqa=WPS111
"""The relative x-coordinate within the `.city` in meters.
On the implied x-y plane, the `.city`'s southwest corner is the origin.
Shortcut for `.location.x`.
"""
return self.location.x
@property
def y(self) -> int: # noqa=WPS111
"""The relative y-coordinate within the `.city` in meters.
On the implied x-y plane, the `.city`'s southwest corner is the origin.
Shortcut for `.location.y`.
"""
return self.location.y
def clear_map(self) -> Address: # pragma: no cover
"""Shortcut to the `.city.clear_map()` method.
Returns:
self: enabling method chaining
""" # noqa:D402,DAR203
self.city.clear_map()
return self
@property # pragma: no cover
def map(self) -> folium.Map: # noqa:WPS125
"""Shortcut to the `.city.map` object."""
return self.city.map
def draw(self, **kwargs: Any) -> folium.Map: # pragma: no cover
"""Draw the address on the `.city.map`.
By default, addresses are shown as black dots.
Use `**kwargs` to overwrite that.
Args:
**kwargs: passed on to `folium.Circle()`; overwrite default settings
Returns:
`.city.map` for convenience in interactive usage
"""
defaults = {
'color': f'{config.NEUTRAL_COLOR}',
'popup': f'{self.street}, {self.zip_code} {self.city_name}',
}
defaults.update(kwargs)
marker = folium.Circle((self.latitude, self.longitude), **defaults)
marker.add_to(self.city.map)
return self.map

316
src/urban_meal_delivery/db/addresses_addresses.py
View file

@ -1,316 +0,0 @@
"""Model for the `Path` relationship between two `Address` objects."""
from __future__ import annotations
import functools
import itertools
import json
from typing import List
import folium
import googlemaps as gm
import ordered_set
import sqlalchemy as sa
from geopy import distance as geo_distance
from sqlalchemy import orm
from sqlalchemy.dialects import postgresql
from urban_meal_delivery import config
from urban_meal_delivery import db
from urban_meal_delivery.db import meta
from urban_meal_delivery.db import utils
class Path(meta.Base):
"""Path between two `Address` objects.
Models the path between two `Address` objects, including directions
for a `Courier` to get from one `Address` to another.
As the couriers are on bicycles, we model the paths as
a symmetric graph (i.e., same distance in both directions).
Implements an association pattern between `Address` and `Address`.
Further info:
https://docs.sqlalchemy.org/en/stable/orm/basic_relationships.html#association-object # noqa:E501
"""
__tablename__ = 'addresses_addresses'
# Columns
first_address_id = sa.Column(sa.Integer, primary_key=True)
second_address_id = sa.Column(sa.Integer, primary_key=True)
city_id = sa.Column(sa.SmallInteger, nullable=False)
# Distances are measured in meters.
air_distance = sa.Column(sa.Integer, nullable=False)
bicycle_distance = sa.Column(sa.Integer, nullable=True)
# The duration is measured in seconds.
bicycle_duration = sa.Column(sa.Integer, nullable=True)
# An array of latitude-longitude pairs approximating a courier's way.
_directions = sa.Column('directions', postgresql.JSON, nullable=True)
# Constraints
__table_args__ = (
# The two `Address` objects must be in the same `.city`.
sa.ForeignKeyConstraint(
['first_address_id', 'city_id'],
['addresses.id', 'addresses.city_id'],
onupdate='RESTRICT',
ondelete='RESTRICT',
),
sa.ForeignKeyConstraint(
['second_address_id', 'city_id'],
['addresses.id', 'addresses.city_id'],
onupdate='RESTRICT',
ondelete='RESTRICT',
),
# Each `Address`-`Address` pair only has one distance.
sa.UniqueConstraint('first_address_id', 'second_address_id'),
sa.CheckConstraint(
'first_address_id < second_address_id',
name='distances_are_symmetric_for_bicycles',
),
sa.CheckConstraint(
'0 <= air_distance AND air_distance < 20000', name='realistic_air_distance',
),
sa.CheckConstraint(
'bicycle_distance < 25000', # `.bicycle_distance` may not be negative
name='realistic_bicycle_distance', # due to the constraint below.
),
sa.CheckConstraint(
'air_distance <= bicycle_distance', name='air_distance_is_shortest',
),
sa.CheckConstraint(
'0 <= bicycle_duration AND bicycle_duration <= 3600',
name='realistic_bicycle_travel_time',
),
)
# Relationships
first_address = orm.relationship(
'Address', foreign_keys='[Path.first_address_id, Path.city_id]',
)
second_address = orm.relationship(
'Address',
foreign_keys='[Path.second_address_id, Path.city_id]',
overlaps='first_address',
)
@classmethod
def from_addresses(
cls, *addresses: db.Address, google_maps: bool = False,
) -> List[Path]:
"""Calculate pair-wise paths for `Address` objects.
This is the main constructor method for the class.
It handles the "sorting" of the `Address` objects by `.id`, which is
the logic that enforces the symmetric graph behind the paths.
Args:
*addresses: to calculate the pair-wise paths for;
must contain at least two `Address` objects
google_maps: if `.bicycle_distance` and `._directions` should be
populated with a query to the Google Maps Directions API;
by default, only the `.air_distance` is calculated with `geopy`
Returns:
paths
"""
paths = []
# We consider all 2-tuples of `Address`es. The symmetric graph is ...
for first, second in itertools.combinations(addresses, 2):
# ... implicitly enforced by a precedence constraint for the `.id`s.
first, second = ( # noqa:WPS211
(first, second) if first.id < second.id else (second, first)
)
# If there is no `Path` object in the database ...
path = (
db.session.query(db.Path)
.filter(db.Path.first_address == first)
.filter(db.Path.second_address == second)
.first()
)
# ... create a new one.
if path is None:
air_distance = geo_distance.great_circle(
first.location.lat_lng, second.location.lat_lng,
)
path = cls(
first_address=first,
second_address=second,
air_distance=round(air_distance.meters),
)
db.session.add(path)
db.session.commit()
paths.append(path)
if google_maps:
for path in paths: # noqa:WPS440
path.sync_with_google_maps()
return paths
@classmethod
def from_order(cls, order: db.Order, google_maps: bool = False) -> Path:
"""Calculate the path for an `Order` object.
The path goes from the `Order.pickup_address` to the `Order.delivery_address`.
Args:
order: to calculate the path for
google_maps: if `.bicycle_distance` and `._directions` should be
populated with a query to the Google Maps Directions API;
by default, only the `.air_distance` is calculated with `geopy`
Returns:
path
"""
return cls.from_addresses(
order.pickup_address, order.delivery_address, google_maps=google_maps,
)[0]
def sync_with_google_maps(self) -> None:
"""Fill in `.bicycle_distance` and `._directions` with Google Maps.
`._directions` will NOT contain the coordinates
of `.first_address` and `.second_address`.
This uses the Google Maps Directions API.
Further info:
https://developers.google.com/maps/documentation/directions
"""
# To save costs, we do not make an API call
# if we already have data from Google Maps.
if self.bicycle_distance is not None:
return
client = gm.Client(config.GOOGLE_MAPS_API_KEY)
response = client.directions(
origin=self.first_address.location.lat_lng,
destination=self.second_address.location.lat_lng,
mode='bicycling',
alternatives=False,
)
# Without "alternatives" and "waypoints", the `response` contains
# exactly one "route" that consists of exactly one "leg".
# Source: https://developers.google.com/maps/documentation/directions/get-directions#Legs # noqa:E501
route = response[0]['legs'][0]
self.bicycle_distance = route['distance']['value'] # noqa:WPS601
self.bicycle_duration = route['duration']['value'] # noqa:WPS601
# Each route consists of many "steps" that are instructions as to how to
# get from A to B. As a step's "start_location" may equal the previous step's
# "end_location", we use an `OrderedSet` to find the unique latitude-longitude
# pairs that make up the path from `.first_address` to `.second_address`.
steps = ordered_set.OrderedSet()
for step in route['steps']:
steps.add( # noqa:WPS221
(step['start_location']['lat'], step['start_location']['lng']),
)
steps.add( # noqa:WPS221
(step['end_location']['lat'], step['end_location']['lng']),
)
steps.discard(self.first_address.location.lat_lng)
steps.discard(self.second_address.location.lat_lng)
self._directions = json.dumps(list(steps)) # noqa:WPS601
db.session.add(self)
db.session.commit()
@property # pragma: no cover
def map(self) -> folium.Map: # noqa:WPS125
"""Convenience property to obtain the underlying `City.map`."""
return self.first_address.city.map
@functools.cached_property
def waypoints(self) -> List[utils.Location]:
"""The couriers' route from `.first_address` to `.second_address`.
The returned `Location`s all relate to `.first_address.city.southwest`.
Implementation detail: This property is cached as none of the
underlying attributes (i.e., `._directions`) are to be changed.
"""
points = [utils.Location(*point) for point in json.loads(self._directions)]
for point in points:
point.relate_to(self.first_address.city.southwest)
return points
def draw( # noqa:WPS211
self,
*,
reverse: bool = False,
start_tooltip: str = 'Start',
end_tooltip: str = 'End',
start_color: str = 'green',
end_color: str = 'red',
path_color: str = 'black',
) -> folium.Map: # pragma: no cover
"""Draw the `.waypoints` from `.first_address` to `.second_address`.
Args:
reverse: by default, `.first_address` is used as the start;
set to `False` to make `.second_address` the start
start_tooltip: text shown on marker at the path's start
end_tooltip: text shown on marker at the path's end
start_color: `folium` color for the path's start
end_color: `folium` color for the path's end
path_color: `folium` color along the path, which
is the line between the `.waypoints`
Returns:
`.map` for convenience in interactive usage
"""
# Without `self._directions` synced from Google Maps,
# the `.waypoints` are not available.
self.sync_with_google_maps()
# First, plot the couriers' path between the start and
# end locations, so that it is below the `folium.Circle`s.
line = folium.PolyLine(
locations=(
self.first_address.location.lat_lng,
*(point.lat_lng for point in self.waypoints),
self.second_address.location.lat_lng,
),
color=path_color,
weight=2,
)
line.add_to(self.map)
# Draw the path's start and end locations, possibly reversed,
# on top of the couriers' path.
if reverse:
start, end = self.second_address, self.first_address
else:
start, end = self.first_address, self.second_address
start.draw(
radius=5,
color=start_color,
fill_color=start_color,
fill_opacity=1,
tooltip=start_tooltip,
)
end.draw(
radius=5,
color=end_color,
fill_color=end_color,
fill_opacity=1,
tooltip=end_tooltip,
)
return self.map

56
src/urban_meal_delivery/db/addresses_pixels.py
View file

@ -1,56 +0,0 @@
"""Model for the many-to-many relationship between `Address` and `Pixel` objects."""
import sqlalchemy as sa
from sqlalchemy import orm
from urban_meal_delivery.db import meta
class AddressPixelAssociation(meta.Base):
"""Association pattern between `Address` and `Pixel`.
This approach is needed here mainly because it implicitly
updates the `city_id` and `grid_id` columns.
Further info:
https://docs.sqlalchemy.org/en/stable/orm/basic_relationships.html#association-object # noqa:E501
"""
__tablename__ = 'addresses_pixels'
# Columns
address_id = sa.Column(sa.Integer, primary_key=True)
city_id = sa.Column(sa.SmallInteger, nullable=False)
grid_id = sa.Column(sa.SmallInteger, nullable=False)
pixel_id = sa.Column(sa.Integer, primary_key=True)
# Constraints
__table_args__ = (
# An `Address` can only be on a `Grid` ...
sa.ForeignKeyConstraint(
['address_id', 'city_id'],
['addresses.id', 'addresses.city_id'],
onupdate='RESTRICT',
ondelete='RESTRICT',
),
# ... if their `.city` attributes match.
sa.ForeignKeyConstraint(
['grid_id', 'city_id'],
['grids.id', 'grids.city_id'],
onupdate='RESTRICT',
ondelete='RESTRICT',
),
# Each `Address` can only be on a `Grid` once.
sa.UniqueConstraint('address_id', 'grid_id'),
# An association must reference an existing `Grid`-`Pixel` pair.
sa.ForeignKeyConstraint(
['pixel_id', 'grid_id'],
['pixels.id', 'pixels.grid_id'],
onupdate='RESTRICT',
ondelete='RESTRICT',
),
)
# Relationships
address = orm.relationship('Address', back_populates='pixels')
pixel = orm.relationship('Pixel', back_populates='addresses')

243
src/urban_meal_delivery/db/cities.py
View file

@ -1,243 +0,0 @@
"""Provide the ORM's `City` model."""
from __future__ import annotations
import functools
import folium
import sqlalchemy as sa
from sqlalchemy import orm
from sqlalchemy.dialects import postgresql
from urban_meal_delivery import config
from urban_meal_delivery import db
from urban_meal_delivery.db import meta
from urban_meal_delivery.db import utils
class City(meta.Base):
"""A city where the UDP operates in."""
__tablename__ = 'cities'
# Generic columns
id = sa.Column( # noqa:WPS125
sa.SmallInteger, primary_key=True, autoincrement=False,
)
name = sa.Column(sa.Unicode(length=10), nullable=False)
kml = sa.Column(sa.UnicodeText, nullable=False)
# Google Maps related columns
center_latitude = sa.Column(postgresql.DOUBLE_PRECISION, nullable=False)
center_longitude = sa.Column(postgresql.DOUBLE_PRECISION, nullable=False)
northeast_latitude = sa.Column(postgresql.DOUBLE_PRECISION, nullable=False)
northeast_longitude = sa.Column(postgresql.DOUBLE_PRECISION, nullable=False)
southwest_latitude = sa.Column(postgresql.DOUBLE_PRECISION, nullable=False)
southwest_longitude = sa.Column(postgresql.DOUBLE_PRECISION, nullable=False)
initial_zoom = sa.Column(sa.SmallInteger, nullable=False)
# Relationships
addresses = orm.relationship('Address', back_populates='city')
grids = orm.relationship('Grid', back_populates='city')
# We do not implement a `.__init__()` method and use SQLAlchemy's default.
# The uninitialized attribute `._map` is computed on the fly. note:d334120ei
def __repr__(self) -> str:
"""Non-literal text representation."""
return '<{cls}({name})>'.format(cls=self.__class__.__name__, name=self.name)
@functools.cached_property
def center(self) -> utils.Location:
"""Location of the city's center.
Implementation detail: This property is cached as none of the
underlying attributes to calculate the value are to be changed.
"""
return utils.Location(self.center_latitude, self.center_longitude)
@functools.cached_property
def northeast(self) -> utils.Location:
"""The city's northeast corner of the Google Maps viewport.
Implementation detail: This property is cached as none of the
underlying attributes to calculate the value are to be changed.
"""
return utils.Location(self.northeast_latitude, self.northeast_longitude)
@functools.cached_property
def southwest(self) -> utils.Location:
"""The city's southwest corner of the Google Maps viewport.
Implementation detail: This property is cached as none of the
underlying attributes to calculate the value are to be changed.
"""
return utils.Location(self.southwest_latitude, self.southwest_longitude)
@property
def total_x(self) -> int:
"""The horizontal distance from the city's west to east end in meters.
The city borders refer to the Google Maps viewport.
"""
return self.northeast.easting - self.southwest.easting
@property
def total_y(self) -> int:
"""The vertical distance from the city's south to north end in meters.
The city borders refer to the Google Maps viewport.
"""
return self.northeast.northing - self.southwest.northing
def clear_map(self) -> City: # pragma: no cover
"""Create a new `folium.Map` object aligned with the city's viewport.
The map is available via the `.map` property. Note that it is mutable
and changed from various locations in the code base.
Returns:
self: enabling method chaining
""" # noqa:DAR203 note:d334120e
self._map = folium.Map(
location=[self.center_latitude, self.center_longitude],
zoom_start=self.initial_zoom,
)
return self
@property # pragma: no cover
def map(self) -> folium.Map: # noqa:WPS125
"""A `folium.Map` object aligned with the city's viewport.
See docstring for `.clear_map()` for further info.
"""
if not hasattr(self, '_map'): # noqa:WPS421 note:d334120e
self.clear_map()
return self._map
def draw_restaurants( # noqa:WPS231
self, order_counts: bool = False, # pragma: no cover
) -> folium.Map:
"""Draw all restaurants on the`.map`.
Args:
order_counts: show the number of orders
Returns:
`.map` for convenience in interactive usage
"""
# Obtain all primary `Address`es in the city that host `Restaurant`s.
addresses = (
db.session.query(db.Address)
.filter(
db.Address.id.in_(
db.session.query(db.Address.primary_id) # noqa:WPS221
.join(db.Restaurant, db.Address.id == db.Restaurant.address_id)
.filter(db.Address.city == self)
.distinct()
.all(),
),
)
.all()
)
for address in addresses:
# Show the restaurant's name if there is only one.
# Otherwise, list all the restaurants' ID's.
restaurants = (
db.session.query(db.Restaurant)
.join(db.Address, db.Restaurant.address_id == db.Address.id)
.filter(db.Address.primary_id == address.id)
.all()
)
if len(restaurants) == 1:
tooltip = f'{restaurants[0].name} (#{restaurants[0].id})' # noqa:WPS221
else:
tooltip = 'Restaurants ' + ', '.join( # noqa:WPS336
f'#{restaurant.id}' for restaurant in restaurants
)
if order_counts:
# Calculate the number of orders for ALL restaurants ...
n_orders = (
db.session.query(db.Order.id)
.join(db.Address, db.Order.pickup_address_id == db.Address.id)
.filter(db.Address.primary_id == address.id)
.count()
)
# ... and adjust the size of the red dot on the `.map`.
if n_orders >= 1000:
radius = 20 # noqa:WPS220
elif n_orders >= 500:
radius = 15 # noqa:WPS220
elif n_orders >= 100:
radius = 10 # noqa:WPS220
elif n_orders >= 10:
radius = 5 # noqa:WPS220
else:
radius = 1 # noqa:WPS220
tooltip += f' | n_orders={n_orders}' # noqa:WPS336
address.draw(
radius=radius,
color=config.RESTAURANT_COLOR,
fill_color=config.RESTAURANT_COLOR,
fill_opacity=0.3,
tooltip=tooltip,
)
else:
address.draw(
radius=1, color=config.RESTAURANT_COLOR, tooltip=tooltip,
)
return self.map
def draw_zip_codes(self) -> folium.Map: # pragma: no cover
"""Draw all addresses on the `.map`, colorized by their `.zip_code`.
This does not make a distinction between restaurant and customer addresses.
Also, due to the high memory usage, the number of orders is not calculated.
Returns:
`.map` for convenience in interactive usage
"""
# First, create a color map with distinct colors for each zip code.
all_zip_codes = sorted(
row[0]
for row in db.session.execute(
sa.text(
f""" -- # noqa:S608
SELECT DISTINCT
{config.CLEAN_SCHEMA}.addresses.zip_code
FROM
{config.CLEAN_SCHEMA}.addresses AS addresses
WHERE
{config.CLEAN_SCHEMA}.addresses.city_id = {self.id};
""",
),
)
)
cmap = utils.make_random_cmap(len(all_zip_codes), bright=False)
colors = {
code: utils.rgb_to_hex(*cmap(index))
for index, code in enumerate(all_zip_codes)
}
# Second, draw every address on the `.map.
for address in self.addresses:
# Non-primary addresses are covered by primary ones anyway.
if not address.is_primary:
continue
marker = folium.Circle( # noqa:WPS317
(address.latitude, address.longitude),
color=colors[address.zip_code],
radius=1,
)
marker.add_to(self.map)
return self.map

28
src/urban_meal_delivery/db/connection.py
View file

@ -1,28 +0,0 @@
"""Provide connection utils for the ORM layer.
This module defines fully configured `engine`, `connection`, and `session`
objects to be used as globals within the `urban_meal_delivery` package.
If a database is not guaranteed to be available, they are set to `None`.
That is the case on the CI server.
"""
import os
import sqlalchemy as sa
from sqlalchemy import engine as engine_mod
from sqlalchemy import orm
import urban_meal_delivery
if os.getenv('TESTING'):
# Specify the types explicitly to make mypy happy.
engine: engine_mod.Engine = None
connection: engine_mod.Connection = None
session: orm.Session = None
else: # pragma: no cover
engine = sa.create_engine(urban_meal_delivery.config.DATABASE_URI)
connection = engine.connect()
session = orm.sessionmaker(bind=connection)()

49
src/urban_meal_delivery/db/couriers.py
View file

@ -1,49 +0,0 @@
"""Provide the ORM's `Courier` model."""
import sqlalchemy as sa
from sqlalchemy import orm
from sqlalchemy.dialects import postgresql
from urban_meal_delivery.db import meta
class Courier(meta.Base):
"""A courier working for the UDP."""
__tablename__ = 'couriers'
# Columns
id = sa.Column(sa.Integer, primary_key=True, autoincrement=False) # noqa:WPS125
created_at = sa.Column(sa.DateTime, nullable=False)
vehicle = sa.Column(sa.Unicode(length=10), nullable=False)
historic_speed = sa.Column('speed', postgresql.DOUBLE_PRECISION, nullable=False)
capacity = sa.Column(sa.SmallInteger, nullable=False)
pay_per_hour = sa.Column(sa.SmallInteger, nullable=False)
pay_per_order = sa.Column(sa.SmallInteger, nullable=False)
# Constraints
__table_args__ = (
sa.CheckConstraint(
"vehicle IN ('bicycle', 'motorcycle')", name='available_vehicle_types',
),
sa.CheckConstraint('0 <= speed AND speed <= 30', name='realistic_speed'),
sa.CheckConstraint(
'0 <= capacity AND capacity <= 200', name='capacity_under_200_liters',
),
sa.CheckConstraint(
'0 <= pay_per_hour AND pay_per_hour <= 1500', name='realistic_pay_per_hour',
),
sa.CheckConstraint(
'0 <= pay_per_order AND pay_per_order <= 650',
name='realistic_pay_per_order',
),
)
# Relationships
orders = orm.relationship('Order', back_populates='courier')
def __repr__(self) -> str:
"""Non-literal text representation."""
return '<{cls}(#{courier_id})>'.format(
cls=self.__class__.__name__, courier_id=self.id,
)

184
src/urban_meal_delivery/db/customers.py
View file

@ -1,184 +0,0 @@
"""Provide the ORM's `Customer` model."""
from __future__ import annotations
import folium
import sqlalchemy as sa
from sqlalchemy import orm
from urban_meal_delivery import config
from urban_meal_delivery import db
from urban_meal_delivery.db import meta
class Customer(meta.Base):
"""A customer of the UDP."""
__tablename__ = 'customers'
# Columns
id = sa.Column(sa.Integer, primary_key=True, autoincrement=False) # noqa:WPS125
def __repr__(self) -> str:
"""Non-literal text representation."""
return '<{cls}(#{customer_id})>'.format(
cls=self.__class__.__name__, customer_id=self.id,
)
# Relationships
orders = orm.relationship('Order', back_populates='customer')
def clear_map(self) -> Customer: # pragma: no cover
"""Shortcut to the `...city.clear_map()` method.
Returns:
self: enabling method chaining
""" # noqa:D402,DAR203
self.orders[0].pickup_address.city.clear_map() # noqa:WPS219
return self
@property # pragma: no cover
def map(self) -> folium.Map: # noqa:WPS125
"""Shortcut to the `...city.map` object."""
return self.orders[0].pickup_address.city.map # noqa:WPS219
def draw( # noqa:C901,WPS210,WPS231
self, restaurants: bool = True, order_counts: bool = False, # pragma: no cover
) -> folium.Map:
"""Draw all the customer's delivery addresses on the `...city.map`.
By default, the pickup locations (= restaurants) are also shown.
Args:
restaurants: show the pickup locations
order_counts: show both the number of pickups at the restaurants
and the number of deliveries at the customer's delivery addresses;
the former is only shown if `restaurants=True`
Returns:
`...city.map` for convenience in interactive usage
"""
# Note: a `Customer` may have more than one delivery `Address`es.
# That is not true for `Restaurant`s after the data cleaning.
# Obtain all primary `Address`es where
# at least one delivery was made to `self`.
delivery_addresses = (
db.session.query(db.Address)
.filter(
db.Address.id.in_(
row.primary_id
for row in (
db.session.query(db.Address.primary_id) # noqa:WPS221
.join(db.Order, db.Address.id == db.Order.delivery_address_id)
.filter(db.Order.customer_id == self.id)
.distinct()
.all()
)
),
)
.all()
)
for address in delivery_addresses:
if order_counts:
n_orders = (
db.session.query(db.Order)
.join(db.Address, db.Order.delivery_address_id == db.Address.id)
.filter(db.Order.customer_id == self.id)
.filter(db.Address.primary_id == address.id)
.count()
)
if n_orders >= 25:
radius = 20 # noqa:WPS220
elif n_orders >= 10:
radius = 15 # noqa:WPS220
elif n_orders >= 5:
radius = 10 # noqa:WPS220
elif n_orders > 1:
radius = 5 # noqa:WPS220
else:
radius = 1 # noqa:WPS220
address.draw(
radius=radius,
color=config.CUSTOMER_COLOR,
fill_color=config.CUSTOMER_COLOR,
fill_opacity=0.3,
tooltip=f'n_orders={n_orders}',
)
else:
address.draw(
radius=1, color=config.CUSTOMER_COLOR,
)
if restaurants:
pickup_addresses = (
db.session.query(db.Address)
.filter(
db.Address.id.in_(
db.session.query(db.Address.primary_id) # noqa:WPS221
.join(db.Order, db.Address.id == db.Order.pickup_address_id)
.filter(db.Order.customer_id == self.id)
.distinct()
.all(),
),
)
.all()
)
for address in pickup_addresses: # noqa:WPS440
# Show the restaurant's name if there is only one.
# Otherwise, list all the restaurants' ID's.
# We cannot show the `Order.restaurant.name` due to the aggregation.
restaurants = (
db.session.query(db.Restaurant)
.join(db.Address, db.Restaurant.address_id == db.Address.id)
.filter(db.Address.primary_id == address.id) # noqa:WPS441
.all()
)
if len(restaurants) == 1: # type:ignore
tooltip = (
f'{restaurants[0].name} (#{restaurants[0].id})' # type:ignore
)
else:
tooltip = 'Restaurants ' + ', '.join( # noqa:WPS336
f'#{restaurant.id}' for restaurant in restaurants # type:ignore
)
if order_counts:
n_orders = (
db.session.query(db.Order)
.join(db.Address, db.Order.pickup_address_id == db.Address.id)
.filter(db.Order.customer_id == self.id)
.filter(db.Address.primary_id == address.id) # noqa:WPS441
.count()
)
if n_orders >= 25:
radius = 20 # noqa:WPS220
elif n_orders >= 10:
radius = 15 # noqa:WPS220
elif n_orders >= 5:
radius = 10 # noqa:WPS220
elif n_orders > 1:
radius = 5 # noqa:WPS220
else:
radius = 1 # noqa:WPS220
tooltip += f' | n_orders={n_orders}' # noqa:WPS336
address.draw( # noqa:WPS441
radius=radius,
color=config.RESTAURANT_COLOR,
fill_color=config.RESTAURANT_COLOR,
fill_opacity=0.3,
tooltip=tooltip,
)
else:
address.draw( # noqa:WPS441
radius=1, color=config.RESTAURANT_COLOR, tooltip=tooltip,
)
return self.map

231
src/urban_meal_delivery/db/forecasts.py
View file

@ -1,231 +0,0 @@
"""Provide the ORM's `Forecast` model."""
from __future__ import annotations
import math
from typing import List
import pandas as pd
import sqlalchemy as sa
from sqlalchemy import orm
from sqlalchemy.dialects import postgresql
from urban_meal_delivery.db import meta
class Forecast(meta.Base):
"""A demand forecast for a `.pixel` and `.time_step` pair.
This table is denormalized on purpose to keep things simple. In particular,
the `.model` and `.actual` hold redundant values.
"""
__tablename__ = 'forecasts'
# Columns
id = sa.Column(sa.Integer, primary_key=True, autoincrement=True) # noqa:WPS125
pixel_id = sa.Column(sa.Integer, nullable=False, index=True)
start_at = sa.Column(sa.DateTime, nullable=False)
time_step = sa.Column(sa.SmallInteger, nullable=False)
train_horizon = sa.Column(sa.SmallInteger, nullable=False)
model = sa.Column(sa.Unicode(length=20), nullable=False)
# We also store the actual order counts for convenient retrieval.
# A `UniqueConstraint` below ensures that redundant values that
# are to be expected are consistent across rows.
actual = sa.Column(sa.SmallInteger, nullable=False)
# Raw `.prediction`s are stored as `float`s (possibly negative).
# The rounding is then done on the fly if required.
prediction = sa.Column(postgresql.DOUBLE_PRECISION, nullable=False)
# The confidence intervals are treated like the `.prediction`s
# but they may be nullable as some methods do not calculate them.
low80 = sa.Column(postgresql.DOUBLE_PRECISION, nullable=True)
high80 = sa.Column(postgresql.DOUBLE_PRECISION, nullable=True)
low95 = sa.Column(postgresql.DOUBLE_PRECISION, nullable=True)
high95 = sa.Column(postgresql.DOUBLE_PRECISION, nullable=True)
# Constraints
__table_args__ = (
sa.ForeignKeyConstraint(
['pixel_id'], ['pixels.id'], onupdate='RESTRICT', ondelete='RESTRICT',
),
sa.CheckConstraint(
"""
NOT (
EXTRACT(HOUR FROM start_at) < 11
OR
EXTRACT(HOUR FROM start_at) > 22
)
""",
name='start_at_must_be_within_operating_hours',
),
sa.CheckConstraint(
'CAST(EXTRACT(MINUTES FROM start_at) AS INTEGER) % 15 = 0',
name='start_at_minutes_must_be_quarters_of_the_hour',
),
sa.CheckConstraint(
'EXTRACT(SECONDS FROM start_at) = 0', name='start_at_allows_no_seconds',
),
sa.CheckConstraint(
'CAST(EXTRACT(MICROSECONDS FROM start_at) AS INTEGER) % 1000000 = 0',
name='start_at_allows_no_microseconds',
),
sa.CheckConstraint('time_step > 0', name='time_step_must_be_positive'),
sa.CheckConstraint(
'train_horizon > 0', name='training_horizon_must_be_positive',
),
sa.CheckConstraint('actual >= 0', name='actuals_must_be_non_negative'),
sa.CheckConstraint(
"""
NOT (
low80 IS NULL AND high80 IS NOT NULL
OR
low80 IS NOT NULL AND high80 IS NULL
OR
low95 IS NULL AND high95 IS NOT NULL
OR
low95 IS NOT NULL AND high95 IS NULL
)
""",
name='ci_upper_and_lower_bounds',
),
sa.CheckConstraint(
"""
NOT (
prediction < low80
OR
prediction < low95
OR
prediction > high80
OR
prediction > high95
)
""",
name='prediction_must_be_within_ci',
),
sa.CheckConstraint(
"""
NOT (
low80 > high80
OR
low95 > high95
)
""",
name='ci_upper_bound_greater_than_lower_bound',
),
sa.CheckConstraint(
"""
NOT (
low80 < low95
OR
high80 > high95
)
""",
name='ci95_must_be_wider_than_ci80',
),
# There can be only one prediction per forecasting setting.
sa.UniqueConstraint(
'pixel_id', 'start_at', 'time_step', 'train_horizon', 'model',
),
)
# Relationships
pixel = orm.relationship('Pixel', back_populates='forecasts')
def __repr__(self) -> str:
"""Non-literal text representation."""
return '<{cls}: {prediction} for pixel ({n_x}|{n_y}) at {start_at}>'.format(
cls=self.__class__.__name__,
prediction=self.prediction,
n_x=self.pixel.n_x,
n_y=self.pixel.n_y,
start_at=self.start_at,
)
@classmethod
def from_dataframe( # noqa:WPS210,WPS211
cls,
pixel: db.Pixel,
time_step: int,
train_horizon: int,
model: str,
data: pd.Dataframe,
) -> List[db.Forecast]:
"""Convert results from the forecasting `*Model`s into `Forecast` objects.
This is an alternative constructor method.
Background: The functions in `urban_meal_delivery.forecasts.methods`
return `pd.Dataframe`s with "start_at" (i.e., `pd.Timestamp` objects)
values in the index and five columns "prediction", "low80", "high80",
"low95", and "high95" with `np.float` values. The `*Model.predict()`
methods in `urban_meal_delivery.forecasts.models` then add an "actual"
column. This constructor converts these results into ORM models.
Also, the `np.float` values are cast as plain `float` ones as
otherwise SQLAlchemy and the database would complain.
Args:
pixel: in which the forecast is made
time_step: length of one time step in minutes
train_horizon: length of the training horizon in weeks
model: name of the forecasting model
data: a `pd.Dataframe` as described above (i.e.,
with the six columns holding `float`s)
Returns:
forecasts: the `data` as `Forecast` objects
""" # noqa:RST215
forecasts = []
for timestamp_idx in data.index:
start_at = timestamp_idx.to_pydatetime()
actual = int(data.loc[timestamp_idx, 'actual'])
prediction = round(data.loc[timestamp_idx, 'prediction'], 5)
# Explicit type casting. SQLAlchemy does not convert
# `float('NaN')`s into plain `None`s.
low80 = data.loc[timestamp_idx, 'low80']
high80 = data.loc[timestamp_idx, 'high80']
low95 = data.loc[timestamp_idx, 'low95']
high95 = data.loc[timestamp_idx, 'high95']
if math.isnan(low80):
low80 = None
else:
low80 = round(low80, 5)
if math.isnan(high80):
high80 = None
else:
high80 = round(high80, 5)
if math.isnan(low95):
low95 = None
else:
low95 = round(low95, 5)
if math.isnan(high95):
high95 = None
else:
high95 = round(high95, 5)
forecasts.append(
cls(
pixel=pixel,
start_at=start_at,
time_step=time_step,
train_horizon=train_horizon,
model=model,
actual=actual,
prediction=prediction,
low80=low80,
high80=high80,
low95=low95,
high95=high95,
),
)
return forecasts
from urban_meal_delivery import db # noqa:E402 isort:skip

137
src/urban_meal_delivery/db/grids.py
View file

@ -1,137 +0,0 @@
"""Provide the ORM's `Grid` model."""
from __future__ import annotations
from typing import Any
import folium
import sqlalchemy as sa
from sqlalchemy import orm
from urban_meal_delivery import db
from urban_meal_delivery.db import meta
class Grid(meta.Base):
"""A grid of `Pixel`s to partition a `City`.
A grid is characterized by the uniform size of the `Pixel`s it contains.
That is configures via the `Grid.side_length` attribute.
"""
__tablename__ = 'grids'
# Columns
id = sa.Column( # noqa:WPS125
sa.SmallInteger, primary_key=True, autoincrement=True,
)
city_id = sa.Column(sa.SmallInteger, nullable=False)
side_length = sa.Column(sa.SmallInteger, nullable=False, unique=True)
# Constraints
__table_args__ = (
sa.ForeignKeyConstraint(
['city_id'], ['cities.id'], onupdate='RESTRICT', ondelete='RESTRICT',
),
# Each `Grid`, characterized by its `.side_length`,
# may only exists once for a given `.city`.
sa.UniqueConstraint('city_id', 'side_length'),
# Needed by a `ForeignKeyConstraint` in `address_pixel_association`.
sa.UniqueConstraint('id', 'city_id'),
)
# Relationships
city = orm.relationship('City', back_populates='grids')
pixels = orm.relationship('Pixel', back_populates='grid')
def __repr__(self) -> str:
"""Non-literal text representation."""
return '<{cls}: {area} sqr. km>'.format(
cls=self.__class__.__name__, area=self.pixel_area,
)
# Convenience properties
@property
def pixel_area(self) -> float:
"""The area of a `Pixel` on the grid in square kilometers."""
return round((self.side_length ** 2) / 1_000_000, 1)
@classmethod
def gridify(cls, city: db.City, side_length: int) -> db.Grid: # noqa:WPS210
"""Create a fully populated `Grid` for a `city`.
The `Grid` contains only `Pixel`s that have at least one
`Order.pickup_address`. `Address` objects outside the `.city`'s
viewport are discarded.
Args:
city: city for which the grid is created
side_length: the length of a square `Pixel`'s side
Returns:
grid: including `grid.pixels` with the associated `city.addresses`
"""
grid = cls(city=city, side_length=side_length)
# `Pixel`s grouped by `.n_x`-`.n_y` coordinates.
pixels = {}
pickup_addresses = (
db.session.query(db.Address)
.join(db.Order, db.Address.id == db.Order.pickup_address_id)
.filter(db.Address.city == city)
.all()
)
for address in pickup_addresses:
# Check if an `address` is not within the `city`'s viewport, ...
not_within_city_viewport = (
address.x < 0
or address.x > city.total_x
or address.y < 0
or address.y > city.total_y
)
# ... and, if so, the `address` does not belong to any `Pixel`.
if not_within_city_viewport:
continue
# Determine which `pixel` the `address` belongs to ...
n_x, n_y = address.x // side_length, address.y // side_length
# ... and create a new `Pixel` object if necessary.
if (n_x, n_y) not in pixels:
pixels[(n_x, n_y)] = db.Pixel(grid=grid, n_x=n_x, n_y=n_y)
pixel = pixels[(n_x, n_y)]
# Create an association between the `address` and `pixel`.
assoc = db.AddressPixelAssociation(address=address, pixel=pixel)
pixel.addresses.append(assoc)
return grid
def clear_map(self) -> Grid: # pragma: no cover
"""Shortcut to the `.city.clear_map()` method.
Returns:
self: enabling method chaining
""" # noqa:D402,DAR203
self.city.clear_map()
return self
@property # pragma: no cover
def map(self) -> folium.Map: # noqa:WPS125
"""Shortcut to the `.city.map` object."""
return self.city.map
def draw(self, **kwargs: Any) -> folium.Map: # pragma: no cover
"""Draw all pixels in the grid.
Args:
**kwargs: passed on to `Pixel.draw()`
Returns:
`.city.map` for convenience in interactive usage
"""
for pixel in self.pixels:
pixel.draw(**kwargs)
return self.map

22
src/urban_meal_delivery/db/meta.py
View file

@ -1,22 +0,0 @@
"""Provide the ORM's declarative base."""
from typing import Any
import sqlalchemy as sa
from sqlalchemy.ext import declarative
import urban_meal_delivery
Base: Any = declarative.declarative_base(
metadata=sa.MetaData(
schema=urban_meal_delivery.config.CLEAN_SCHEMA,
naming_convention={
'pk': 'pk_%(table_name)s', # noqa:WPS323
'fk': 'fk_%(table_name)s_to_%(referred_table_name)s_via_%(column_0_N_name)s', # noqa:E501,WPS323
'uq': 'uq_%(table_name)s_on_%(column_0_N_name)s', # noqa:WPS323
'ix': 'ix_%(table_name)s_on_%(column_0_N_name)s', # noqa:WPS323
'ck': 'ck_%(table_name)s_on_%(constraint_name)s', # noqa:WPS323
},
),
)

562
src/urban_meal_delivery/db/orders.py
View file

@ -1,562 +0,0 @@
"""Provide the ORM's `Order` model."""
import datetime
import folium
import sqlalchemy as sa
from sqlalchemy import orm
from sqlalchemy.dialects import postgresql
from urban_meal_delivery import config
from urban_meal_delivery import db
from urban_meal_delivery.db import meta
class Order(meta.Base): # noqa:WPS214
"""An order by a `Customer` of the UDP."""
__tablename__ = 'orders'
# Generic columns
id = sa.Column(sa.Integer, primary_key=True, autoincrement=False) # noqa:WPS125
_delivery_id = sa.Column('delivery_id', sa.Integer, index=True, unique=True)
customer_id = sa.Column(sa.Integer, nullable=False, index=True)
placed_at = sa.Column(sa.DateTime, nullable=False, index=True)
ad_hoc = sa.Column(sa.Boolean, nullable=False)
scheduled_delivery_at = sa.Column(sa.DateTime, index=True)
scheduled_delivery_at_corrected = sa.Column(sa.Boolean, index=True)
first_estimated_delivery_at = sa.Column(sa.DateTime)
cancelled = sa.Column(sa.Boolean, nullable=False, index=True)
cancelled_at = sa.Column(sa.DateTime)
cancelled_at_corrected = sa.Column(sa.Boolean, index=True)
# Price-related columns
sub_total = sa.Column(sa.Integer, nullable=False)
delivery_fee = sa.Column(sa.SmallInteger, nullable=False)
total = sa.Column(sa.Integer, nullable=False)
# Restaurant-related columns
restaurant_id = sa.Column(sa.SmallInteger, nullable=False, index=True)
restaurant_notified_at = sa.Column(sa.DateTime)
restaurant_notified_at_corrected = sa.Column(sa.Boolean, index=True)
restaurant_confirmed_at = sa.Column(sa.DateTime)
restaurant_confirmed_at_corrected = sa.Column(sa.Boolean, index=True)
estimated_prep_duration = sa.Column(sa.Integer, index=True)
estimated_prep_duration_corrected = sa.Column(sa.Boolean, index=True)
estimated_prep_buffer = sa.Column(sa.Integer, nullable=False, index=True)
# Dispatch-related columns
courier_id = sa.Column(sa.Integer, index=True)
dispatch_at = sa.Column(sa.DateTime)
dispatch_at_corrected = sa.Column(sa.Boolean, index=True)
courier_notified_at = sa.Column(sa.DateTime)
courier_notified_at_corrected = sa.Column(sa.Boolean, index=True)
courier_accepted_at = sa.Column(sa.DateTime)
courier_accepted_at_corrected = sa.Column(sa.Boolean, index=True)
utilization = sa.Column(sa.SmallInteger, nullable=False)
# Pickup-related columns
pickup_address_id = sa.Column(sa.Integer, nullable=False, index=True)
reached_pickup_at = sa.Column(sa.DateTime)
pickup_at = sa.Column(sa.DateTime)
pickup_at_corrected = sa.Column(sa.Boolean, index=True)
pickup_not_confirmed = sa.Column(sa.Boolean)
left_pickup_at = sa.Column(sa.DateTime)
left_pickup_at_corrected = sa.Column(sa.Boolean, index=True)
# Delivery-related columns
delivery_address_id = sa.Column(sa.Integer, nullable=False, index=True)
reached_delivery_at = sa.Column(sa.DateTime)
delivery_at = sa.Column(sa.DateTime)
delivery_at_corrected = sa.Column(sa.Boolean, index=True)
delivery_not_confirmed = sa.Column(sa.Boolean)
_courier_waited_at_delivery = sa.Column('courier_waited_at_delivery', sa.Boolean)
# Statistical columns
logged_delivery_distance = sa.Column(sa.SmallInteger, nullable=True)
logged_avg_speed = sa.Column(postgresql.DOUBLE_PRECISION, nullable=True)
logged_avg_speed_distance = sa.Column(sa.SmallInteger, nullable=True)
# Constraints
__table_args__ = (
sa.ForeignKeyConstraint(
['customer_id'], ['customers.id'], onupdate='RESTRICT', ondelete='RESTRICT',
),
sa.ForeignKeyConstraint(
['courier_id'], ['couriers.id'], onupdate='RESTRICT', ondelete='RESTRICT',
),
sa.ForeignKeyConstraint(
['pickup_address_id'],
['addresses.id'],
onupdate='RESTRICT',
ondelete='RESTRICT',
),
sa.ForeignKeyConstraint(
# This foreign key ensures that there is only
# one `.pickup_address` per `.restaurant`
['restaurant_id', 'pickup_address_id'],
['restaurants.id', 'restaurants.address_id'],
onupdate='RESTRICT',
ondelete='RESTRICT',
),
sa.ForeignKeyConstraint(
['delivery_address_id'],
['addresses.id'],
onupdate='RESTRICT',
ondelete='RESTRICT',
),
sa.CheckConstraint(
"""
(ad_hoc IS TRUE AND scheduled_delivery_at IS NULL)
OR
(ad_hoc IS FALSE AND scheduled_delivery_at IS NOT NULL)
""",
name='either_ad_hoc_or_scheduled_order',
),
sa.CheckConstraint(
"""
NOT (
ad_hoc IS TRUE
AND (
EXTRACT(HOUR FROM placed_at) < 11
OR
EXTRACT(HOUR FROM placed_at) > 22
)
)
""",
name='ad_hoc_orders_within_business_hours',
),
sa.CheckConstraint(
"""
NOT (
ad_hoc IS FALSE
AND (
(
EXTRACT(HOUR FROM scheduled_delivery_at) <= 11
AND
NOT (
EXTRACT(HOUR FROM scheduled_delivery_at) = 11
AND
EXTRACT(MINUTE FROM scheduled_delivery_at) = 45
)
)
OR
EXTRACT(HOUR FROM scheduled_delivery_at) > 22
)
)
""",
name='scheduled_orders_within_business_hours',
),
sa.CheckConstraint(
"""
NOT (
EXTRACT(EPOCH FROM scheduled_delivery_at - placed_at) < 1800
)
""",
name='scheduled_orders_not_within_30_minutes',
),
sa.CheckConstraint(
"""
NOT (
cancelled IS FALSE
AND
cancelled_at IS NOT NULL
)
""",
name='only_cancelled_orders_may_have_cancelled_at',
),
sa.CheckConstraint(
"""
NOT (
cancelled IS TRUE
AND
delivery_at IS NOT NULL
)
""",
name='cancelled_orders_must_not_be_delivered',
),
sa.CheckConstraint(
'0 <= estimated_prep_duration AND estimated_prep_duration <= 2700',
name='estimated_prep_duration_between_0_and_2700',
),
sa.CheckConstraint(
'estimated_prep_duration % 60 = 0',
name='estimated_prep_duration_must_be_whole_minutes',
),
sa.CheckConstraint(
'0 <= estimated_prep_buffer AND estimated_prep_buffer <= 900',
name='estimated_prep_buffer_between_0_and_900',
),
sa.CheckConstraint(
'estimated_prep_buffer % 60 = 0',
name='estimated_prep_buffer_must_be_whole_minutes',
),
sa.CheckConstraint(
'0 <= utilization AND utilization <= 100',
name='utilization_between_0_and_100',
),
*(
sa.CheckConstraint(
f"""
({column} IS NULL AND {column}_corrected IS NULL)
OR
({column} IS NULL AND {column}_corrected IS TRUE)
OR
({column} IS NOT NULL AND {column}_corrected IS NOT NULL)
""",
name=f'corrections_only_for_set_value_{index}',
)
for index, column in enumerate(
(
'scheduled_delivery_at',
'cancelled_at',
'restaurant_notified_at',
'restaurant_confirmed_at',
'estimated_prep_duration',
'dispatch_at',
'courier_notified_at',
'courier_accepted_at',
'pickup_at',
'left_pickup_at',
'delivery_at',
),
)
),
*(
sa.CheckConstraint(
f"""
({event}_at IS NULL AND {event}_not_confirmed IS NULL)
OR
({event}_at IS NOT NULL AND {event}_not_confirmed IS NOT NULL)
""",
name=f'{event}_not_confirmed_only_if_{event}',
)
for event in ('pickup', 'delivery')
),
sa.CheckConstraint(
"""
(delivery_at IS NULL AND courier_waited_at_delivery IS NULL)
OR
(delivery_at IS NOT NULL AND courier_waited_at_delivery IS NOT NULL)
""",
name='courier_waited_at_delivery_only_if_delivery',
),
*(
sa.CheckConstraint(
constraint, name='ordered_timestamps_{index}'.format(index=index),
)
for index, constraint in enumerate(
(
'placed_at < scheduled_delivery_at',
'placed_at < first_estimated_delivery_at',
'placed_at < cancelled_at',
'placed_at < restaurant_notified_at',
'placed_at < restaurant_confirmed_at',
'placed_at < dispatch_at',
'placed_at < courier_notified_at',
'placed_at < courier_accepted_at',
'placed_at < reached_pickup_at',
'placed_at < left_pickup_at',
'placed_at < reached_delivery_at',
'placed_at < delivery_at',
'cancelled_at > restaurant_notified_at',
'cancelled_at > restaurant_confirmed_at',
'cancelled_at > dispatch_at',
'cancelled_at > courier_notified_at',
'cancelled_at > courier_accepted_at',
'cancelled_at > reached_pickup_at',
'cancelled_at > pickup_at',
'cancelled_at > left_pickup_at',
'cancelled_at > reached_delivery_at',
'cancelled_at > delivery_at',
'restaurant_notified_at < restaurant_confirmed_at',
'restaurant_notified_at < pickup_at',
'restaurant_confirmed_at < pickup_at',
'dispatch_at < courier_notified_at',
'dispatch_at < courier_accepted_at',
'dispatch_at < reached_pickup_at',
'dispatch_at < pickup_at',
'dispatch_at < left_pickup_at',
'dispatch_at < reached_delivery_at',
'dispatch_at < delivery_at',
'courier_notified_at < courier_accepted_at',
'courier_notified_at < reached_pickup_at',
'courier_notified_at < pickup_at',
'courier_notified_at < left_pickup_at',
'courier_notified_at < reached_delivery_at',
'courier_notified_at < delivery_at',
'courier_accepted_at < reached_pickup_at',
'courier_accepted_at < pickup_at',
'courier_accepted_at < left_pickup_at',
'courier_accepted_at < reached_delivery_at',
'courier_accepted_at < delivery_at',
'reached_pickup_at < pickup_at',
'reached_pickup_at < left_pickup_at',
'reached_pickup_at < reached_delivery_at',
'reached_pickup_at < delivery_at',
'pickup_at < left_pickup_at',
'pickup_at < reached_delivery_at',
'pickup_at < delivery_at',
'left_pickup_at < reached_delivery_at',
'left_pickup_at < delivery_at',
'reached_delivery_at < delivery_at',
),
)
),
)
# Relationships
customer = orm.relationship('Customer', back_populates='orders')
restaurant = orm.relationship(
'Restaurant',
back_populates='orders',
primaryjoin='Restaurant.id == Order.restaurant_id',
)
courier = orm.relationship('Courier', back_populates='orders')
pickup_address = orm.relationship(
'Address',
back_populates='orders_picked_up',
foreign_keys='[Order.pickup_address_id]',
)
delivery_address = orm.relationship(
'Address',
back_populates='orders_delivered',
foreign_keys='[Order.delivery_address_id]',
)
# Convenience properties
@property
def scheduled(self) -> bool:
"""Inverse of `.ad_hoc`."""
return not self.ad_hoc
@property
def completed(self) -> bool:
"""Inverse of `.cancelled`."""
return not self.cancelled
@property
def corrected(self) -> bool:
"""If any timestamp was corrected as compared to the original data."""
return (
self.scheduled_delivery_at_corrected # noqa:WPS222 => too much logic
or self.cancelled_at_corrected
or self.restaurant_notified_at_corrected
or self.restaurant_confirmed_at_corrected
or self.dispatch_at_corrected
or self.courier_notified_at_corrected
or self.courier_accepted_at_corrected
or self.pickup_at_corrected
or self.left_pickup_at_corrected
or self.delivery_at_corrected
)
# Timing-related properties
@property
def time_to_accept(self) -> datetime.timedelta:
"""Time until the `.courier` accepted the order.
This measures the time it took the UDP to notify the `.courier` after dispatch.
"""
if not self.dispatch_at:
raise RuntimeError('dispatch_at is not set')
if not self.courier_accepted_at:
raise RuntimeError('courier_accepted_at is not set')
return self.courier_accepted_at - self.dispatch_at
@property
def time_to_react(self) -> datetime.timedelta:
"""Time the `.courier` took to accept an order.
A subset of `.time_to_accept`.
"""
if not self.courier_notified_at:
raise RuntimeError('courier_notified_at is not set')
if not self.courier_accepted_at:
raise RuntimeError('courier_accepted_at is not set')
return self.courier_accepted_at - self.courier_notified_at
@property
def time_to_pickup(self) -> datetime.timedelta:
"""Time from the `.courier`'s acceptance to arrival at `.pickup_address`."""
if not self.courier_accepted_at:
raise RuntimeError('courier_accepted_at is not set')
if not self.reached_pickup_at:
raise RuntimeError('reached_pickup_at is not set')
return self.reached_pickup_at - self.courier_accepted_at
@property
def time_at_pickup(self) -> datetime.timedelta:
"""Time the `.courier` stayed at the `.pickup_address`."""
if not self.reached_pickup_at:
raise RuntimeError('reached_pickup_at is not set')
if not self.pickup_at:
raise RuntimeError('pickup_at is not set')
return self.pickup_at - self.reached_pickup_at
@property
def scheduled_pickup_at(self) -> datetime.datetime:
"""Point in time at which the pickup was scheduled."""
if not self.restaurant_notified_at:
raise RuntimeError('restaurant_notified_at is not set')
if not self.estimated_prep_duration:
raise RuntimeError('estimated_prep_duration is not set')
delta = datetime.timedelta(seconds=self.estimated_prep_duration)
return self.restaurant_notified_at + delta
@property
def courier_early(self) -> datetime.timedelta:
"""Time by which the `.courier` is early for pickup.
Measured relative to `.scheduled_pickup_at`.
`datetime.timedelta(seconds=0)` if the `.courier` is on time or late.
Goes together with `.courier_late`.
"""
return max(
datetime.timedelta(), self.scheduled_pickup_at - self.reached_pickup_at,
)
@property
def courier_late(self) -> datetime.timedelta:
"""Time by which the `.courier` is late for pickup.
Measured relative to `.scheduled_pickup_at`.
`datetime.timedelta(seconds=0)` if the `.courier` is on time or early.
Goes together with `.courier_early`.
"""
return max(
datetime.timedelta(), self.reached_pickup_at - self.scheduled_pickup_at,
)
@property
def restaurant_early(self) -> datetime.timedelta:
"""Time by which the `.restaurant` is early for pickup.
Measured relative to `.scheduled_pickup_at`.
`datetime.timedelta(seconds=0)` if the `.restaurant` is on time or late.
Goes together with `.restaurant_late`.
"""
return max(datetime.timedelta(), self.scheduled_pickup_at - self.pickup_at)
@property
def restaurant_late(self) -> datetime.timedelta:
"""Time by which the `.restaurant` is late for pickup.
Measured relative to `.scheduled_pickup_at`.
`datetime.timedelta(seconds=0)` if the `.restaurant` is on time or early.
Goes together with `.restaurant_early`.
"""
return max(datetime.timedelta(), self.pickup_at - self.scheduled_pickup_at)
@property
def time_to_delivery(self) -> datetime.timedelta:
"""Time the `.courier` took from `.pickup_address` to `.delivery_address`."""
if not self.pickup_at:
raise RuntimeError('pickup_at is not set')
if not self.reached_delivery_at:
raise RuntimeError('reached_delivery_at is not set')
return self.reached_delivery_at - self.pickup_at
@property
def time_at_delivery(self) -> datetime.timedelta:
"""Time the `.courier` stayed at the `.delivery_address`."""
if not self.reached_delivery_at:
raise RuntimeError('reached_delivery_at is not set')
if not self.delivery_at:
raise RuntimeError('delivery_at is not set')
return self.delivery_at - self.reached_delivery_at
@property
def courier_waited_at_delivery(self) -> datetime.timedelta:
"""Time the `.courier` waited at the `.delivery_address`."""
if self._courier_waited_at_delivery:
return self.time_at_delivery
return datetime.timedelta()
@property
def delivery_early(self) -> datetime.timedelta:
"""Time by which a `.scheduled` order was early.
Measured relative to `.scheduled_delivery_at`.
`datetime.timedelta(seconds=0)` if the delivery is on time or late.
Goes together with `.delivery_late`.
"""
if not self.scheduled:
raise AttributeError('Makes sense only for scheduled orders')
return max(datetime.timedelta(), self.scheduled_delivery_at - self.delivery_at)
@property
def delivery_late(self) -> datetime.timedelta:
"""Time by which a `.scheduled` order was late.
Measured relative to `.scheduled_delivery_at`.
`datetime.timedelta(seconds=0)` if the delivery is on time or early.
Goes together with `.delivery_early`.
"""
if not self.scheduled:
raise AttributeError('Makes sense only for scheduled orders')
return max(datetime.timedelta(), self.delivery_at - self.scheduled_delivery_at)
@property
def total_time(self) -> datetime.timedelta:
"""Time from order placement to delivery for an `.ad_hoc` order."""
if self.scheduled:
raise AttributeError('Scheduled orders have no total_time')
if self.cancelled:
raise RuntimeError('Cancelled orders have no total_time')
return self.delivery_at - self.placed_at
# Other Methods
def __repr__(self) -> str:
"""Non-literal text representation."""
return '<{cls}(#{order_id})>'.format(
cls=self.__class__.__name__, order_id=self.id,
)
def draw(self) -> folium.Map: # pragma: no cover
"""Draw the `.waypoints` from `.pickup_address` to `.delivery_address`.
Important: Do not put this in an automated script as a method call
triggers an API call to the Google Maps API and may result in costs.
Returns:
`...city.map` for convenience in interactive usage
"""
path = db.Path.from_order(self)
restaurant_tooltip = f'{self.restaurant.name} (#{self.restaurant.id})'
customer_tooltip = f'Customer #{self.customer.id}'
# Because the underlying distance matrix is symmetric (i.e., a DB constraint),
# we must check if the `.pickup_address` is the couriers' `Path`'s start.
if path.first_address is self.pickup_address:
reverse = False
start_tooltip, end_tooltip = restaurant_tooltip, customer_tooltip
else:
reverse = True
start_tooltip, end_tooltip = customer_tooltip, restaurant_tooltip
# This triggers `Path.sync_with_google_maps()` behind the scenes.
return path.draw(
reverse=reverse,
start_tooltip=start_tooltip,
end_tooltip=end_tooltip,
start_color=config.RESTAURANT_COLOR,
end_color=config.CUSTOMER_COLOR,
path_color=config.NEUTRAL_COLOR,
)

254
src/urban_meal_delivery/db/pixels.py
View file

@ -1,254 +0,0 @@
"""Provide the ORM's `Pixel` model."""
from __future__ import annotations
import functools
from typing import List
import folium
import sqlalchemy as sa
import utm
from sqlalchemy import orm
from urban_meal_delivery import config
from urban_meal_delivery import db
from urban_meal_delivery.db import meta
from urban_meal_delivery.db import utils
class Pixel(meta.Base):
"""A pixel in a `Grid`.
Square pixels aggregate `Address` objects within a `City`.
Every `Address` belongs to exactly one `Pixel` in a `Grid`.
Every `Pixel` has a unique `n_x`-`n_y` coordinate within the `Grid`.
"""
__tablename__ = 'pixels'
# Columns
id = sa.Column(sa.Integer, primary_key=True, autoincrement=True) # noqa:WPS125
grid_id = sa.Column(sa.SmallInteger, nullable=False, index=True)
n_x = sa.Column(sa.SmallInteger, nullable=False, index=True)
n_y = sa.Column(sa.SmallInteger, nullable=False, index=True)
# Constraints
__table_args__ = (
sa.ForeignKeyConstraint(
['grid_id'], ['grids.id'], onupdate='RESTRICT', ondelete='RESTRICT',
),
sa.CheckConstraint('0 <= n_x', name='n_x_is_positive'),
sa.CheckConstraint('0 <= n_y', name='n_y_is_positive'),
# Needed by a `ForeignKeyConstraint` in `AddressPixelAssociation`.
sa.UniqueConstraint('id', 'grid_id'),
# Each coordinate within the same `grid` is used at most once.
sa.UniqueConstraint('grid_id', 'n_x', 'n_y'),
)
# Relationships
grid = orm.relationship('Grid', back_populates='pixels')
addresses = orm.relationship('AddressPixelAssociation', back_populates='pixel')
forecasts = orm.relationship('Forecast', back_populates='pixel')
def __repr__(self) -> str:
"""Non-literal text representation."""
return '<{cls}: ({x}|{y})>'.format(
cls=self.__class__.__name__, x=self.n_x, y=self.n_y,
)
# Convenience properties
@property
def side_length(self) -> int:
"""The length of one side of a pixel in meters."""
return self.grid.side_length
@property
def area(self) -> float:
"""The area of a pixel in square kilometers."""
return self.grid.pixel_area
@functools.cached_property
def northeast(self) -> utils.Location:
"""The pixel's northeast corner, relative to `.grid.city.southwest`.
Implementation detail: This property is cached as none of the
underlying attributes to calculate the value are to be changed.
"""
easting, northing = (
self.grid.city.southwest.easting + ((self.n_x + 1) * self.side_length),
self.grid.city.southwest.northing + ((self.n_y + 1) * self.side_length),
)
latitude, longitude = utm.to_latlon(
easting, northing, *self.grid.city.southwest.zone_details,
)
location = utils.Location(latitude, longitude)
location.relate_to(self.grid.city.southwest)
return location
@functools.cached_property
def southwest(self) -> utils.Location:
"""The pixel's southwest corner, relative to `.grid.city.southwest`.
Implementation detail: This property is cached as none of the
underlying attributes to calculate the value are to be changed.
"""
easting, northing = (
self.grid.city.southwest.easting + (self.n_x * self.side_length),
self.grid.city.southwest.northing + (self.n_y * self.side_length),
)
latitude, longitude = utm.to_latlon(
easting, northing, *self.grid.city.southwest.zone_details,
)
location = utils.Location(latitude, longitude)
location.relate_to(self.grid.city.southwest)
return location
@functools.cached_property
def restaurants(self) -> List[db.Restaurant]: # pragma: no cover
"""Obtain all `Restaurant`s in `self`."""
return (
db.session.query(db.Restaurant)
.join(
db.AddressPixelAssociation,
db.Restaurant.address_id == db.AddressPixelAssociation.address_id,
)
.filter(db.AddressPixelAssociation.pixel_id == self.id)
.all()
)
def clear_map(self) -> Pixel: # pragma: no cover
"""Shortcut to the `.city.clear_map()` method.
Returns:
self: enabling method chaining
""" # noqa:D402,DAR203
self.grid.city.clear_map()
return self
@property # pragma: no cover
def map(self) -> folium.Map: # noqa:WPS125
"""Shortcut to the `.city.map` object."""
return self.grid.city.map
def draw( # noqa:C901,WPS210,WPS231
self, restaurants: bool = True, order_counts: bool = False, # pragma: no cover
) -> folium.Map:
"""Draw the pixel on the `.grid.city.map`.
Args:
restaurants: include the restaurants
order_counts: show the number of orders at a restaurant
Returns:
`.grid.city.map` for convenience in interactive usage
"""
bounds = (
(self.southwest.latitude, self.southwest.longitude),
(self.northeast.latitude, self.northeast.longitude),
)
info_text = f'Pixel({self.n_x}|{self.n_y})'
# Make the `Pixel`s look like a checkerboard.
if (self.n_x + self.n_y) % 2:
color = '#808000'
else:
color = '#ff8c00'
marker = folium.Rectangle(
bounds=bounds,
color='gray',
opacity=0.2,
weight=5,
fill_color=color,
fill_opacity=0.2,
popup=info_text,
tooltip=info_text,
)
marker.add_to(self.grid.city.map)
if restaurants:
# Obtain all primary `Address`es in the city that host `Restaurant`s
# and are in the `self` `Pixel`.
addresses = (
db.session.query(db.Address)
.filter(
db.Address.id.in_(
(
db.session.query(db.Address.primary_id)
.join(
db.Restaurant,
db.Address.id == db.Restaurant.address_id,
)
.join(
db.AddressPixelAssociation,
db.Address.id == db.AddressPixelAssociation.address_id,
)
.filter(db.AddressPixelAssociation.pixel_id == self.id)
)
.distinct()
.all(),
),
)
.all()
)
for address in addresses:
# Show the restaurant's name if there is only one.
# Otherwise, list all the restaurants' ID's.
restaurants = (
db.session.query(db.Restaurant)
.join(db.Address, db.Restaurant.address_id == db.Address.id)
.filter(db.Address.primary_id == address.id)
.all()
)
if len(restaurants) == 1: # type:ignore
tooltip = (
f'{restaurants[0].name} (#{restaurants[0].id})' # type:ignore
)
else:
tooltip = 'Restaurants ' + ', '.join( # noqa:WPS336
f'#{restaurant.id}' for restaurant in restaurants # type:ignore
)
if order_counts:
# Calculate the number of orders for ALL restaurants ...
n_orders = (
db.session.query(db.Order.id)
.join(db.Address, db.Order.pickup_address_id == db.Address.id)
.filter(db.Address.primary_id == address.id)
.count()
)
# ... and adjust the size of the red dot on the `.map`.
if n_orders >= 1000:
radius = 20 # noqa:WPS220
elif n_orders >= 500:
radius = 15 # noqa:WPS220
elif n_orders >= 100:
radius = 10 # noqa:WPS220
elif n_orders >= 10:
radius = 5 # noqa:WPS220
else:
radius = 1 # noqa:WPS220
tooltip += f' | n_orders={n_orders}' # noqa:WPS336
address.draw(
radius=radius,
color=config.RESTAURANT_COLOR,
fill_color=config.RESTAURANT_COLOR,
fill_opacity=0.3,
tooltip=tooltip,
)
else:
address.draw(
radius=1, color=config.RESTAURANT_COLOR, tooltip=tooltip,
)
return self.map

150
src/urban_meal_delivery/db/restaurants.py
View file

@ -1,150 +0,0 @@
"""Provide the ORM's `Restaurant` model."""
from __future__ import annotations
import folium
import sqlalchemy as sa
from sqlalchemy import orm
from urban_meal_delivery import config
from urban_meal_delivery import db
from urban_meal_delivery.db import meta
class Restaurant(meta.Base):
"""A restaurant selling meals on the UDP.
In the historic dataset, a `Restaurant` may have changed its `Address`
throughout its life time. The ORM model only stores the current one,
which in most cases is also the only one.
"""
__tablename__ = 'restaurants'
# Columns
id = sa.Column( # noqa:WPS125
sa.SmallInteger, primary_key=True, autoincrement=False,
)
created_at = sa.Column(sa.DateTime, nullable=False)
name = sa.Column(sa.Unicode(length=45), nullable=False)
address_id = sa.Column(sa.Integer, nullable=False, index=True)
estimated_prep_duration = sa.Column(sa.SmallInteger, nullable=False)
# Constraints
__table_args__ = (
sa.ForeignKeyConstraint(
['address_id'], ['addresses.id'], onupdate='RESTRICT', ondelete='RESTRICT',
),
sa.CheckConstraint(
'0 <= estimated_prep_duration AND estimated_prep_duration <= 2400',
name='realistic_estimated_prep_duration',
),
# Needed by a `ForeignKeyConstraint` in `Order`.
sa.UniqueConstraint('id', 'address_id'),
)
# Relationships
address = orm.relationship('Address', back_populates='restaurants')
orders = orm.relationship(
'Order',
back_populates='restaurant',
overlaps='orders_picked_up,pickup_address',
)
def __repr__(self) -> str:
"""Non-literal text representation."""
return '<{cls}({name})>'.format(cls=self.__class__.__name__, name=self.name)
def clear_map(self) -> Restaurant: # pragma: no cover
"""Shortcut to the `.address.city.clear_map()` method.
Returns:
self: enabling method chaining
""" # noqa:D402,DAR203
self.address.city.clear_map()
return self
@property # pragma: no cover
def map(self) -> folium.Map: # noqa:WPS125
"""Shortcut to the `.address.city.map` object."""
return self.address.city.map
def draw( # noqa:WPS231
self, customers: bool = True, order_counts: bool = False, # pragma: no cover
) -> folium.Map:
"""Draw the restaurant on the `.address.city.map`.
By default, the restaurant's delivery locations are also shown.
Args:
customers: show the restaurant's delivery locations
order_counts: show the number of orders at the delivery locations;
only useful if `customers=True`
Returns:
`.address.city.map` for convenience in interactive usage
"""
if customers:
# Obtain all primary `Address`es in the city that
# received at least one delivery from `self`.
delivery_addresses = (
db.session.query(db.Address)
.filter(
db.Address.id.in_(
row.primary_id
for row in (
db.session.query(db.Address.primary_id) # noqa:WPS221
.join(
db.Order, db.Address.id == db.Order.delivery_address_id,
)
.filter(db.Order.restaurant_id == self.id)
.distinct()
.all()
)
),
)
.all()
)
for address in delivery_addresses:
if order_counts:
n_orders = (
db.session.query(db.Order)
.join(db.Address, db.Order.delivery_address_id == db.Address.id)
.filter(db.Order.restaurant_id == self.id)
.filter(db.Address.primary_id == address.id)
.count()
)
if n_orders >= 25:
radius = 20 # noqa:WPS220
elif n_orders >= 10:
radius = 15 # noqa:WPS220
elif n_orders >= 5:
radius = 10 # noqa:WPS220
elif n_orders > 1:
radius = 5 # noqa:WPS220
else:
radius = 1 # noqa:WPS220
address.draw(
radius=radius,
color=config.CUSTOMER_COLOR,
fill_color=config.CUSTOMER_COLOR,
fill_opacity=0.3,
tooltip=f'n_orders={n_orders}',
)
else:
address.draw(
radius=1, color=config.CUSTOMER_COLOR,
)
self.address.draw(
radius=20,
color=config.RESTAURANT_COLOR,
fill_color=config.RESTAURANT_COLOR,
fill_opacity=0.3,
tooltip=f'{self.name} (#{self.id}) | n_orders={len(self.orders)}',
)
return self.map

5
src/urban_meal_delivery/db/utils/__init__.py
View file

@ -1,5 +0,0 @@
"""Utilities used by the ORM models."""
from urban_meal_delivery.db.utils.colors import make_random_cmap
from urban_meal_delivery.db.utils.colors import rgb_to_hex
from urban_meal_delivery.db.utils.locations import Location

69
src/urban_meal_delivery/db/utils/colors.py
View file

@ -1,69 +0,0 @@
"""Utilities for drawing maps with `folium`."""
import colorsys
import numpy as np
from matplotlib import colors
def make_random_cmap(
n_colors: int, bright: bool = True, # pragma: no cover
) -> colors.LinearSegmentedColormap:
"""Create a random `Colormap` with `n_colors` different colors.
Args:
n_colors: number of of different colors; size of `Colormap`
bright: `True` for strong colors, `False` for pastel colors
Returns:
colormap
"""
np.random.seed(42)
if bright:
hsv_colors = [
(
np.random.uniform(low=0.0, high=1),
np.random.uniform(low=0.2, high=1),
np.random.uniform(low=0.9, high=1),
)
for _ in range(n_colors)
]
rgb_colors = []
for color in hsv_colors:
rgb_colors.append(colorsys.hsv_to_rgb(*color))
else:
low = 0.0
high = 0.66
rgb_colors = [
(
np.random.uniform(low=low, high=high),
np.random.uniform(low=low, high=high),
np.random.uniform(low=low, high=high),
)
for _ in range(n_colors)
]
return colors.LinearSegmentedColormap.from_list(
'random_color_map', rgb_colors, N=n_colors,
)
def rgb_to_hex(*args: float) -> str: # pragma: no cover
"""Convert RGB colors into hexadecimal notation.
Args:
*args: percentages (0% - 100%) for the RGB channels
Returns:
hexadecimal_representation
"""
red, green, blue = (
int(255 * args[0]),
int(255 * args[1]),
int(255 * args[2]),
)
return f'#{red:02x}{green:02x}{blue:02x}' # noqa:WPS221

147
src/urban_meal_delivery/db/utils/locations.py
View file

@ -1,147 +0,0 @@
"""A `Location` class to unify working with coordinates."""
from __future__ import annotations
from typing import Optional, Tuple
import utm
class Location: # noqa:WPS214
"""A location represented in WGS84 and UTM coordinates.
WGS84:
- "conventional" system with latitude-longitude pairs
- assumes earth is a sphere and models the location in 3D
UTM:
- the Universal Transverse Mercator system
- projects WGS84 coordinates onto a 2D map
- can be used for visualizations and calculations directly
- distances are in meters
Further info how WGS84 and UTM are related:
https://en.wikipedia.org/wiki/Universal_Transverse_Mercator_coordinate_system
"""
def __init__(self, latitude: float, longitude: float) -> None:
"""Create a location from a WGS84-conforming `latitude`-`longitude` pair."""
# The SQLAlchemy columns come as `Decimal`s due to the `DOUBLE_PRECISION`.
self._latitude = float(latitude)
self._longitude = float(longitude)
easting, northing, zone, band = utm.from_latlon(self._latitude, self._longitude)
# `.easting` and `.northing` as `int`s are precise enough.
self._easting = int(easting)
self._northing = int(northing)
self._zone = zone
self._band = band.upper()
self._normalized_easting: Optional[int] = None
self._normalized_northing: Optional[int] = None
def __repr__(self) -> str:
"""A non-literal text representation in the UTM system.
Convention is {ZONE} {EASTING} {NORTHING}.
Example:
`<Location: 17T 630084 4833438>'`
"""
return f'<Location: {self.zone} {self.easting} {self.northing}>' # noqa:WPS221
@property
def latitude(self) -> float:
"""The latitude of the location in degrees (WGS84).
Between -90 and +90 degrees.
"""
return self._latitude
@property
def longitude(self) -> float:
"""The longitude of the location in degrees (WGS84).
Between -180 and +180 degrees.
"""
return self._longitude
@property
def lat_lng(self) -> Tuple[float, float]:
"""The `.latitude` and `.longitude` as a 2-`tuple`."""
return self._latitude, self._longitude
@property
def easting(self) -> int:
"""The easting of the location in meters (UTM)."""
return self._easting
@property
def northing(self) -> int:
"""The northing of the location in meters (UTM)."""
return self._northing
@property
def zone(self) -> str:
"""The UTM zone of the location."""
return f'{self._zone}{self._band}'
@property
def zone_details(self) -> Tuple[int, str]:
"""The UTM zone of the location as the zone number and the band."""
return self._zone, self._band
def __eq__(self, other: object) -> bool:
"""Check if two `Location` objects are the same location."""
if not isinstance(other, Location):
return NotImplemented
if self.zone != other.zone:
raise ValueError('locations must be in the same zone, including the band')
return (self.easting, self.northing) == (other.easting, other.northing)
@property
def x(self) -> int: # noqa:WPS111
"""The `.easting` of the location in meters, relative to some origin.
The origin, which defines the `(0, 0)` coordinate, is set with `.relate_to()`.
"""
if self._normalized_easting is None:
raise RuntimeError('an origin to relate to must be set first')
return self._normalized_easting
@property
def y(self) -> int: # noqa:WPS111
"""The `.northing` of the location in meters, relative to some origin.
The origin, which defines the `(0, 0)` coordinate, is set with `.relate_to()`.
"""
if self._normalized_northing is None:
raise RuntimeError('an origin to relate to must be set first')
return self._normalized_northing
def relate_to(self, other: Location) -> None:
"""Make the origin in the lower-left corner relative to `other`.
The `.x` and `.y` properties are the `.easting` and `.northing` values
of `self` minus the ones from `other`. So, `.x` and `.y` make up a
Cartesian coordinate system where the `other` origin is `(0, 0)`.
To prevent semantic errors in calculations based on the `.x` and `.y`
properties, the `other` origin may only be set once!
"""
if self._normalized_easting is not None:
raise RuntimeError('the `other` origin may only be set once')
if not isinstance(other, Location):
raise TypeError('`other` is not a `Location` object')
if self.zone != other.zone:
raise ValueError('`other` must be in the same zone, including the band')
self._normalized_easting = self.easting - other.easting
self._normalized_northing = self.northing - other.northing

29
src/urban_meal_delivery/forecasts/__init__.py
View file

@ -1,29 +0,0 @@
"""Demand forecasting utilities.
This sub-package is divided into further sub-packages and modules as follows:
`methods` contains various time series related statistical methods, implemented
as plain `function` objects that are used to predict into the future given a
time series of historic order counts. The methods are context-agnostic, meaning
that they only take and return `pd.Series/DataFrame`s holding numbers and
are not concerned with how these numbers were generated or what they mean.
Some functions, like `arima.predict()` or `ets.predict()` wrap functions called
in R using the `rpy2` library. Others, like `extrapolate_season.predict()`, are
written in plain Python.
`timify` defines an `OrderHistory` class that abstracts away the communication
with the database and provides `pd.Series` objects with the order counts that
are fed into the `methods`. In particular, it uses SQL statements behind the
scenes to calculate the historic order counts on a per-`Pixel` level. Once the
data is loaded from the database, an `OrderHistory` instance provides various
ways to slice out, or generate, different kinds of order time series (e.g.,
"horizontal" vs. "vertical" time series).
`models` defines various forecasting `*Model`s that combine a given kind of
time series with one of the forecasting `methods`. For example, the ETS method
applied to a horizontal time series is implemented in the `HorizontalETSModel`.
"""
from urban_meal_delivery.forecasts import methods
from urban_meal_delivery.forecasts import models
from urban_meal_delivery.forecasts import timify

6
src/urban_meal_delivery/forecasts/methods/__init__.py
View file

@ -1,6 +0,0 @@
"""Various forecasting methods implemented as functions."""
from urban_meal_delivery.forecasts.methods import arima
from urban_meal_delivery.forecasts.methods import decomposition
from urban_meal_delivery.forecasts.methods import ets
from urban_meal_delivery.forecasts.methods import extrapolate_season

76
src/urban_meal_delivery/forecasts/methods/arima.py
View file

@ -1,76 +0,0 @@
"""A wrapper around R's "auto.arima" function."""
import pandas as pd
from rpy2 import robjects
from rpy2.robjects import pandas2ri
def predict(
training_ts: pd.Series,
forecast_interval: pd.DatetimeIndex,
*,
frequency: int,
seasonal_fit: bool = False,
) -> pd.DataFrame:
"""Predict with an automatically chosen ARIMA model.
Note: The function does not check if the `forecast_interval`
extends the `training_ts`'s interval without a gap!
Args:
training_ts: past observations to be fitted
forecast_interval: interval into which the `training_ts` is forecast;
its length becomes the step size `h` in the forecasting model in R
frequency: frequency of the observations in the `training_ts`
seasonal_fit: if a seasonal ARIMA model should be fitted
Returns:
predictions: point forecasts (i.e., the "prediction" column) and
confidence intervals (i.e, the four "low/high80/95" columns)
Raises:
ValueError: if `training_ts` contains `NaN` values
"""
# Initialize R only if it is actually used.
# For example, the nox session "ci-tests-fast" does not use it.
from urban_meal_delivery import init_r # noqa:F401,WPS433
# Re-seed R every time it is used to ensure reproducibility.
robjects.r('set.seed(42)')
if training_ts.isnull().any():
raise ValueError('`training_ts` must not contain `NaN` values')
# Copy the data from Python to R.
robjects.globalenv['data'] = robjects.r['ts'](
pandas2ri.py2rpy(training_ts), frequency=frequency,
)
seasonal = 'TRUE' if bool(seasonal_fit) else 'FALSE'
n_steps_ahead = len(forecast_interval)
# Make the predictions in R.
result = robjects.r(
f"""
as.data.frame(
forecast(
auto.arima(data, approximation = TRUE, seasonal = {seasonal:s}),
h = {n_steps_ahead:d}
)
)
""",
)
# Convert the results into a nice `pd.DataFrame` with the right `.index`.
forecasts = pandas2ri.rpy2py(result)
forecasts.index = forecast_interval
return forecasts.round(5).rename(
columns={
'Point Forecast': 'prediction',
'Lo 80': 'low80',
'Hi 80': 'high80',
'Lo 95': 'low95',
'Hi 95': 'high95',
},
)

181
src/urban_meal_delivery/forecasts/methods/decomposition.py
View file

@ -1,181 +0,0 @@
"""Seasonal-trend decomposition procedure based on LOESS (STL).
This module defines a `stl()` function that wraps R's STL decomposition function
using the `rpy2` library.
"""
import math
import pandas as pd
from rpy2 import robjects
from rpy2.robjects import pandas2ri
def stl( # noqa:C901,WPS210,WPS211,WPS231
time_series: pd.Series,
*,
frequency: int,
ns: int,
nt: int = None,
nl: int = None,
ds: int = 0,
dt: int = 1,
dl: int = 1,
js: int = None,
jt: int = None,
jl: int = None,
ni: int = 2,
no: int = 0, # noqa:WPS110
) -> pd.DataFrame:
"""Decompose a time series into seasonal, trend, and residual components.
This is a Python wrapper around the corresponding R function.
Further info on the STL method:
https://www.nniiem.ru/file/news/2016/stl-statistical-model.pdf
https://otexts.com/fpp2/stl.html
Further info on the R's "stl" function:
https://www.rdocumentation.org/packages/stats/versions/3.6.2/topics/stl
Args:
time_series: time series with a `DateTime` based index;
must not contain `NaN` values
frequency: frequency of the observations in the `time_series`
ns: smoothing parameter for the seasonal component
(= window size of the seasonal smoother);
must be odd and `>= 7` so that the seasonal component is smooth;
the greater `ns`, the smoother the seasonal component;
so, this is a hyper-parameter optimized in accordance with the application
nt: smoothing parameter for the trend component
(= window size of the trend smoother);
must be odd and `>= (1.5 * frequency) / [1 - (1.5 / ns)]`;
the latter threshold is the default value;
the greater `nt`, the smoother the trend component
nl: smoothing parameter for the low-pass filter;
must be odd and `>= frequency`;
the least odd number `>= frequency` is the default
ds: degree of locally fitted polynomial in seasonal smoothing;
must be `0` or `1`
dt: degree of locally fitted polynomial in trend smoothing;
must be `0` or `1`
dl: degree of locally fitted polynomial in low-pass smoothing;
must be `0` or `1`
js: number of steps by which the seasonal smoother skips ahead
and then linearly interpolates between observations;
if set to `1`, the smoother is evaluated at all points;
to make the STL decomposition faster, increase this value;
by default, `js` is the smallest integer `>= 0.1 * ns`
jt: number of steps by which the trend smoother skips ahead
and then linearly interpolates between observations;
if set to `1`, the smoother is evaluated at all points;
to make the STL decomposition faster, increase this value;
by default, `jt` is the smallest integer `>= 0.1 * nt`
jl: number of steps by which the low-pass smoother skips ahead
and then linearly interpolates between observations;
if set to `1`, the smoother is evaluated at all points;
to make the STL decomposition faster, increase this value;
by default, `jl` is the smallest integer `>= 0.1 * nl`
ni: number of iterations of the inner loop that updates the
seasonal and trend components;
usually, a low value (e.g., `2`) suffices
no: number of iterations of the outer loop that handles outliers;
also known as the "robustness" loop;
if no outliers need to be handled, set `no=0`;
otherwise, `no=5` or `no=10` combined with `ni=1` is a good choice
Returns:
result: a DataFrame with three columns ("seasonal", "trend", and "residual")
providing time series of the individual components
Raises:
ValueError: some argument does not adhere to the specifications above
"""
# Validate all arguments and set default values.
if time_series.isnull().any():
raise ValueError('`time_series` must not contain `NaN` values')
if ns % 2 == 0 or ns < 7:
raise ValueError('`ns` must be odd and `>= 7`')
default_nt = math.ceil((1.5 * frequency) / (1 - (1.5 / ns)))
if nt is not None:
if nt % 2 == 0 or nt < default_nt:
raise ValueError(
'`nt` must be odd and `>= (1.5 * frequency) / [1 - (1.5 / ns)]`, '
+ 'which is {0}'.format(default_nt),
)
else:
nt = default_nt
if nt % 2 == 0: # pragma: no cover => hard to construct edge case
nt += 1
if nl is not None:
if nl % 2 == 0 or nl < frequency:
raise ValueError('`nl` must be odd and `>= frequency`')
elif frequency % 2 == 0:
nl = frequency + 1
else: # pragma: no cover => hard to construct edge case
nl = frequency
if ds not in {0, 1}:
raise ValueError('`ds` must be either `0` or `1`')
if dt not in {0, 1}:
raise ValueError('`dt` must be either `0` or `1`')
if dl not in {0, 1}:
raise ValueError('`dl` must be either `0` or `1`')
if js is not None:
if js <= 0:
raise ValueError('`js` must be positive')
else:
js = math.ceil(ns / 10)
if jt is not None:
if jt <= 0:
raise ValueError('`jt` must be positive')
else:
jt = math.ceil(nt / 10)
if jl is not None:
if jl <= 0:
raise ValueError('`jl` must be positive')
else:
jl = math.ceil(nl / 10)
if ni <= 0:
raise ValueError('`ni` must be positive')
if no < 0:
raise ValueError('`no` must be non-negative')
elif no > 0:
robust = True
else:
robust = False
# Initialize R only if it is actually used.
# For example, the nox session "ci-tests-fast" does not use it.
from urban_meal_delivery import init_r # noqa:F401,WPS433
# Re-seed R every time it is used to ensure reproducibility.
robjects.r('set.seed(42)')
# Call the STL function in R.
ts = robjects.r['ts'](pandas2ri.py2rpy(time_series), frequency=frequency)
result = robjects.r['stl'](
ts, ns, ds, nt, dt, nl, dl, js, jt, jl, robust, ni, no, # noqa:WPS221
)
# Unpack the result to a `pd.DataFrame`.
result = pandas2ri.rpy2py(result[0])
result = pd.DataFrame(
data={
'seasonal': result[:, 0],
'trend': result[:, 1],
'residual': result[:, 2],
},
index=time_series.index,
)
return result.round(5)

77
src/urban_meal_delivery/forecasts/methods/ets.py
View file

@ -1,77 +0,0 @@
"""A wrapper around R's "ets" function."""
import pandas as pd
from rpy2 import robjects
from rpy2.robjects import pandas2ri
def predict(
training_ts: pd.Series,
forecast_interval: pd.DatetimeIndex,
*,
frequency: int,
seasonal_fit: bool = False,
) -> pd.DataFrame:
"""Predict with an automatically calibrated ETS model.
Note: The function does not check if the `forecast_interval`
extends the `training_ts`'s interval without a gap!
Args:
training_ts: past observations to be fitted
forecast_interval: interval into which the `training_ts` is forecast;
its length becomes the step size `h` in the forecasting model in R
frequency: frequency of the observations in the `training_ts`
seasonal_fit: if a "ZZZ" (seasonal) or a "ZZN" (non-seasonal)
type ETS model should be fitted
Returns:
predictions: point forecasts (i.e., the "prediction" column) and
confidence intervals (i.e, the four "low/high80/95" columns)
Raises:
ValueError: if `training_ts` contains `NaN` values
"""
# Initialize R only if it is actually used.
# For example, the nox session "ci-tests-fast" does not use it.
from urban_meal_delivery import init_r # noqa:F401,WPS433
# Re-seed R every time it is used to ensure reproducibility.
robjects.r('set.seed(42)')
if training_ts.isnull().any():
raise ValueError('`training_ts` must not contain `NaN` values')
# Copy the data from Python to R.
robjects.globalenv['data'] = robjects.r['ts'](
pandas2ri.py2rpy(training_ts), frequency=frequency,
)
model = 'ZZZ' if bool(seasonal_fit) else 'ZZN'
n_steps_ahead = len(forecast_interval)
# Make the predictions in R.
result = robjects.r(
f"""
as.data.frame(
forecast(
ets(data, model = "{model:s}"),
h = {n_steps_ahead:d}
)
)
""",
)
# Convert the results into a nice `pd.DataFrame` with the right `.index`.
forecasts = pandas2ri.rpy2py(result)
forecasts.index = forecast_interval
return forecasts.round(5).rename(
columns={
'Point Forecast': 'prediction',
'Lo 80': 'low80',
'Hi 80': 'high80',
'Lo 95': 'low95',
'Hi 95': 'high95',
},
)

72
src/urban_meal_delivery/forecasts/methods/extrapolate_season.py
View file

@ -1,72 +0,0 @@
"""Forecast by linear extrapolation of a seasonal component."""
import pandas as pd
from statsmodels.tsa import api as ts_stats
def predict(
training_ts: pd.Series, forecast_interval: pd.DatetimeIndex, *, frequency: int,
) -> pd.DataFrame:
"""Extrapolate a seasonal component with a linear model.
A naive forecast for each time unit of the day is calculated by linear
extrapolation from all observations of the same time of day and on the same
day of the week (i.e., same seasonal lag).
Note: The function does not check if the `forecast_interval`
extends the `training_ts`'s interval without a gap!
Args:
training_ts: past observations to be fitted;
assumed to be a seasonal component after time series decomposition
forecast_interval: interval into which the `training_ts` is forecast;
its length becomes the numbers of time steps to be forecast
frequency: frequency of the observations in the `training_ts`
Returns:
predictions: point forecasts (i.e., the "prediction" column);
includes the four "low/high80/95" columns for the confidence intervals
that only contain `NaN` values as this method does not make
any statistical assumptions about the time series process
Raises:
ValueError: if `training_ts` contains `NaN` values or some predictions
could not be made for time steps in the `forecast_interval`
"""
if training_ts.isnull().any():
raise ValueError('`training_ts` must not contain `NaN` values')
extrapolated_ts = pd.Series(index=forecast_interval, dtype=float)
seasonal_lag = frequency * (training_ts.index[1] - training_ts.index[0])
for lag in range(frequency):
# Obtain all `observations` of the same seasonal lag and
# fit a straight line through them (= `trend`).
observations = training_ts[slice(lag, 999_999_999, frequency)]
trend = observations - ts_stats.detrend(observations)
# Create a point forecast by linear extrapolation
# for one or even more time steps ahead.
slope = trend[-1] - trend[-2]
prediction = trend[-1] + slope
idx = observations.index.max() + seasonal_lag
while idx <= forecast_interval.max():
if idx in forecast_interval:
extrapolated_ts.loc[idx] = prediction
prediction += slope
idx += seasonal_lag
# Sanity check.
if extrapolated_ts.isnull().any(): # pragma: no cover
raise ValueError('missing predictions in the `forecast_interval`')
return pd.DataFrame(
data={
'prediction': extrapolated_ts.round(5),
'low80': float('NaN'),
'high80': float('NaN'),
'low95': float('NaN'),
'high95': float('NaN'),
},
index=forecast_interval,
)

37
src/urban_meal_delivery/forecasts/models/__init__.py
View file

@ -1,37 +0,0 @@
"""Define the forecasting `*Model`s used in this project.
`*Model`s are different from plain forecasting `methods` in that they are tied
to a given kind of historic order time series, as provided by the `OrderHistory`
class in the `timify` module. For example, the ARIMA model applied to a vertical
time series becomes the `VerticalARIMAModel`.
An overview of the `*Model`s used for tactical forecasting can be found in section
"3.6 Forecasting Models" in the paper "Real-time Demand Forecasting for an Urban
Delivery Platform" that is part of the `urban-meal-delivery` research project.
For the paper check:
https://github.com/webartifex/urban-meal-delivery-demand-forecasting/blob/main/paper.pdf
https://www.sciencedirect.com/science/article/pii/S1366554520307936
This sub-package is organized as follows. The `base` module defines an abstract
`ForecastingModelABC` class that unifies how the concrete `*Model`s work.
While the abstract `.predict()` method returns a `pd.DataFrame` (= basically,
the result of one of the forecasting `methods`, the concrete `.make_forecast()`
method converts the results into `Forecast` (=ORM) objects.
Also, `.make_forecast()` implements a caching strategy where already made
`Forecast`s are loaded from the database instead of calculating them again,
which could be a heavier computation.
The `tactical` sub-package contains all the `*Model`s used to implement the
predictive routing strategy employed by the UDP.
A future `planning` sub-package will contain the `*Model`s used to plan the
`Courier`'s shifts a week ahead.
""" # noqa:RST215
from urban_meal_delivery.forecasts.models.base import ForecastingModelABC
from urban_meal_delivery.forecasts.models.tactical.horizontal import HorizontalETSModel
from urban_meal_delivery.forecasts.models.tactical.horizontal import HorizontalSMAModel
from urban_meal_delivery.forecasts.models.tactical.other import TrivialModel
from urban_meal_delivery.forecasts.models.tactical.realtime import RealtimeARIMAModel
from urban_meal_delivery.forecasts.models.tactical.vertical import VerticalARIMAModel

116
src/urban_meal_delivery/forecasts/models/base.py
View file

@ -1,116 +0,0 @@
"""The abstract blueprint for a forecasting `*Model`."""
import abc
import datetime as dt
import pandas as pd
from urban_meal_delivery import db
from urban_meal_delivery.forecasts import timify
class ForecastingModelABC(abc.ABC):
"""An abstract interface of a forecasting `*Model`."""
def __init__(self, order_history: timify.OrderHistory) -> None:
"""Initialize a new forecasting model.
Args:
order_history: an abstraction providing the time series data
"""
self._order_history = order_history
@property
@abc.abstractmethod
def name(self) -> str:
"""The name of the model.
Used to identify `Forecast`s of the same `*Model` in the database.
So, these must be chosen carefully and must not be changed later on!
Example: "hets" or "varima" for tactical demand forecasting
"""
@abc.abstractmethod
def predict(
self, pixel: db.Pixel, predict_at: dt.datetime, train_horizon: int,
) -> pd.DataFrame:
"""Concrete implementation of how a `*Model` makes a prediction.
This method is called by the unified `*Model.make_forecast()` method,
which implements the caching logic with the database.
Args:
pixel: pixel in which the prediction is made
predict_at: time step (i.e., "start_at") to make the prediction for
train_horizon: weeks of historic data used to predict `predict_at`
Returns:
actuals, predictions, and possibly 80%/95% confidence intervals;
includes a row for the time step starting at `predict_at` and
may contain further rows for other time steps on the same day
""" # noqa:DAR202
def make_forecast(
self, pixel: db.Pixel, predict_at: dt.datetime, train_horizon: int,
) -> db.Forecast:
"""Make a forecast for the time step starting at `predict_at`.
Important: This method uses a unified `predict_at` argument.
Some `*Model`s, in particular vertical ones, are only trained once per
day and then make a prediction for all time steps on that day, and
therefore, work with a `predict_day` argument instead of `predict_at`
behind the scenes. Then, all `Forecast`s are stored into the database
and only the one starting at `predict_at` is returned.
Args:
pixel: pixel in which the `Forecast` is made
predict_at: time step (i.e., "start_at") to make the `Forecast` for
train_horizon: weeks of historic data used to forecast `predict_at`
Returns:
actual, prediction, and possibly 80%/95% confidence intervals
for the time step starting at `predict_at`
# noqa:DAR401 RuntimeError
"""
if ( # noqa:WPS337
cached_forecast := db.session.query(db.Forecast) # noqa:WPS221
.filter_by(pixel=pixel)
.filter_by(start_at=predict_at)
.filter_by(time_step=self._order_history.time_step)
.filter_by(train_horizon=train_horizon)
.filter_by(model=self.name)
.first()
) :
return cached_forecast
# Horizontal and real-time `*Model`s return a `pd.DataFrame` with one
# row corresponding to the time step starting at `predict_at` whereas
# vertical models return several rows, covering all time steps of a day.
predictions = self.predict(pixel, predict_at, train_horizon)
# Convert the `predictions` into a `list` of `Forecast` objects.
forecasts = db.Forecast.from_dataframe(
pixel=pixel,
time_step=self._order_history.time_step,
train_horizon=train_horizon,
model=self.name,
data=predictions,
)
# We persist all `Forecast`s into the database to
# not have to run the same model training again.
db.session.add_all(forecasts)
db.session.commit()
# The one `Forecast` object asked for must be in `forecasts`
# if the concrete `*Model.predict()` method works correctly; ...
for forecast in forecasts:
if forecast.start_at == predict_at:
return forecast
# ..., however, we put in a loud error, just in case.
raise RuntimeError( # pragma: no cover
'`Forecast` for `predict_at` was not returned by `*Model.predict()`',
)

16
src/urban_meal_delivery/forecasts/models/tactical/__init__.py
View file

@ -1,16 +0,0 @@
"""Forecasting `*Model`s to predict demand for tactical purposes.
The `*Model`s in this module predict only a small number (e.g., one)
of time steps into the near future and are used to implement the
predictive routing strategies employed by the UDP.
They are classified into "horizontal", "vertical", and "real-time" models
with respect to what historic data they are trained on and how often they
are re-trained on the day to be predicted. For the details, check section
"3.6 Forecasting Models" in the paper "Real-time Demand Forecasting for an
Urban Delivery Platform".
For the paper check:
https://github.com/webartifex/urban-meal-delivery-demand-forecasting/blob/main/paper.pdf
https://www.sciencedirect.com/science/article/pii/S1366554520307936
""" # noqa:RST215

130
src/urban_meal_delivery/forecasts/models/tactical/horizontal.py
View file

@ -1,130 +0,0 @@
"""Horizontal forecasting `*Model`s to predict demand for tactical purposes.
Horizontal `*Model`s take the historic order counts only from time steps
corresponding to the same time of day as the one to be predicted (i.e., the
one starting at `predict_at`). Then, they make a prediction for only one day
into the future. Thus, the training time series have a `frequency` of `7`, the
number of days in a week.
""" # noqa:RST215
import datetime as dt
import pandas as pd
from urban_meal_delivery import db
from urban_meal_delivery.forecasts import methods
from urban_meal_delivery.forecasts.models import base
class HorizontalETSModel(base.ForecastingModelABC):
"""The ETS model applied on a horizontal time series."""
name = 'hets'
def predict(
self, pixel: db.Pixel, predict_at: dt.datetime, train_horizon: int,
) -> pd.DataFrame:
"""Predict demand for a time step.
Args:
pixel: pixel in which the prediction is made
predict_at: time step (i.e., "start_at") to make the prediction for
train_horizon: weeks of historic data used to predict `predict_at`
Returns:
actual order counts (i.e., the "actual" column),
point forecasts (i.e., the "prediction" column), and
confidence intervals (i.e, the four "low/high/80/95" columns);
contains one row for the `predict_at` time step
# noqa:DAR401 RuntimeError
"""
# Generate the historic (and horizontal) order time series.
training_ts, frequency, actuals_ts = self._order_history.make_horizontal_ts(
pixel_id=pixel.id, predict_at=predict_at, train_horizon=train_horizon,
)
# Sanity check.
if frequency != 7: # pragma: no cover
raise RuntimeError('`frequency` should be `7`')
# Make `predictions` with the seasonal ETS method ("ZZZ" model).
predictions = methods.ets.predict(
training_ts=training_ts,
forecast_interval=pd.DatetimeIndex(actuals_ts.index),
frequency=frequency, # `== 7`, the number of weekdays
seasonal_fit=True, # because there was no decomposition before
)
predictions.insert(loc=0, column='actual', value=actuals_ts)
# Sanity checks.
if predictions.isnull().sum().any(): # pragma: no cover
raise RuntimeError('missing predictions in hets model')
if predict_at not in predictions.index: # pragma: no cover
raise RuntimeError('missing prediction for `predict_at`')
return predictions
class HorizontalSMAModel(base.ForecastingModelABC):
"""A simple moving average model applied on a horizontal time series."""
name = 'hsma'
def predict(
self, pixel: db.Pixel, predict_at: dt.datetime, train_horizon: int,
) -> pd.DataFrame:
"""Predict demand for a time step.
Args:
pixel: pixel in which the prediction is made
predict_at: time step (i.e., "start_at") to make the prediction for
train_horizon: weeks of historic data used to predict `predict_at`
Returns:
actual order counts (i.e., the "actual" column) and
point forecasts (i.e., the "prediction" column);
this model does not support confidence intervals;
contains one row for the `predict_at` time step
# noqa:DAR401 RuntimeError
"""
# Generate the historic (and horizontal) order time series.
training_ts, frequency, actuals_ts = self._order_history.make_horizontal_ts(
pixel_id=pixel.id, predict_at=predict_at, train_horizon=train_horizon,
)
# Sanity checks.
if frequency != 7: # pragma: no cover
raise RuntimeError('`frequency` should be `7`')
if len(actuals_ts) != 1: # pragma: no cover
raise RuntimeError(
'the hsma model can only predict one step into the future',
)
# The "prediction" is calculated as the `np.mean()`.
# As the `training_ts` covers only full week horizons,
# no adjustment regarding the weekly seasonality is needed.
predictions = pd.DataFrame(
data={
'actual': actuals_ts,
'prediction': training_ts.values.mean(),
'low80': float('NaN'),
'high80': float('NaN'),
'low95': float('NaN'),
'high95': float('NaN'),
},
index=actuals_ts.index,
)
# Sanity checks.
if ( # noqa:WPS337
predictions[['actual', 'prediction']].isnull().any().any()
): # pragma: no cover
raise RuntimeError('missing predictions in hsma model')
if predict_at not in predictions.index: # pragma: no cover
raise RuntimeError('missing prediction for `predict_at`')
return predictions

75
src/urban_meal_delivery/forecasts/models/tactical/other.py
View file

@ -1,75 +0,0 @@
"""Forecasting `*Model`s to predict demand for tactical purposes ...
... that cannot be classified into either "horizontal", "vertical",
or "real-time".
""" # noqa:RST215
import datetime as dt
import pandas as pd
from urban_meal_delivery import db
from urban_meal_delivery.forecasts.models import base
class TrivialModel(base.ForecastingModelABC):
"""A trivial model predicting `0` demand.
No need to distinguish between a "horizontal", "vertical", or
"real-time" model here as all give the same prediction for all time steps.
"""
name = 'trivial'
def predict(
self, pixel: db.Pixel, predict_at: dt.datetime, train_horizon: int,
) -> pd.DataFrame:
"""Predict demand for a time step.
Args:
pixel: pixel in which the prediction is made
predict_at: time step (i.e., "start_at") to make the prediction for
train_horizon: weeks of historic data used to predict `predict_at`
Returns:
actual order counts (i.e., the "actual" column) and
point forecasts (i.e., the "prediction" column);
this model does not support confidence intervals;
contains one row for the `predict_at` time step
# noqa:DAR401 RuntimeError
"""
# Generate the historic order time series mainly to check if a valid
# `training_ts` exists (i.e., the demand history is long enough).
_, frequency, actuals_ts = self._order_history.make_horizontal_ts(
pixel_id=pixel.id, predict_at=predict_at, train_horizon=train_horizon,
)
# Sanity checks.
if frequency != 7: # pragma: no cover
raise RuntimeError('`frequency` should be `7`')
if len(actuals_ts) != 1: # pragma: no cover
raise RuntimeError(
'the trivial model can only predict one step into the future',
)
# The "prediction" is simply `0.0`.
predictions = pd.DataFrame(
data={
'actual': actuals_ts,
'prediction': 0.0,
'low80': float('NaN'),
'high80': float('NaN'),
'low95': float('NaN'),
'high95': float('NaN'),
},
index=actuals_ts.index,
)
# Sanity checks.
if predictions['actual'].isnull().any(): # pragma: no cover
raise RuntimeError('missing actuals in trivial model')
if predict_at not in predictions.index: # pragma: no cover
raise RuntimeError('missing prediction for `predict_at`')
return predictions

117
src/urban_meal_delivery/forecasts/models/tactical/realtime.py
View file

@ -1,117 +0,0 @@
"""Real-time forecasting `*Model`s to predict demand for tactical purposes.
Real-time `*Model`s take order counts of all time steps in the training data
and make a prediction for only one time step on the day to be predicted (i.e.,
the one starting at `predict_at`). Thus, the training time series have a
`frequency` of the number of weekdays, `7`, times the number of time steps on a
day. For example, for 60-minute time steps, the `frequency` becomes `7 * 12`
(= operating hours from 11 am to 11 pm), which is `84`. Real-time `*Model`s
train the forecasting `methods` on a seasonally decomposed time series internally.
""" # noqa:RST215
import datetime as dt
import pandas as pd
from urban_meal_delivery import db
from urban_meal_delivery.forecasts import methods
from urban_meal_delivery.forecasts.models import base
class RealtimeARIMAModel(base.ForecastingModelABC):
"""The ARIMA model applied on a real-time time series."""
name = 'rtarima'
def predict(
self, pixel: db.Pixel, predict_at: dt.datetime, train_horizon: int,
) -> pd.DataFrame:
"""Predict demand for a time step.
Args:
pixel: pixel in which the prediction is made
predict_at: time step (i.e., "start_at") to make the prediction for
train_horizon: weeks of historic data used to predict `predict_at`
Returns:
actual order counts (i.e., the "actual" column),
point forecasts (i.e., the "prediction" column), and
confidence intervals (i.e, the four "low/high/80/95" columns);
contains one row for the `predict_at` time step
# noqa:DAR401 RuntimeError
"""
# Generate the historic (and real-time) order time series.
training_ts, frequency, actuals_ts = self._order_history.make_realtime_ts(
pixel_id=pixel.id, predict_at=predict_at, train_horizon=train_horizon,
)
# Decompose the `training_ts` to make predictions for the seasonal
# component and the seasonally adjusted observations separately.
decomposed_training_ts = methods.decomposition.stl(
time_series=training_ts,
frequency=frequency,
# "Periodic" `ns` parameter => same seasonal component value
# for observations of the same lag.
ns=999,
)
# Make predictions for the seasonal component by linear extrapolation.
seasonal_predictions = methods.extrapolate_season.predict(
training_ts=decomposed_training_ts['seasonal'],
forecast_interval=pd.DatetimeIndex(actuals_ts.index),
frequency=frequency,
)
# Make predictions with the ARIMA model on the seasonally adjusted time series.
seasonally_adjusted_predictions = methods.arima.predict(
training_ts=(
decomposed_training_ts['trend'] + decomposed_training_ts['residual']
),
forecast_interval=pd.DatetimeIndex(actuals_ts.index),
# Because the seasonality was taken out before,
# the `training_ts` has, by definition, a `frequency` of `1`.
frequency=1,
seasonal_fit=False,
)
# The overall `predictions` are the sum of the separate predictions above.
# As the linear extrapolation of the seasonal component has no
# confidence interval, we put the one from the ARIMA model around
# the extrapolated seasonal component.
predictions = pd.DataFrame(
data={
'actual': actuals_ts,
'prediction': (
seasonal_predictions['prediction'] # noqa:WPS204
+ seasonally_adjusted_predictions['prediction']
),
'low80': (
seasonal_predictions['prediction']
+ seasonally_adjusted_predictions['low80']
),
'high80': (
seasonal_predictions['prediction']
+ seasonally_adjusted_predictions['high80']
),
'low95': (
seasonal_predictions['prediction']
+ seasonally_adjusted_predictions['low95']
),
'high95': (
seasonal_predictions['prediction']
+ seasonally_adjusted_predictions['high95']
),
},
index=actuals_ts.index,
)
# Sanity checks.
if len(predictions) != 1: # pragma: no cover
raise RuntimeError('real-time models should predict exactly one time step')
if predictions.isnull().sum().any(): # pragma: no cover
raise RuntimeError('missing predictions in rtarima model')
if predict_at not in predictions.index: # pragma: no cover
raise RuntimeError('missing prediction for `predict_at`')
return predictions

119
src/urban_meal_delivery/forecasts/models/tactical/vertical.py
View file

@ -1,119 +0,0 @@
"""Vertical forecasting `*Model`s to predict demand for tactical purposes.
Vertical `*Model`s take order counts of all time steps in the training data
and make a prediction for all time steps on the day to be predicted at once.
Thus, the training time series have a `frequency` of the number of weekdays,
`7`, times the number of time steps on a day. For example, with 60-minute time
steps, the `frequency` becomes `7 * 12` (= operating hours from 11 am to 11 pm),
which is `84`. Vertical `*Model`s train the forecasting `methods` on a seasonally
decomposed time series internally.
""" # noqa:RST215
import datetime as dt
import pandas as pd
from urban_meal_delivery import db
from urban_meal_delivery.forecasts import methods
from urban_meal_delivery.forecasts.models import base
class VerticalARIMAModel(base.ForecastingModelABC):
"""The ARIMA model applied on a vertical time series."""
name = 'varima'
def predict(
self, pixel: db.Pixel, predict_at: dt.datetime, train_horizon: int,
) -> pd.DataFrame:
"""Predict demand for a time step.
Args:
pixel: pixel in which the prediction is made
predict_at: time step (i.e., "start_at") to make the prediction for
train_horizon: weeks of historic data used to predict `predict_at`
Returns:
actual order counts (i.e., the "actual" column),
point forecasts (i.e., the "prediction" column), and
confidence intervals (i.e, the four "low/high/80/95" columns);
contains several rows, including one for the `predict_at` time step
# noqa:DAR401 RuntimeError
"""
# Generate the historic (and vertical) order time series.
training_ts, frequency, actuals_ts = self._order_history.make_vertical_ts(
pixel_id=pixel.id,
predict_day=predict_at.date(),
train_horizon=train_horizon,
)
# Decompose the `training_ts` to make predictions for the seasonal
# component and the seasonally adjusted observations separately.
decomposed_training_ts = methods.decomposition.stl(
time_series=training_ts,
frequency=frequency,
# "Periodic" `ns` parameter => same seasonal component value
# for observations of the same lag.
ns=999,
)
# Make predictions for the seasonal component by linear extrapolation.
seasonal_predictions = methods.extrapolate_season.predict(
training_ts=decomposed_training_ts['seasonal'],
forecast_interval=pd.DatetimeIndex(actuals_ts.index),
frequency=frequency,
)
# Make predictions with the ARIMA model on the seasonally adjusted time series.
seasonally_adjusted_predictions = methods.arima.predict(
training_ts=(
decomposed_training_ts['trend'] + decomposed_training_ts['residual']
),
forecast_interval=pd.DatetimeIndex(actuals_ts.index),
# Because the seasonality was taken out before,
# the `training_ts` has, by definition, a `frequency` of `1`.
frequency=1,
seasonal_fit=False,
)
# The overall `predictions` are the sum of the separate predictions above.
# As the linear extrapolation of the seasonal component has no
# confidence interval, we put the one from the ARIMA model around
# the extrapolated seasonal component.
predictions = pd.DataFrame(
data={
'actual': actuals_ts,
'prediction': (
seasonal_predictions['prediction'] # noqa:WPS204
+ seasonally_adjusted_predictions['prediction']
),
'low80': (
seasonal_predictions['prediction']
+ seasonally_adjusted_predictions['low80']
),
'high80': (
seasonal_predictions['prediction']
+ seasonally_adjusted_predictions['high80']
),
'low95': (
seasonal_predictions['prediction']
+ seasonally_adjusted_predictions['low95']
),
'high95': (
seasonal_predictions['prediction']
+ seasonally_adjusted_predictions['high95']
),
},
index=actuals_ts.index,
)
# Sanity checks.
if len(predictions) <= 1: # pragma: no cover
raise RuntimeError('vertical models should predict several time steps')
if predictions.isnull().sum().any(): # pragma: no cover
raise RuntimeError('missing predictions in varima model')
if predict_at not in predictions.index: # pragma: no cover
raise RuntimeError('missing prediction for `predict_at`')
return predictions

569
src/urban_meal_delivery/forecasts/timify.py
View file

@ -1,569 +0,0 @@
"""Obtain and work with time series data."""
from __future__ import annotations
import datetime as dt
from typing import Tuple
import pandas as pd
import sqlalchemy as sa
from urban_meal_delivery import config
from urban_meal_delivery import db
from urban_meal_delivery.forecasts import models
class OrderHistory:
"""Generate time series from the `Order` model in the database.
The purpose of this class is to abstract away the managing of the order data
in memory and the slicing the data into various kinds of time series.
"""
def __init__(self, grid: db.Grid, time_step: int) -> None:
"""Initialize a new `OrderHistory` object.
Args:
grid: pixel grid used to aggregate orders spatially
time_step: interval length (in minutes) into which orders are aggregated
# noqa:DAR401 RuntimeError
"""
self._grid = grid
self._time_step = time_step
# Number of daily time steps must be a whole multiple of `time_step` length.
n_daily_time_steps = (
60 * (config.SERVICE_END - config.SERVICE_START) / time_step
)
if n_daily_time_steps != int(n_daily_time_steps): # pragma: no cover
raise RuntimeError('Internal error: configuration has invalid TIME_STEPS')
self._n_daily_time_steps = int(n_daily_time_steps)
# The `_data` are populated by `.aggregate_orders()`.
self._data = None
@property
def time_step(self) -> int:
"""The length of one time step."""
return self._time_step
@property
def totals(self) -> pd.DataFrame:
"""The order totals by `Pixel` and `.time_step`.
The returned object should not be mutated!
Returns:
order_totals: a one-column `DataFrame` with a `MultiIndex` of the
"pixel_id"s and "start_at"s (i.e., beginnings of the intervals);
the column with data is "n_orders"
"""
if self._data is None:
self._data = self.aggregate_orders()
return self._data
def aggregate_orders(self) -> pd.DataFrame: # pragma: no cover
"""Generate and load all order totals from the database."""
# `data` is probably missing "pixel_id"-"start_at" pairs.
# This happens when there is no demand in the `Pixel` in the given `time_step`.
data = pd.read_sql_query(
sa.text(
f""" -- # noqa:WPS221
SELECT
pixel_id,
start_at,
COUNT(*) AS n_orders
FROM (
SELECT
pixel_id,
placed_at_without_seconds - minutes_to_be_cut AS start_at
FROM (
SELECT
pixels.pixel_id,
DATE_TRUNC('MINUTE', orders.placed_at)
AS placed_at_without_seconds,
(
(
(
EXTRACT(MINUTES FROM orders.placed_at)::INTEGER
% {self._time_step}
)::TEXT
||
' MINUTES'
)::INTERVAL
) AS minutes_to_be_cut
FROM (
SELECT
{config.CLEAN_SCHEMA}.orders.id,
{config.CLEAN_SCHEMA}.orders.placed_at,
{config.CLEAN_SCHEMA}.orders.pickup_address_id
FROM
{config.CLEAN_SCHEMA}.orders
INNER JOIN (
SELECT
{config.CLEAN_SCHEMA}.addresses.id AS address_id
FROM
{config.CLEAN_SCHEMA}.addresses
WHERE
{config.CLEAN_SCHEMA}.addresses.city_id
= {self._grid.city.id}
) AS in_city
ON {config.CLEAN_SCHEMA}.orders.pickup_address_id
= in_city.address_id
WHERE
{config.CLEAN_SCHEMA}.orders.ad_hoc IS TRUE
) AS
orders
INNER JOIN (
SELECT
{config.CLEAN_SCHEMA}.addresses_pixels.address_id,
{config.CLEAN_SCHEMA}.addresses_pixels.pixel_id
FROM
{config.CLEAN_SCHEMA}.addresses_pixels
WHERE
{config.CLEAN_SCHEMA}.addresses_pixels.grid_id
= {self._grid.id}
AND
{config.CLEAN_SCHEMA}.addresses_pixels.city_id
= {self._grid.city.id} -- -> sanity check
) AS pixels
ON orders.pickup_address_id = pixels.address_id
) AS placed_at_aggregated_into_start_at
) AS pixel_start_at_combinations
GROUP BY
pixel_id,
start_at
ORDER BY
pixel_id,
start_at;
""",
), # noqa:WPS355
con=db.connection,
index_col=['pixel_id', 'start_at'],
)
if data.empty:
return data
# Calculate the first and last "start_at" value ...
start_day = data.index.levels[1].min().date()
start = dt.datetime(
start_day.year, start_day.month, start_day.day, config.SERVICE_START,
)
end_day = data.index.levels[1].max().date()
end = dt.datetime(end_day.year, end_day.month, end_day.day, config.SERVICE_END)
# ... and all possible `tuple`s of "pixel_id"-"start_at" combinations.
# The "start_at" values must lie within the operating hours.
gen = (
(pixel_id, start_at)
for pixel_id in sorted(data.index.levels[0])
for start_at in pd.date_range(start, end, freq=f'{self._time_step}T')
if config.SERVICE_START <= start_at.hour < config.SERVICE_END
)
# Re-index `data` filling in `0`s where there is no demand.
index = pd.MultiIndex.from_tuples(gen)
index.names = ['pixel_id', 'start_at']
return data.reindex(index, fill_value=0)
def first_order_at(self, pixel_id: int) -> dt.datetime:
"""Get the time step with the first order in a pixel.
Args:
pixel_id: pixel for which to get the first order
Returns:
minimum "start_at" from when orders take place
Raises:
LookupError: `pixel_id` not in `grid`
# noqa:DAR401 RuntimeError
"""
try:
intra_pixel = self.totals.loc[pixel_id]
except KeyError:
raise LookupError('The `pixel_id` is not in the `grid`') from None
first_order = intra_pixel[intra_pixel['n_orders'] > 0].index.min()
# Sanity check: without an `Order`, the `Pixel` should not exist.
if first_order is pd.NaT: # pragma: no cover
raise RuntimeError('no orders in the pixel')
# Return a proper `datetime.datetime` object.
return dt.datetime(
first_order.year,
first_order.month,
first_order.day,
first_order.hour,
first_order.minute,
)
def last_order_at(self, pixel_id: int) -> dt.datetime:
"""Get the time step with the last order in a pixel.
Args:
pixel_id: pixel for which to get the last order
Returns:
maximum "start_at" from when orders take place
Raises:
LookupError: `pixel_id` not in `grid`
# noqa:DAR401 RuntimeError
"""
try:
intra_pixel = self.totals.loc[pixel_id]
except KeyError:
raise LookupError('The `pixel_id` is not in the `grid`') from None
last_order = intra_pixel[intra_pixel['n_orders'] > 0].index.max()
# Sanity check: without an `Order`, the `Pixel` should not exist.
if last_order is pd.NaT: # pragma: no cover
raise RuntimeError('no orders in the pixel')
# Return a proper `datetime.datetime` object.
return dt.datetime(
last_order.year,
last_order.month,
last_order.day,
last_order.hour,
last_order.minute,
)
def make_horizontal_ts( # noqa:WPS210
self, pixel_id: int, predict_at: dt.datetime, train_horizon: int,
) -> Tuple[pd.Series, int, pd.Series]:
"""Slice a horizontal time series out of the `.totals`.
Create a time series covering `train_horizon` weeks that can be used
for training a forecasting model to predict the demand at `predict_at`.
For explanation of the terms "horizontal", "vertical", and "real-time"
in the context of time series, see section 3.2 in the following paper:
https://github.com/webartifex/urban-meal-delivery-demand-forecasting/blob/main/paper.pdf
Args:
pixel_id: pixel in which the time series is aggregated
predict_at: time step (i.e., "start_at") for which a prediction is made
train_horizon: weeks of historic data used to predict `predict_at`
Returns:
training time series, frequency, actual order count at `predict_at`
Raises:
LookupError: `pixel_id` not in `grid` or `predict_at` not in `.totals`
RuntimeError: desired time series slice is not entirely in `.totals`
"""
try:
intra_pixel = self.totals.loc[pixel_id]
except KeyError:
raise LookupError('The `pixel_id` is not in the `grid`') from None
if predict_at >= config.CUTOFF_DAY: # pragma: no cover
raise RuntimeError('Internal error: cannot predict beyond the given data')
# The first and last training day are just before the `predict_at` day
# and span exactly `train_horizon` weeks covering only the times of the
# day equal to the hour/minute of `predict_at`.
first_train_day = predict_at.date() - dt.timedelta(weeks=train_horizon)
first_start_at = dt.datetime(
first_train_day.year,
first_train_day.month,
first_train_day.day,
predict_at.hour,
predict_at.minute,
)
last_train_day = predict_at.date() - dt.timedelta(days=1)
last_start_at = dt.datetime(
last_train_day.year,
last_train_day.month,
last_train_day.day,
predict_at.hour,
predict_at.minute,
)
# The frequency is the number of weekdays.
frequency = 7
# Take only the counts at the `predict_at` time.
training_ts = intra_pixel.loc[
first_start_at : last_start_at : self._n_daily_time_steps, # type:ignore
'n_orders',
]
if len(training_ts) != frequency * train_horizon:
raise RuntimeError('Not enough historic data for `predict_at`')
actuals_ts = intra_pixel.loc[[predict_at], 'n_orders']
if not len(actuals_ts): # pragma: no cover
raise LookupError('`predict_at` is not in the order history')
return training_ts, frequency, actuals_ts
def make_vertical_ts( # noqa:WPS210
self, pixel_id: int, predict_day: dt.date, train_horizon: int,
) -> Tuple[pd.Series, int, pd.Series]:
"""Slice a vertical time series out of the `.totals`.
Create a time series covering `train_horizon` weeks that can be used
for training a forecasting model to predict the demand on the `predict_day`.
For explanation of the terms "horizontal", "vertical", and "real-time"
in the context of time series, see section 3.2 in the following paper:
https://github.com/webartifex/urban-meal-delivery-demand-forecasting/blob/main/paper.pdf
Args:
pixel_id: pixel in which the time series is aggregated
predict_day: day for which predictions are made
train_horizon: weeks of historic data used to predict `predict_at`
Returns:
training time series, frequency, actual order counts on `predict_day`
Raises:
LookupError: `pixel_id` not in `grid` or `predict_day` not in `.totals`
RuntimeError: desired time series slice is not entirely in `.totals`
"""
try:
intra_pixel = self.totals.loc[pixel_id]
except KeyError:
raise LookupError('The `pixel_id` is not in the `grid`') from None
if predict_day >= config.CUTOFF_DAY.date(): # pragma: no cover
raise RuntimeError('Internal error: cannot predict beyond the given data')
# The first and last training day are just before the `predict_day`
# and span exactly `train_horizon` weeks covering all times of the day.
first_train_day = predict_day - dt.timedelta(weeks=train_horizon)
first_start_at = dt.datetime(
first_train_day.year,
first_train_day.month,
first_train_day.day,
config.SERVICE_START,
0,
)
last_train_day = predict_day - dt.timedelta(days=1)
last_start_at = dt.datetime(
last_train_day.year,
last_train_day.month,
last_train_day.day,
config.SERVICE_END, # subtract one `time_step` below
0,
) - dt.timedelta(minutes=self._time_step)
# The frequency is the number of weekdays times the number of daily time steps.
frequency = 7 * self._n_daily_time_steps
# Take all the counts between `first_train_day` and `last_train_day`.
training_ts = intra_pixel.loc[
first_start_at:last_start_at, # type:ignore
'n_orders',
]
if len(training_ts) != frequency * train_horizon:
raise RuntimeError('Not enough historic data for `predict_day`')
first_prediction_at = dt.datetime(
predict_day.year,
predict_day.month,
predict_day.day,
config.SERVICE_START,
0,
)
last_prediction_at = dt.datetime(
predict_day.year,
predict_day.month,
predict_day.day,
config.SERVICE_END, # subtract one `time_step` below
0,
) - dt.timedelta(minutes=self._time_step)
actuals_ts = intra_pixel.loc[
first_prediction_at:last_prediction_at, # type:ignore
'n_orders',
]
if not len(actuals_ts): # pragma: no cover
raise LookupError('`predict_day` is not in the order history')
return training_ts, frequency, actuals_ts
def make_realtime_ts( # noqa:WPS210
self, pixel_id: int, predict_at: dt.datetime, train_horizon: int,
) -> Tuple[pd.Series, int, pd.Series]:
"""Slice a vertical real-time time series out of the `.totals`.
Create a time series covering `train_horizon` weeks that can be used
for training a forecasting model to predict the demand at `predict_at`.
For explanation of the terms "horizontal", "vertical", and "real-time"
in the context of time series, see section 3.2 in the following paper:
https://github.com/webartifex/urban-meal-delivery-demand-forecasting/blob/main/paper.pdf
Args:
pixel_id: pixel in which the time series is aggregated
predict_at: time step (i.e., "start_at") for which a prediction is made
train_horizon: weeks of historic data used to predict `predict_at`
Returns:
training time series, frequency, actual order count at `predict_at`
Raises:
LookupError: `pixel_id` not in `grid` or `predict_at` not in `.totals`
RuntimeError: desired time series slice is not entirely in `.totals`
"""
try:
intra_pixel = self.totals.loc[pixel_id]
except KeyError:
raise LookupError('The `pixel_id` is not in the `grid`') from None
if predict_at >= config.CUTOFF_DAY: # pragma: no cover
raise RuntimeError('Internal error: cannot predict beyond the given data')
# The first and last training day are just before the `predict_at` day
# and span exactly `train_horizon` weeks covering all times of the day,
# including times on the `predict_at` day that are earlier than `predict_at`.
first_train_day = predict_at.date() - dt.timedelta(weeks=train_horizon)
first_start_at = dt.datetime(
first_train_day.year,
first_train_day.month,
first_train_day.day,
config.SERVICE_START,
0,
)
# Predicting the first time step on the `predict_at` day is a corner case.
# Then, the previous day is indeed the `last_train_day`. Predicting any
# other time step implies that the `predict_at` day is the `last_train_day`.
# `last_train_time` is the last "start_at" before the one being predicted.
if predict_at.hour == config.SERVICE_START:
last_train_day = predict_at.date() - dt.timedelta(days=1)
last_train_time = dt.time(config.SERVICE_END, 0)
else:
last_train_day = predict_at.date()
last_train_time = predict_at.time()
last_start_at = dt.datetime(
last_train_day.year,
last_train_day.month,
last_train_day.day,
last_train_time.hour,
last_train_time.minute,
) - dt.timedelta(minutes=self._time_step)
# The frequency is the number of weekdays times the number of daily time steps.
frequency = 7 * self._n_daily_time_steps
# Take all the counts between `first_train_day` and `last_train_day`,
# including the ones on the `predict_at` day prior to `predict_at`.
training_ts = intra_pixel.loc[
first_start_at:last_start_at, # type:ignore
'n_orders',
]
n_time_steps_on_predict_day = (
(
predict_at
- dt.datetime(
predict_at.year,
predict_at.month,
predict_at.day,
config.SERVICE_START,
0,
)
).seconds
// 60 # -> minutes
// self._time_step
)
if len(training_ts) != frequency * train_horizon + n_time_steps_on_predict_day:
raise RuntimeError('Not enough historic data for `predict_day`')
actuals_ts = intra_pixel.loc[[predict_at], 'n_orders']
if not len(actuals_ts): # pragma: no cover
raise LookupError('`predict_at` is not in the order history')
return training_ts, frequency, actuals_ts
def avg_daily_demand(
self, pixel_id: int, predict_day: dt.date, train_horizon: int,
) -> float:
"""Calculate the average daily demand (ADD) for a `Pixel`.
The ADD is defined as the average number of daily `Order`s in a
`Pixel` within the training horizon preceding the `predict_day`.
The ADD is primarily used for the rule-based heuristic to determine
the best forecasting model for a `Pixel` on the `predict_day`.
Implementation note: To calculate the ADD, the order counts are
generated as a vertical time series. That must be so as we need to
include all time steps of the days before the `predict_day` and
no time step of the latter.
Args:
pixel_id: pixel for which the ADD is calculated
predict_day: following the `train_horizon` on which the ADD is calculated
train_horizon: time horizon over which the ADD is calculated
Returns:
average number of orders per day
"""
training_ts, _, _ = self.make_vertical_ts( # noqa:WPS434
pixel_id=pixel_id, predict_day=predict_day, train_horizon=train_horizon,
)
first_day = training_ts.index.min().date()
last_day = training_ts.index.max().date()
# `+1` as both `first_day` and `last_day` are included.
n_days = (last_day - first_day).days + 1
return round(training_ts.sum() / n_days, 1)
def choose_tactical_model(
self, pixel_id: int, predict_day: dt.date, train_horizon: int,
) -> models.ForecastingModelABC:
"""Choose the most promising forecasting `*Model` for tactical purposes.
The rules are deduced from "Table 1: Top-3 models by ..." in the article
"Real-time demand forecasting for an urban delivery platform", the first
research paper published for this `urban-meal-delivery` project.
According to the research findings in the article "Real-time demand
forecasting for an urban delivery platform", the best model is a function
of the average daily demand (ADD) and the length of the training horizon.
For the paper check:
https://github.com/webartifex/urban-meal-delivery-demand-forecasting/blob/main/paper.pdf
https://www.sciencedirect.com/science/article/pii/S1366554520307936
Args:
pixel_id: pixel for which a forecasting `*Model` is chosen
predict_day: day for which demand is to be predicted with the `*Model`
train_horizon: time horizon available for training the `*Model`
Returns:
most promising forecasting `*Model`
# noqa:DAR401 RuntimeError
""" # noqa:RST215
add = self.avg_daily_demand(
pixel_id=pixel_id, predict_day=predict_day, train_horizon=train_horizon,
)
# For now, we only make forecasts with 7 and 8 weeks
# as the training horizon (note:4f79e8fa).
if train_horizon in {7, 8}:
if add >= 25: # = "high demand"
return models.HorizontalETSModel(order_history=self)
elif add >= 10: # = "medium demand"
return models.HorizontalETSModel(order_history=self)
elif add >= 2.5: # = "low demand"
return models.HorizontalSMAModel(order_history=self)
# = "no demand"
return models.TrivialModel(order_history=self)
raise RuntimeError(
'no rule for the given average daily demand and training horizon',
)

28
src/urban_meal_delivery/init_r.py
View file

@ -1,28 +0,0 @@
"""Initialize the R dependencies.
The purpose of this module is to import all the R packages that are installed
into a sub-folder (see `config.R_LIBS_PATH`) in the project's root directory.
The Jupyter notebook "research/00_r_dependencies.ipynb" can be used to install all
R dependencies on a Ubuntu/Debian based system.
"""
from rpy2.rinterface_lib import callbacks as rcallbacks
from rpy2.robjects import packages as rpackages
# Suppress R's messages to stdout and stderr.
# Source: https://stackoverflow.com/a/63220287
rcallbacks.consolewrite_print = lambda msg: None # pragma: no cover
rcallbacks.consolewrite_warnerror = lambda msg: None # pragma: no cover
# For clarity and convenience, re-raise the error that results from missing R
# dependencies with clearer instructions as to how to deal with it.
try: # noqa:WPS229
rpackages.importr('forecast')
rpackages.importr('zoo')
except rpackages.PackageNotInstalledError: # pragma: no cover
msg = 'See the "research/00_r_dependencies.ipynb" notebook!'
raise rpackages.PackageNotInstalledError(msg) from None

34
tests/config.py
View file

@ -1,34 +0,0 @@
"""Globals used when testing."""
import datetime as dt
from urban_meal_delivery import config
# The day on which most test cases take place.
YEAR, MONTH, DAY = 2016, 7, 1
# The hour when most test cases take place.
NOON = 12
# `START` and `END` constitute a 57-day time span, 8 full weeks plus 1 day.
# That implies a maximum `train_horizon` of `8` as that needs full 7-day weeks.
START = dt.datetime(YEAR, MONTH, DAY, config.SERVICE_START, 0)
_end = START + dt.timedelta(days=56) # `56` as `START` is not included
END = dt.datetime(_end.year, _end.month, _end.day, config.SERVICE_END, 0)
# Default time steps (in minutes), for example, for `OrderHistory` objects.
LONG_TIME_STEP = 60
SHORT_TIME_STEP = 30
TIME_STEPS = (SHORT_TIME_STEP, LONG_TIME_STEP)
# The `frequency` of vertical time series is the number of days in a week, 7,
# times the number of time steps per day. With 12 operating hours (11 am - 11 pm)
# the `frequency`s are 84 and 168 for the `LONG/SHORT_TIME_STEP`s.
VERTICAL_FREQUENCY_LONG = 7 * 12
VERTICAL_FREQUENCY_SHORT = 7 * 24
# Default training horizons, for example, for
# `OrderHistory.make_horizontal_time_series()`.
LONG_TRAIN_HORIZON = 8
SHORT_TRAIN_HORIZON = 2
TRAIN_HORIZONS = (SHORT_TRAIN_HORIZON, LONG_TRAIN_HORIZON)

119
tests/conftest.py
View file

@ -1,119 +0,0 @@
"""Fixtures for testing the entire package.
The ORM related fixtures are placed here too as some integration tests
in the CLI layer need access to the database.
"""
import os
import warnings
import pytest
import sqlalchemy as sa
from alembic import command as migrations_cmd
from alembic import config as migrations_config
from sqlalchemy import orm
from tests.db import fake_data
from urban_meal_delivery import config
from urban_meal_delivery import db
# The TESTING environment variable is set
# in setup.cfg in pytest's config section.
if not os.getenv('TESTING'):
raise RuntimeError('Tests must be executed with TESTING set in the environment')
if not config.TESTING:
raise RuntimeError('The testing configuration was not loaded')
@pytest.fixture(scope='session', params=['all_at_once', 'sequentially'])
def db_connection(request):
"""Create all tables given the ORM models.
The tables are put into a distinct PostgreSQL schema
that is removed after all tests are over.
The database connection used to do that is yielded.
There are two modes for this fixture:
- "all_at_once": build up the tables all at once with MetaData.create_all()
- "sequentially": build up the tables sequentially with `alembic upgrade head`
This ensures that Alembic's migration files are consistent.
"""
# We need a fresh database connection for each of the two `params`.
# Otherwise, the first test of the parameter run second will fail.
engine = sa.create_engine(config.DATABASE_URI)
connection = engine.connect()
# Monkey patch the package's global `engine` and `connection` objects,
# just in case if it is used somewhere in the code base.
db.engine = engine
db.connection = connection
if request.param == 'all_at_once':
connection.execute(f'CREATE SCHEMA {config.CLEAN_SCHEMA};')
db.Base.metadata.create_all(connection)
else:
cfg = migrations_config.Config('alembic.ini')
migrations_cmd.upgrade(cfg, 'head')
try:
yield connection
finally:
connection.execute(f'DROP SCHEMA {config.CLEAN_SCHEMA} CASCADE;')
if request.param == 'sequentially':
tmp_alembic_version = f'{config.ALEMBIC_TABLE}_{config.CLEAN_SCHEMA}'
connection.execute(
f'DROP TABLE {config.ALEMBIC_TABLE_SCHEMA}.{tmp_alembic_version};',
)
connection.close()
@pytest.fixture
def db_session(db_connection):
"""A SQLAlchemy session that rolls back everything after a test case."""
# Begin the outermost transaction
# that is rolled back at the end of each test case.
transaction = db_connection.begin()
# Create a session bound to the same connection as the `transaction`.
# Using any other session would not result in the roll back.
session = orm.sessionmaker()(bind=db_connection)
# Monkey patch the package's global `session` object,
# which is used heavily in the code base.
db.session = session
try:
yield session
finally:
session.close()
with warnings.catch_warnings(record=True):
transaction.rollback()
# Import the fixtures from the `fake_data` sub-package.
make_address = fake_data.make_address
make_courier = fake_data.make_courier
make_customer = fake_data.make_customer
make_order = fake_data.make_order
make_restaurant = fake_data.make_restaurant
address = fake_data.address
city = fake_data.city
city_data = fake_data.city_data
courier = fake_data.courier
customer = fake_data.customer
order = fake_data.order
restaurant = fake_data.restaurant
grid = fake_data.grid
pixel = fake_data.pixel

5
tests/console/__init__.py
View file

@ -1,5 +0,0 @@
"""Test the CLI scripts in the urban-meal-delivery package.
Some tests require a database. Therefore, the corresponding code is excluded
from coverage reporting with "pragma: no cover" (grep:b1f68d24).
"""

10
tests/console/conftest.py
View file

@ -1,10 +0,0 @@
"""Fixture for testing the CLI scripts."""
import pytest
from click import testing as click_testing
@pytest.fixture
def cli() -> click_testing.CliRunner:
"""Initialize Click's CLI Test Runner."""
return click_testing.CliRunner()

48
tests/console/test_gridify.py
View file

@ -1,48 +0,0 @@
"""Integration test for the `urban_meal_delivery.console.gridify` module."""
import pytest
import urban_meal_delivery
from urban_meal_delivery import db
from urban_meal_delivery.console import gridify
@pytest.mark.db
def test_two_pixels_with_two_addresses( # noqa:WPS211
cli, db_session, monkeypatch, city, make_address, make_restaurant, make_order,
):
"""Two `Address` objects in distinct `Pixel` objects.
This is roughly the same test case as
`tests.db.test_grids.test_two_pixels_with_two_addresses`.
The difference is that the result is written to the database.
"""
# Create two `Address` objects in distinct `Pixel`s.
# One `Address` in the lower-left `Pixel`, ...
address1 = make_address(latitude=48.8357377, longitude=2.2517412)
# ... and another one in the upper-right one.
address2 = make_address(latitude=48.8898312, longitude=2.4357622)
# Locate a `Restaurant` at the two `Address` objects and
# place one `Order` for each of them so that the `Address`
# objects are used as `Order.pickup_address`s.
restaurant1 = make_restaurant(address=address1)
restaurant2 = make_restaurant(address=address2)
order1 = make_order(restaurant=restaurant1)
order2 = make_order(restaurant=restaurant2)
db_session.add(order1)
db_session.add(order2)
db_session.commit()
side_length = max(city.total_x // 2, city.total_y // 2) + 1
# Hack the configuration regarding the grids to be created.
monkeypatch.setattr(urban_meal_delivery.config, 'GRID_SIDE_LENGTHS', [side_length])
result = cli.invoke(gridify.gridify)
assert result.exit_code == 0
assert db_session.query(db.Grid).count() == 1
assert db_session.query(db.Pixel).count() == 2

1
tests/db/__init__.py
View file

@ -1 +0,0 @@
"""Test the ORM layer."""

16
tests/db/fake_data/__init__.py
View file

@ -1,16 +0,0 @@
"""Fixtures for testing the ORM layer with fake data."""
from tests.db.fake_data.fixture_makers import make_address
from tests.db.fake_data.fixture_makers import make_courier
from tests.db.fake_data.fixture_makers import make_customer
from tests.db.fake_data.fixture_makers import make_order
from tests.db.fake_data.fixture_makers import make_restaurant
from tests.db.fake_data.static_fixtures import address
from tests.db.fake_data.static_fixtures import city
from tests.db.fake_data.static_fixtures import city_data
from tests.db.fake_data.static_fixtures import courier
from tests.db.fake_data.static_fixtures import customer
from tests.db.fake_data.static_fixtures import grid
from tests.db.fake_data.static_fixtures import order
from tests.db.fake_data.static_fixtures import pixel
from tests.db.fake_data.static_fixtures import restaurant

378
tests/db/fake_data/factories.py
View file

@ -1,378 +0,0 @@
"""Factories to create instances for the SQLAlchemy models."""
import datetime as dt
import random
import string
import factory
import faker
from factory import alchemy
from geopy import distance
from tests import config as test_config
from urban_meal_delivery import db
def _random_timespan( # noqa:WPS211
*,
min_hours=0,
min_minutes=0,
min_seconds=0,
max_hours=0,
max_minutes=0,
max_seconds=0,
):
"""A randomized `timedelta` object between the specified arguments."""
total_min_seconds = min_hours * 3600 + min_minutes * 60 + min_seconds
total_max_seconds = max_hours * 3600 + max_minutes * 60 + max_seconds
return dt.timedelta(seconds=random.randint(total_min_seconds, total_max_seconds))
def _early_in_the_morning():
"""A randomized `datetime` object early in the morning."""
early = dt.datetime(test_config.YEAR, test_config.MONTH, test_config.DAY, 3, 0)
return early + _random_timespan(max_hours=2)
class AddressFactory(alchemy.SQLAlchemyModelFactory):
"""Create instances of the `db.Address` model."""
class Meta:
model = db.Address
sqlalchemy_get_or_create = ('id',)
id = factory.Sequence(lambda num: num) # noqa:WPS125
created_at = factory.LazyFunction(_early_in_the_morning)
# When testing, all addresses are considered primary ones.
# As non-primary addresses have no different behavior and
# the property is only kept from the original dataset for
# completeness sake, that is ok to do.
primary_id = factory.LazyAttribute(lambda obj: obj.id)
# Mimic a Google Maps Place ID with just random characters.
place_id = factory.LazyFunction(
lambda: ''.join(random.choice(string.ascii_lowercase) for _ in range(20)),
)
# Place the addresses somewhere in downtown Paris.
latitude = factory.Faker('coordinate', center=48.855, radius=0.01)
longitude = factory.Faker('coordinate', center=2.34, radius=0.03)
# city -> set by the `make_address` fixture as there is only one `city`
city_name = 'Paris'
zip_code = factory.LazyFunction(lambda: random.randint(75001, 75020))
street = factory.Faker('street_address', locale='fr_FR')
class CourierFactory(alchemy.SQLAlchemyModelFactory):
"""Create instances of the `db.Courier` model."""
class Meta:
model = db.Courier
sqlalchemy_get_or_create = ('id',)
id = factory.Sequence(lambda num: num) # noqa:WPS125
created_at = factory.LazyFunction(_early_in_the_morning)
vehicle = 'bicycle'
historic_speed = 7.89
capacity = 100
pay_per_hour = 750
pay_per_order = 200
class CustomerFactory(alchemy.SQLAlchemyModelFactory):
"""Create instances of the `db.Customer` model."""
class Meta:
model = db.Customer
sqlalchemy_get_or_create = ('id',)
id = factory.Sequence(lambda num: num) # noqa:WPS125
_restaurant_names = faker.Faker()
class RestaurantFactory(alchemy.SQLAlchemyModelFactory):
"""Create instances of the `db.Restaurant` model."""
class Meta:
model = db.Restaurant
sqlalchemy_get_or_create = ('id',)
id = factory.Sequence(lambda num: num) # noqa:WPS125
created_at = factory.LazyFunction(_early_in_the_morning)
name = factory.LazyFunction(
lambda: f"{_restaurant_names.first_name()}'s Restaurant",
)
# address -> set by the `make_restaurant` fixture as there is only one `city`
estimated_prep_duration = 1000
class AdHocOrderFactory(alchemy.SQLAlchemyModelFactory):
"""Create instances of the `db.Order` model.
This factory creates ad-hoc `Order`s while the `ScheduledOrderFactory`
below creates pre-orders. They are split into two classes mainly
because the logic regarding how the timestamps are calculated from
each other differs.
See the docstring in the contained `Params` class for
flags to adapt how the `Order` is created.
"""
class Meta:
model = db.Order
sqlalchemy_get_or_create = ('id',)
class Params:
"""Define flags that overwrite some attributes.
The `factory.Trait` objects in this class are executed after all
the normal attributes in the `OrderFactory` classes were evaluated.
Flags:
cancel_before_pickup
cancel_after_pickup
"""
# Timestamps after `cancelled_at` are discarded
# by the `post_generation` hook at the end of the `OrderFactory`.
cancel_ = factory.Trait( # noqa:WPS120 -> leading underscore does not work
cancelled=True, cancelled_at_corrected=False,
)
cancel_before_pickup = factory.Trait(
cancel_=True,
cancelled_at=factory.LazyAttribute(
lambda obj: obj.dispatch_at
+ _random_timespan(
max_seconds=(obj.pickup_at - obj.dispatch_at).total_seconds(),
),
),
)
cancel_after_pickup = factory.Trait(
cancel_=True,
cancelled_at=factory.LazyAttribute(
lambda obj: obj.pickup_at
+ _random_timespan(
max_seconds=(obj.delivery_at - obj.pickup_at).total_seconds(),
),
),
)
# Generic attributes
id = factory.Sequence(lambda num: num) # noqa:WPS125
# customer -> set by the `make_order` fixture for better control
# Attributes regarding the specialization of an `Order`: ad-hoc or scheduled.
# Ad-hoc `Order`s are placed between 11.45 and 14.15.
placed_at = factory.LazyFunction(
lambda: dt.datetime(
test_config.YEAR, test_config.MONTH, test_config.DAY, 11, 45,
)
+ _random_timespan(max_hours=2, max_minutes=30),
)
ad_hoc = True
scheduled_delivery_at = None
scheduled_delivery_at_corrected = None
# Without statistical info, we assume an ad-hoc `Order` delivered after 45 minutes.
first_estimated_delivery_at = factory.LazyAttribute(
lambda obj: obj.placed_at + dt.timedelta(minutes=45),
)
# Attributes regarding the cancellation of an `Order`.
# May be overwritten with the `cancel_before_pickup` or `cancel_after_pickup` flags.
cancelled = False
cancelled_at = None
cancelled_at_corrected = None
# Price-related attributes -> sample realistic prices
sub_total = factory.LazyFunction(lambda: 100 * random.randint(15, 25))
delivery_fee = 250
total = factory.LazyAttribute(lambda obj: obj.sub_total + obj.delivery_fee)
# Restaurant-related attributes
# restaurant -> set by the `make_order` fixture for better control
restaurant_notified_at = factory.LazyAttribute(
lambda obj: obj.placed_at + _random_timespan(min_seconds=30, max_seconds=90),
)
restaurant_notified_at_corrected = False
restaurant_confirmed_at = factory.LazyAttribute(
lambda obj: obj.restaurant_notified_at
+ _random_timespan(min_seconds=30, max_seconds=150),
)
restaurant_confirmed_at_corrected = False
# Use the database defaults of the historic data.
estimated_prep_duration = 900
estimated_prep_duration_corrected = False
estimated_prep_buffer = 480
# Dispatch-related columns
# courier -> set by the `make_order` fixture for better control
dispatch_at = factory.LazyAttribute(
lambda obj: obj.placed_at + _random_timespan(min_seconds=600, max_seconds=1080),
)
dispatch_at_corrected = False
courier_notified_at = factory.LazyAttribute(
lambda obj: obj.dispatch_at
+ _random_timespan(min_seconds=100, max_seconds=140),
)
courier_notified_at_corrected = False
courier_accepted_at = factory.LazyAttribute(
lambda obj: obj.courier_notified_at
+ _random_timespan(min_seconds=15, max_seconds=45),
)
courier_accepted_at_corrected = False
# Sample a realistic utilization.
utilization = factory.LazyFunction(lambda: random.choice([50, 60, 70, 80, 90, 100]))
# Pickup-related attributes
# pickup_address -> aligned with `restaurant.address` by the `make_order` fixture
reached_pickup_at = factory.LazyAttribute(
lambda obj: obj.courier_accepted_at
+ _random_timespan(min_seconds=300, max_seconds=600),
)
pickup_at = factory.LazyAttribute(
lambda obj: obj.reached_pickup_at
+ _random_timespan(min_seconds=120, max_seconds=600),
)
pickup_at_corrected = False
pickup_not_confirmed = False
left_pickup_at = factory.LazyAttribute(
lambda obj: obj.pickup_at + _random_timespan(min_seconds=60, max_seconds=180),
)
left_pickup_at_corrected = False
# Delivery-related attributes
# delivery_address -> set by the `make_order` fixture as there is only one `city`
reached_delivery_at = factory.LazyAttribute(
lambda obj: obj.left_pickup_at
+ _random_timespan(min_seconds=240, max_seconds=480),
)
delivery_at = factory.LazyAttribute(
lambda obj: obj.reached_delivery_at
+ _random_timespan(min_seconds=240, max_seconds=660),
)
delivery_at_corrected = False
delivery_not_confirmed = False
_courier_waited_at_delivery = factory.LazyAttribute(
lambda obj: False if obj.delivery_at else None,
)
# Statistical attributes -> calculate realistic stats
logged_delivery_distance = factory.LazyAttribute(
lambda obj: distance.great_circle( # noqa:WPS317
(obj.pickup_address.latitude, obj.pickup_address.longitude),
(obj.delivery_address.latitude, obj.delivery_address.longitude),
).meters,
)
logged_avg_speed = factory.LazyAttribute( # noqa:ECE001
lambda obj: round(
(
obj.logged_avg_speed_distance
/ (obj.delivery_at - obj.pickup_at).total_seconds()
),
2,
),
)
logged_avg_speed_distance = factory.LazyAttribute(
lambda obj: 0.95 * obj.logged_delivery_distance,
)
@factory.post_generation
def post( # noqa:C901,WPS231
obj, create, extracted, **kwargs, # noqa:B902,N805
):
"""Discard timestamps that occur after cancellation."""
if obj.cancelled:
if obj.cancelled_at <= obj.restaurant_notified_at:
obj.restaurant_notified_at = None
obj.restaurant_notified_at_corrected = None
if obj.cancelled_at <= obj.restaurant_confirmed_at:
obj.restaurant_confirmed_at = None
obj.restaurant_confirmed_at_corrected = None
if obj.cancelled_at <= obj.dispatch_at:
obj.dispatch_at = None
obj.dispatch_at_corrected = None
if obj.cancelled_at <= obj.courier_notified_at:
obj.courier_notified_at = None
obj.courier_notified_at_corrected = None
if obj.cancelled_at <= obj.courier_accepted_at:
obj.courier_accepted_at = None
obj.courier_accepted_at_corrected = None
if obj.cancelled_at <= obj.reached_pickup_at:
obj.reached_pickup_at = None
if obj.cancelled_at <= obj.pickup_at:
obj.pickup_at = None
obj.pickup_at_corrected = None
obj.pickup_not_confirmed = None
if obj.cancelled_at <= obj.left_pickup_at:
obj.left_pickup_at = None
obj.left_pickup_at_corrected = None
if obj.cancelled_at <= obj.reached_delivery_at:
obj.reached_delivery_at = None
if obj.cancelled_at <= obj.delivery_at:
obj.delivery_at = None
obj.delivery_at_corrected = None
obj.delivery_not_confirmed = None
obj._courier_waited_at_delivery = None
class ScheduledOrderFactory(AdHocOrderFactory):
"""Create instances of the `db.Order` model.
This class takes care of the various timestamps for pre-orders.
Pre-orders are placed long before the test day's lunch time starts.
All timestamps are relative to either `.dispatch_at` or `.restaurant_notified_at`
and calculated backwards from `.scheduled_delivery_at`.
"""
# Attributes regarding the specialization of an `Order`: ad-hoc or scheduled.
placed_at = factory.LazyFunction(_early_in_the_morning)
ad_hoc = False
# Discrete `datetime` objects in the "core" lunch time are enough.
scheduled_delivery_at = factory.LazyFunction(
lambda: random.choice(
[
dt.datetime(
test_config.YEAR, test_config.MONTH, test_config.DAY, 12, 0,
),
dt.datetime(
test_config.YEAR, test_config.MONTH, test_config.DAY, 12, 15,
),
dt.datetime(
test_config.YEAR, test_config.MONTH, test_config.DAY, 12, 30,
),
dt.datetime(
test_config.YEAR, test_config.MONTH, test_config.DAY, 12, 45,
),
dt.datetime(
test_config.YEAR, test_config.MONTH, test_config.DAY, 13, 0,
),
dt.datetime(
test_config.YEAR, test_config.MONTH, test_config.DAY, 13, 15,
),
dt.datetime(
test_config.YEAR, test_config.MONTH, test_config.DAY, 13, 30,
),
],
),
)
scheduled_delivery_at_corrected = False
# Assume the `Order` is on time.
first_estimated_delivery_at = factory.LazyAttribute(
lambda obj: obj.scheduled_delivery_at,
)
# Restaurant-related attributes
restaurant_notified_at = factory.LazyAttribute(
lambda obj: obj.scheduled_delivery_at
- _random_timespan(min_minutes=45, max_minutes=50),
)
# Dispatch-related attributes
dispatch_at = factory.LazyAttribute(
lambda obj: obj.scheduled_delivery_at
- _random_timespan(min_minutes=40, max_minutes=45),
)

105
tests/db/fake_data/fixture_makers.py
View file

@ -1,105 +0,0 @@
"""Fixture factories for testing the ORM layer with fake data."""
import pytest
from tests.db.fake_data import factories
@pytest.fixture
def make_address(city):
"""Replaces `AddressFactory.build()`: Create an `Address` in the `city`."""
# Reset the identifiers before every test.
factories.AddressFactory.reset_sequence(1)
def func(**kwargs):
"""Create an `Address` object in the `city`."""
return factories.AddressFactory.build(city=city, **kwargs)
return func
@pytest.fixture
def make_courier():
"""Replaces `CourierFactory.build()`: Create a `Courier`."""
# Reset the identifiers before every test.
factories.CourierFactory.reset_sequence(1)
def func(**kwargs):
"""Create a new `Courier` object."""
return factories.CourierFactory.build(**kwargs)
return func
@pytest.fixture
def make_customer():
"""Replaces `CustomerFactory.build()`: Create a `Customer`."""
# Reset the identifiers before every test.
factories.CustomerFactory.reset_sequence(1)
def func(**kwargs):
"""Create a new `Customer` object."""
return factories.CustomerFactory.build(**kwargs)
return func
@pytest.fixture
def make_restaurant(make_address):
"""Replaces `RestaurantFactory.build()`: Create a `Restaurant`."""
# Reset the identifiers before every test.
factories.RestaurantFactory.reset_sequence(1)
def func(address=None, **kwargs):
"""Create a new `Restaurant` object.
If no `address` is provided, a new `Address` is created.
"""
if address is None:
address = make_address()
return factories.RestaurantFactory.build(address=address, **kwargs)
return func
@pytest.fixture
def make_order(make_address, make_courier, make_customer, make_restaurant):
"""Replaces `OrderFactory.build()`: Create a `Order`."""
# Reset the identifiers before every test.
factories.AdHocOrderFactory.reset_sequence(1)
def func(scheduled=False, restaurant=None, courier=None, **kwargs):
"""Create a new `Order` object.
Each `Order` is made by a new `Customer` with a unique `Address` for delivery.
Args:
scheduled: if an `Order` is a pre-order
restaurant: who receives the `Order`; defaults to a new `Restaurant`
courier: who delivered the `Order`; defaults to a new `Courier`
kwargs: additional keyword arguments forwarded to the `OrderFactory`
Returns:
order
"""
if scheduled:
factory_cls = factories.ScheduledOrderFactory
else:
factory_cls = factories.AdHocOrderFactory
if restaurant is None:
restaurant = make_restaurant()
if courier is None:
courier = make_courier()
return factory_cls.build(
customer=make_customer(), # assume a unique `Customer` per order
restaurant=restaurant,
courier=courier,
pickup_address=restaurant.address, # no `Address` history
delivery_address=make_address(), # unique `Customer` => new `Address`
**kwargs,
)
return func

70
tests/db/fake_data/static_fixtures.py
View file

@ -1,70 +0,0 @@
"""Fake data for testing the ORM layer."""
import pytest
from urban_meal_delivery import db
@pytest.fixture
def city_data():
"""The data for the one and only `City` object as a `dict`."""
return {
'id': 1,
'name': 'Paris',
'kml': "<?xml version='1.0' encoding='UTF-8'?> ...",
'center_latitude': 48.856614,
'center_longitude': 2.3522219,
'northeast_latitude': 48.9021449,
'northeast_longitude': 2.4699208,
'southwest_latitude': 48.815573,
'southwest_longitude': 2.225193,
'initial_zoom': 12,
}
@pytest.fixture
def city(city_data):
"""The one and only `City` object."""
return db.City(**city_data)
@pytest.fixture
def address(make_address):
"""An `Address` object in the `city`."""
return make_address()
@pytest.fixture
def courier(make_courier):
"""A `Courier` object."""
return make_courier()
@pytest.fixture
def customer(make_customer):
"""A `Customer` object."""
return make_customer()
@pytest.fixture
def restaurant(address, make_restaurant):
"""A `Restaurant` object located at the `address`."""
return make_restaurant(address=address)
@pytest.fixture
def order(make_order, restaurant):
"""An `Order` object for the `restaurant`."""
return make_order(restaurant=restaurant)
@pytest.fixture
def grid(city):
"""A `Grid` with a pixel area of 1 square kilometer."""
return db.Grid(city=city, side_length=1000)
@pytest.fixture
def pixel(grid):
"""The `Pixel` in the lower-left corner of the `grid`."""
return db.Pixel(id=1, grid=grid, n_x=0, n_y=0)

154
tests/db/test_addresses.py
View file

@ -1,154 +0,0 @@
"""Test the ORM's `Address` model."""
import pytest
import sqlalchemy as sqla
from sqlalchemy import exc as sa_exc
from urban_meal_delivery import db
from urban_meal_delivery.db import utils
class TestSpecialMethods:
"""Test special methods in `Address`."""
def test_create_address(self, address):
"""Test instantiation of a new `Address` object."""
assert address is not None
def test_text_representation(self, address):
"""`Address` has a non-literal text representation."""
result = repr(address)
assert result == f'<Address({address.street} in {address.city_name})>'
@pytest.mark.db
@pytest.mark.no_cover
class TestConstraints:
"""Test the database constraints defined in `Address`."""
def test_insert_into_database(self, db_session, address):
"""Insert an instance into the (empty) database."""
assert db_session.query(db.Address).count() == 0
db_session.add(address)
db_session.commit()
assert db_session.query(db.Address).count() == 1
def test_delete_a_referenced_address(self, db_session, address, make_address):
"""Remove a record that is referenced with a FK."""
db_session.add(address)
# Fake another_address that has the same `.primary_id` as `address`.
db_session.add(make_address(primary_id=address.id))
db_session.commit()
db_session.delete(address)
with pytest.raises(
sa_exc.IntegrityError, match='fk_addresses_to_addresses_via_primary_id',
):
db_session.commit()
def test_delete_a_referenced_city(self, db_session, address):
"""Remove a record that is referenced with a FK."""
db_session.add(address)
db_session.commit()
# Must delete without ORM as otherwise an UPDATE statement is emitted.
stmt = sqla.delete(db.City).where(db.City.id == address.city.id)
with pytest.raises(
sa_exc.IntegrityError, match='fk_addresses_to_cities_via_city_id',
):
db_session.execute(stmt)
@pytest.mark.parametrize('latitude', [-91, 91])
def test_invalid_latitude(self, db_session, address, latitude):
"""Insert an instance with invalid data."""
address.latitude = latitude
db_session.add(address)
with pytest.raises(
sa_exc.IntegrityError, match='latitude_between_90_degrees',
):
db_session.commit()
@pytest.mark.parametrize('longitude', [-181, 181])
def test_invalid_longitude(self, db_session, address, longitude):
"""Insert an instance with invalid data."""
address.longitude = longitude
db_session.add(address)
with pytest.raises(
sa_exc.IntegrityError, match='longitude_between_180_degrees',
):
db_session.commit()
@pytest.mark.parametrize('zip_code', [-1, 0, 9999, 100000])
def test_invalid_zip_code(self, db_session, address, zip_code):
"""Insert an instance with invalid data."""
address.zip_code = zip_code
db_session.add(address)
with pytest.raises(sa_exc.IntegrityError, match='valid_zip_code'):
db_session.commit()
@pytest.mark.parametrize('floor', [-1, 41])
def test_invalid_floor(self, db_session, address, floor):
"""Insert an instance with invalid data."""
address.floor = floor
db_session.add(address)
with pytest.raises(sa_exc.IntegrityError, match='realistic_floor'):
db_session.commit()
class TestProperties:
"""Test properties in `Address`."""
def test_is_primary(self, address):
"""Test `Address.is_primary` property."""
assert address.id == address.primary_id
result = address.is_primary
assert result is True
def test_is_not_primary(self, address):
"""Test `Address.is_primary` property."""
address.primary_id = 999
result = address.is_primary
assert result is False
def test_location(self, address):
"""Test `Address.location` property."""
latitude = float(address.latitude)
longitude = float(address.longitude)
result = address.location
assert isinstance(result, utils.Location)
assert result.latitude == pytest.approx(latitude)
assert result.longitude == pytest.approx(longitude)
def test_location_is_cached(self, address):
"""Test `Address.location` property."""
result1 = address.location
result2 = address.location
assert result1 is result2
def test_x_is_positive(self, address):
"""Test `Address.x` property."""
result = address.x
assert result > 0
def test_y_is_positive(self, address):
"""Test `Address.y` property."""
result = address.y
assert result > 0

682
tests/db/test_addresses_addresses.py
View file

@ -1,682 +0,0 @@
"""Test the ORM's `Path` model."""
import json
import googlemaps
import pytest
import sqlalchemy as sqla
from geopy import distance
from sqlalchemy import exc as sa_exc
from urban_meal_delivery import db
from urban_meal_delivery.db import utils
@pytest.fixture
def another_address(make_address):
"""Another `Address` object in the `city`."""
return make_address()
@pytest.fixture
def path(address, another_address, make_address):
"""A `Path` from `address` to `another_address`."""
air_distance = distance.great_circle( # noqa:WPS317
address.location.lat_lng, another_address.location.lat_lng,
).meters
# We put 5 latitude-longitude pairs as the "path" from
# `.first_address` to `.second_address`.
directions = json.dumps(
[
(float(add.latitude), float(add.longitude))
for add in (make_address() for _ in range(5)) # noqa:WPS335
],
)
return db.Path(
first_address=address,
second_address=another_address,
air_distance=round(air_distance),
bicycle_distance=round(1.25 * air_distance),
bicycle_duration=300,
_directions=directions,
)
class TestSpecialMethods:
"""Test special methods in `Path`."""
def test_create_an_address_address_association(self, path):
"""Test instantiation of a new `Path` object."""
assert path is not None
@pytest.mark.db
@pytest.mark.no_cover
class TestConstraints:
"""Test the database constraints defined in `Path`."""
def test_insert_into_database(self, db_session, path):
"""Insert an instance into the (empty) database."""
assert db_session.query(db.Path).count() == 0
db_session.add(path)
db_session.commit()
assert db_session.query(db.Path).count() == 1
def test_delete_a_referenced_first_address(self, db_session, path):
"""Remove a record that is referenced with a FK."""
db_session.add(path)
db_session.commit()
# Must delete without ORM as otherwise an UPDATE statement is emitted.
stmt = sqla.delete(db.Address).where(db.Address.id == path.first_address.id)
with pytest.raises(
sa_exc.IntegrityError,
match='fk_addresses_addresses_to_addresses_via_first_address', # shortened
):
db_session.execute(stmt)
def test_delete_a_referenced_second_address(self, db_session, path):
"""Remove a record that is referenced with a FK."""
db_session.add(path)
db_session.commit()
# Must delete without ORM as otherwise an UPDATE statement is emitted.
stmt = sqla.delete(db.Address).where(db.Address.id == path.second_address.id)
with pytest.raises(
sa_exc.IntegrityError,
match='fk_addresses_addresses_to_addresses_via_second_address', # shortened
):
db_session.execute(stmt)
def test_reference_an_invalid_city(self, db_session, address, another_address):
"""Insert a record with an invalid foreign key."""
db_session.add(address)
db_session.add(another_address)
db_session.commit()
# Must insert without ORM as otherwise SQLAlchemy figures out
# that something is wrong before any query is sent to the database.
stmt = sqla.insert(db.Path).values(
first_address_id=address.id,
second_address_id=another_address.id,
city_id=999,
air_distance=123,
)
with pytest.raises(
sa_exc.IntegrityError,
match='fk_addresses_addresses_to_addresses_via_first_address', # shortened
):
db_session.execute(stmt)
def test_redundant_addresses(self, db_session, path):
"""Insert a record that violates a unique constraint."""
db_session.add(path)
db_session.commit()
# Must insert without ORM as otherwise SQLAlchemy figures out
# that something is wrong before any query is sent to the database.
stmt = sqla.insert(db.Path).values(
first_address_id=path.first_address.id,
second_address_id=path.second_address.id,
city_id=path.city_id,
air_distance=path.air_distance,
)
with pytest.raises(sa_exc.IntegrityError, match='duplicate key value'):
db_session.execute(stmt)
def test_symmetric_addresses(self, db_session, path):
"""Insert a record that violates a check constraint."""
db_session.add(path)
db_session.commit()
another_path = db.Path(
first_address=path.second_address,
second_address=path.first_address,
air_distance=path.air_distance,
)
db_session.add(another_path)
with pytest.raises(
sa_exc.IntegrityError,
match='ck_addresses_addresses_on_distances_are_symmetric_for_bicycles',
):
db_session.commit()
def test_negative_air_distance(self, db_session, path):
"""Insert an instance with invalid data."""
path.air_distance = -1
db_session.add(path)
with pytest.raises(sa_exc.IntegrityError, match='realistic_air_distance'):
db_session.commit()
def test_air_distance_too_large(self, db_session, path):
"""Insert an instance with invalid data."""
path.air_distance = 20_000
path.bicycle_distance = 21_000
db_session.add(path)
with pytest.raises(sa_exc.IntegrityError, match='realistic_air_distance'):
db_session.commit()
def test_bicycle_distance_too_large(self, db_session, path):
"""Insert an instance with invalid data."""
path.bicycle_distance = 25_000
db_session.add(path)
with pytest.raises(sa_exc.IntegrityError, match='realistic_bicycle_distance'):
db_session.commit()
def test_air_distance_shorter_than_bicycle_distance(self, db_session, path):
"""Insert an instance with invalid data."""
path.bicycle_distance = round(0.75 * path.air_distance)
db_session.add(path)
with pytest.raises(sa_exc.IntegrityError, match='air_distance_is_shortest'):
db_session.commit()
@pytest.mark.parametrize('duration', [-1, 3601])
def test_unrealistic_bicycle_travel_time(self, db_session, path, duration):
"""Insert an instance with invalid data."""
path.bicycle_duration = duration
db_session.add(path)
with pytest.raises(
sa_exc.IntegrityError, match='realistic_bicycle_travel_time',
):
db_session.commit()
@pytest.mark.db
class TestFromAddresses:
"""Test the alternative constructor `Path.from_addresses()`.
Includes tests for the convenience method `Path.from_order()`,
which redirects to `Path.from_addresses()`.
"""
@pytest.fixture
def _prepare_db(self, db_session, address):
"""Put the `address` into the database.
`Address`es must be in the database as otherwise the `.city_id` column
cannot be resolved in `Path.from_addresses()`.
"""
db_session.add(address)
@pytest.mark.usefixtures('_prepare_db')
def test_make_path_instance(
self, db_session, address, another_address,
):
"""Test instantiation of a new `Path` instance."""
assert db_session.query(db.Path).count() == 0
db.Path.from_addresses(address, another_address)
assert db_session.query(db.Path).count() == 1
@pytest.mark.usefixtures('_prepare_db')
def test_make_the_same_path_instance_twice(
self, db_session, address, another_address,
):
"""Test instantiation of a new `Path` instance."""
assert db_session.query(db.Path).count() == 0
db.Path.from_addresses(address, another_address)
assert db_session.query(db.Path).count() == 1
db.Path.from_addresses(another_address, address)
assert db_session.query(db.Path).count() == 1
@pytest.mark.usefixtures('_prepare_db')
def test_structure_of_return_value(self, db_session, address, another_address):
"""Test instantiation of a new `Path` instance."""
results = db.Path.from_addresses(address, another_address)
assert isinstance(results, list)
@pytest.mark.usefixtures('_prepare_db')
def test_instances_must_have_air_distance(
self, db_session, address, another_address,
):
"""Test instantiation of a new `Path` instance."""
paths = db.Path.from_addresses(address, another_address)
result = paths[0]
assert result.air_distance is not None
@pytest.mark.usefixtures('_prepare_db')
def test_do_not_sync_instances_with_google_maps(
self, db_session, address, another_address,
):
"""Test instantiation of a new `Path` instance."""
paths = db.Path.from_addresses(address, another_address)
result = paths[0]
assert result.bicycle_distance is None
assert result.bicycle_duration is None
@pytest.mark.usefixtures('_prepare_db')
def test_sync_instances_with_google_maps(
self, db_session, address, another_address, monkeypatch,
):
"""Test instantiation of a new `Path` instance."""
def sync(self):
self.bicycle_distance = 1.25 * self.air_distance
self.bicycle_duration = 300
monkeypatch.setattr(db.Path, 'sync_with_google_maps', sync)
paths = db.Path.from_addresses(address, another_address, google_maps=True)
result = paths[0]
assert result.bicycle_distance is not None
assert result.bicycle_duration is not None
@pytest.mark.usefixtures('_prepare_db')
def test_one_path_for_two_addresses(self, db_session, address, another_address):
"""Test instantiation of a new `Path` instance."""
result = len(db.Path.from_addresses(address, another_address))
assert result == 1
@pytest.mark.usefixtures('_prepare_db')
def test_two_paths_for_three_addresses(self, db_session, make_address):
"""Test instantiation of a new `Path` instance."""
result = len(db.Path.from_addresses(*[make_address() for _ in range(3)]))
assert result == 3
@pytest.mark.usefixtures('_prepare_db')
def test_six_paths_for_four_addresses(self, db_session, make_address):
"""Test instantiation of a new `Path` instance."""
result = len(db.Path.from_addresses(*[make_address() for _ in range(4)]))
assert result == 6
# Tests for the `Path.from_order()` convenience method.
@pytest.mark.usefixtures('_prepare_db')
def test_make_path_instance_from_order(
self, db_session, order,
):
"""Test instantiation of a new `Path` instance."""
assert db_session.query(db.Path).count() == 0
db.Path.from_order(order)
assert db_session.query(db.Path).count() == 1
@pytest.mark.usefixtures('_prepare_db')
def test_make_the_same_path_instance_from_order_twice(
self, db_session, order,
):
"""Test instantiation of a new `Path` instance."""
assert db_session.query(db.Path).count() == 0
db.Path.from_order(order)
assert db_session.query(db.Path).count() == 1
db.Path.from_order(order)
assert db_session.query(db.Path).count() == 1
@pytest.mark.usefixtures('_prepare_db')
def test_structure_of_return_value_from_order(self, db_session, order):
"""Test instantiation of a new `Path` instance."""
result = db.Path.from_order(order)
assert isinstance(result, db.Path)
@pytest.mark.usefixtures('_prepare_db')
def test_sync_instance_from_order_with_google_maps(
self, db_session, order, monkeypatch,
):
"""Test instantiation of a new `Path` instance."""
def sync(self):
self.bicycle_distance = 1.25 * self.air_distance
self.bicycle_duration = 300
monkeypatch.setattr(db.Path, 'sync_with_google_maps', sync)
result = db.Path.from_order(order, google_maps=True)
assert result.bicycle_distance is not None
assert result.bicycle_duration is not None
@pytest.mark.db
class TestSyncWithGoogleMaps:
"""Test the `Path.sync_with_google_maps()` method."""
@pytest.fixture
def api_response(self):
"""A typical (shortened) response by the Google Maps Directions API."""
return [ # noqa:ECE001
{
'bounds': {
'northeast': {'lat': 44.8554284, 'lng': -0.5652398},
'southwest': {'lat': 44.8342256, 'lng': -0.5708206},
},
'copyrights': 'Map data ©2021',
'legs': [
{
'distance': {'text': '3.0 km', 'value': 2999},
'duration': {'text': '10 mins', 'value': 596},
'end_address': '13 Place Paul et Jean Paul Avisseau, ...',
'end_location': {'lat': 44.85540839999999, 'lng': -0.5672105},
'start_address': '59 Rue Saint-François, 33000 Bordeaux, ...',
'start_location': {'lat': 44.8342256, 'lng': -0.570372},
'steps': [
{
'distance': {'text': '0.1 km', 'value': 138},
'duration': {'text': '1 min', 'value': 43},
'end_location': {
'lat': 44.83434380000001,
'lng': -0.5690105999999999,
},
'html_instructions': 'Head <b>east</b> on <b> ...',
'polyline': {'points': '}tspGxknBKcDIkB'},
'start_location': {'lat': 44.8342256, 'lng': -0.57032},
'travel_mode': 'BICYCLING',
},
{
'distance': {'text': '0.1 km', 'value': 115},
'duration': {'text': '1 min', 'value': 22},
'end_location': {'lat': 44.8353651, 'lng': -0.569199},
'html_instructions': 'Turn <b>left</b> onto <b> ...',
'maneuver': 'turn-left',
'polyline': {'points': 'suspGhcnBc@JE@_@DiAHA?w@F'},
'start_location': {
'lat': 44.83434380000001,
'lng': -0.5690105999999999,
},
'travel_mode': 'BICYCLING',
},
{
'distance': {'text': '0.3 km', 'value': 268},
'duration': {'text': '1 min', 'value': 59},
'end_location': {'lat': 44.8362675, 'lng': -0.5660914},
'html_instructions': 'Turn <b>right</b> onto <b> ...',
'maneuver': 'turn-right',
'polyline': {
'points': 'a|spGndnBEYEQKi@Mi@Is@CYCOE]CQIq@ ...',
},
'start_location': {'lat': 44.8353651, 'lng': -0.56919},
'travel_mode': 'BICYCLING',
},
{
'distance': {'text': '0.1 km', 'value': 95},
'duration': {'text': '1 min', 'value': 29},
'end_location': {'lat': 44.8368458, 'lng': -0.5652398},
'html_instructions': 'Slight <b>left</b> onto <b> ...',
'maneuver': 'turn-slight-left',
'polyline': {
'points': 'uatpG`qmBg@aAGM?ACE[k@CICGACEGCCAAEAG?',
},
'start_location': {
'lat': 44.8362675,
'lng': -0.5660914,
},
'travel_mode': 'BICYCLING',
},
{
'distance': {'text': '23 m', 'value': 23},
'duration': {'text': '1 min', 'value': 4},
'end_location': {'lat': 44.83697, 'lng': -0.5654425},
'html_instructions': 'Slight <b>left</b> to stay ...',
'maneuver': 'turn-slight-left',
'polyline': {
'points': 'ietpGvkmBA@C?CBCBEHA@AB?B?B?B?@',
},
'start_location': {
'lat': 44.8368458,
'lng': -0.5652398,
},
'travel_mode': 'BICYCLING',
},
{
'distance': {'text': '0.2 km', 'value': 185},
'duration': {'text': '1 min', 'value': 23},
'end_location': {'lat': 44.8382126, 'lng': -0.5669969},
'html_instructions': 'Take the ramp to <b>Le Lac ...',
'polyline': {
'points': 'aftpG~lmBY^[^sAdB]`@CDKLQRa@h@A@IZ',
},
'start_location': {'lat': 44.83697, 'lng': -0.5654425},
'travel_mode': 'BICYCLING',
},
{
'distance': {'text': '0.3 km', 'value': 253},
'duration': {'text': '1 min', 'value': 43},
'end_location': {'lat': 44.840163, 'lng': -0.5686525},
'html_instructions': 'Merge onto <b>Quai Richelieu</b>',
'maneuver': 'merge',
'polyline': {
'points': 'ymtpGvvmBeAbAe@b@_@ZUN[To@f@e@^A?g ...',
},
'start_location': {
'lat': 44.8382126,
'lng': -0.5669969,
},
'travel_mode': 'BICYCLING',
},
{
'distance': {'text': '0.1 km', 'value': 110},
'duration': {'text': '1 min', 'value': 21},
'end_location': {'lat': 44.841079, 'lng': -0.5691835},
'html_instructions': 'Continue onto <b>Quai de la ...',
'polyline': {'points': '_ztpG`anBUNQLULUJOHMFKDWN'},
'start_location': {'lat': 44.840163, 'lng': -0.56865},
'travel_mode': 'BICYCLING',
},
{
'distance': {'text': '0.3 km', 'value': 262},
'duration': {'text': '1 min', 'value': 44},
'end_location': {'lat': 44.8433375, 'lng': -0.5701161},
'html_instructions': 'Continue onto <b>Quai du ...',
'polyline': {
'points': 'w_upGjdnBeBl@sBn@gA^[JIBc@Nk@Nk@L',
},
'start_location': {'lat': 44.841079, 'lng': -0.56915},
'travel_mode': 'BICYCLING',
},
{
'distance': {'text': '0.6 km', 'value': 550},
'duration': {'text': '2 mins', 'value': 97},
'end_location': {
'lat': 44.84822339999999,
'lng': -0.5705307,
},
'html_instructions': 'Continue onto <b>Quai ...',
'polyline': {
'points': '{mupGfjnBYFI@IBaAPUD{AX}@NK@]Fe@H ...',
},
'start_location': {
'lat': 44.8433375,
'lng': -0.5701161,
},
'travel_mode': 'BICYCLING',
},
{
'distance': {'text': '0.5 km', 'value': 508},
'duration': {'text': '1 min', 'value': 87},
'end_location': {'lat': 44.8523224, 'lng': -0.5678223},
'html_instructions': 'Continue onto ...',
'polyline': {
'points': 'klvpGxlnBWEUGWGSGMEOEOE[KMEQGIA] ...',
},
'start_location': {
'lat': 44.84822339999999,
'lng': -0.5705307,
},
'travel_mode': 'BICYCLING',
},
{
'distance': {'text': '28 m', 'value': 28},
'duration': {'text': '1 min', 'value': 45},
'end_location': {
'lat': 44.85245620000001,
'lng': -0.5681259,
},
'html_instructions': 'Turn <b>left</b> onto <b> ...',
'maneuver': 'turn-left',
'polyline': {'points': '_fwpGz{mBGLADGPCFEN'},
'start_location': {
'lat': 44.8523224,
'lng': -0.5678223,
},
'travel_mode': 'BICYCLING',
},
{
'distance': {'text': '0.2 km', 'value': 176},
'duration': {'text': '1 min', 'value': 31},
'end_location': {'lat': 44.8536857, 'lng': -0.5667282},
'html_instructions': 'Turn <b>right</b> onto <b> ...',
'maneuver': 'turn-right',
'polyline': {
'points': '{fwpGx}mB_@c@mAuAOQi@m@m@y@_@c@',
},
'start_location': {
'lat': 44.85245620000001,
'lng': -0.5681259,
},
'travel_mode': 'BICYCLING',
},
{
'distance': {'text': '0.2 km', 'value': 172},
'duration': {'text': '1 min', 'value': 28},
'end_location': {'lat': 44.8547766, 'lng': -0.5682825},
'html_instructions': 'Turn <b>left</b> onto <b> ... ',
'maneuver': 'turn-left',
'polyline': {'points': 'qnwpG`umBW`@UkDtF'},
'start_location': {
'lat': 44.8536857,
'lng': -0.5667282,
},
'travel_mode': 'BICYCLING',
},
{
'distance': {'text': '0.1 km', 'value': 101},
'duration': {'text': '1 min', 'value': 17},
'end_location': {'lat': 44.8554284, 'lng': -0.5673822},
'html_instructions': 'Turn <b>right</b> onto ...',
'maneuver': 'turn-right',
'polyline': {'points': 'kuwpGv~mBa@q@cA_B[a@'},
'start_location': {
'lat': 44.8547766,
'lng': -0.5682825,
},
'travel_mode': 'BICYCLING',
},
{
'distance': {'text': '15 m', 'value': 15},
'duration': {'text': '1 min', 'value': 3},
'end_location': {
'lat': 44.85540839999999,
'lng': -0.5672105,
},
'html_instructions': 'Turn <b>right</b> onto <b> ...',
'maneuver': 'turn-right',
'polyline': {'points': 'mywpGbymBBC@C?E?C?E?EAC'},
'start_location': {
'lat': 44.8554284,
'lng': -0.5673822,
},
'travel_mode': 'BICYCLING',
},
],
'traffic_speed_entry': [],
'via_waypoint': [],
},
],
'overview_polyline': {
'points': '}tspGxknBUoGi@LcDVe@_CW{Ba@sC[eA_@} ...',
},
'summary': 'Quai des Chartrons',
'warnings': ['Bicycling directions are in beta ...'],
'waypoint_order': [],
},
]
@pytest.fixture
def _fake_google_api(self, api_response, monkeypatch):
"""Patch out the call to the Google Maps Directions API."""
def directions(_self, *_args, **_kwargs):
return api_response
monkeypatch.setattr(googlemaps.Client, 'directions', directions)
@pytest.mark.usefixtures('_fake_google_api')
def test_sync_instances_with_google_maps(self, db_session, path):
"""Call the method for a `Path` object without Google data."""
path.bicycle_distance = None
path.bicycle_duration = None
path._directions = None
path.sync_with_google_maps()
assert path.bicycle_distance == 2_999
assert path.bicycle_duration == 596
assert path._directions is not None
@pytest.mark.usefixtures('_fake_google_api')
def test_repeated_sync_instances_with_google_maps(self, db_session, path):
"""Call the method for a `Path` object with Google data.
That call should immediately return without changing any data.
We use the `path`'s "Google" data from above.
"""
old_distance = path.bicycle_distance
old_duration = path.bicycle_duration
old_directions = path._directions
path.sync_with_google_maps()
assert path.bicycle_distance is old_distance
assert path.bicycle_duration is old_duration
assert path._directions is old_directions
class TestProperties:
"""Test properties in `Path`."""
def test_waypoints_structure(self, path):
"""Test `Path.waypoints` property."""
result = path.waypoints
assert isinstance(result, list)
assert isinstance(result[0], utils.Location)
def test_waypoints_content(self, path):
"""Test `Path.waypoints` property."""
result = path.waypoints
# There are 5 inner points, excluding start and end,
# i.e., the `.first_address` and `second_address`.
assert len(result) == 5
def test_waypoints_is_cached(self, path):
"""Test `Path.waypoints` property."""
result1 = path.waypoints
result2 = path.waypoints
assert result1 is result2

135
tests/db/test_addresses_pixels.py
View file

@ -1,135 +0,0 @@
"""Test the ORM's `AddressPixelAssociation` model.
Implementation notes:
The test suite has 100% coverage without the test cases in this module.
That is so as the `AddressPixelAssociation` model is imported into the
`urban_meal_delivery.db` namespace so that the `Address` and `Pixel` models
can find it upon initialization. Yet, none of the other unit tests run any
code associated with it. Therefore, we test it here as non-e2e tests and do
not measure its coverage.
"""
import pytest
import sqlalchemy as sqla
from sqlalchemy import exc as sa_exc
from urban_meal_delivery import db
@pytest.fixture
def assoc(address, pixel):
"""An association between `address` and `pixel`."""
return db.AddressPixelAssociation(address=address, pixel=pixel)
@pytest.mark.no_cover
class TestSpecialMethods:
"""Test special methods in `AddressPixelAssociation`."""
def test_create_an_address_pixel_association(self, assoc):
"""Test instantiation of a new `AddressPixelAssociation` object."""
assert assoc is not None
@pytest.mark.db
@pytest.mark.no_cover
class TestConstraints:
"""Test the database constraints defined in `AddressPixelAssociation`.
The foreign keys to `City` and `Grid` are tested via INSERT and not
DELETE statements as the latter would already fail because of foreign
keys defined in `Address` and `Pixel`.
"""
def test_insert_into_database(self, db_session, assoc):
"""Insert an instance into the (empty) database."""
assert db_session.query(db.AddressPixelAssociation).count() == 0
db_session.add(assoc)
db_session.commit()
assert db_session.query(db.AddressPixelAssociation).count() == 1
def test_delete_a_referenced_address(self, db_session, assoc):
"""Remove a record that is referenced with a FK."""
db_session.add(assoc)
db_session.commit()
# Must delete without ORM as otherwise an UPDATE statement is emitted.
stmt = sqla.delete(db.Address).where(db.Address.id == assoc.address.id)
with pytest.raises(
sa_exc.IntegrityError,
match='fk_addresses_pixels_to_addresses_via_address_id_city_id',
):
db_session.execute(stmt)
def test_reference_an_invalid_city(self, db_session, address, pixel):
"""Insert a record with an invalid foreign key."""
db_session.add(address)
db_session.add(pixel)
db_session.commit()
# Must insert without ORM as otherwise SQLAlchemy figures out
# that something is wrong before any query is sent to the database.
stmt = sqla.insert(db.AddressPixelAssociation).values(
address_id=address.id,
city_id=999,
grid_id=pixel.grid.id,
pixel_id=pixel.id,
)
with pytest.raises(
sa_exc.IntegrityError,
match='fk_addresses_pixels_to_addresses_via_address_id_city_id',
):
db_session.execute(stmt)
def test_reference_an_invalid_grid(self, db_session, address, pixel):
"""Insert a record with an invalid foreign key."""
db_session.add(address)
db_session.add(pixel)
db_session.commit()
# Must insert without ORM as otherwise SQLAlchemy figures out
# that something is wrong before any query is sent to the database.
stmt = sqla.insert(db.AddressPixelAssociation).values(
address_id=address.id,
city_id=address.city.id,
grid_id=999,
pixel_id=pixel.id,
)
with pytest.raises(
sa_exc.IntegrityError,
match='fk_addresses_pixels_to_grids_via_grid_id_city_id',
):
db_session.execute(stmt)
def test_delete_a_referenced_pixel(self, db_session, assoc):
"""Remove a record that is referenced with a FK."""
db_session.add(assoc)
db_session.commit()
# Must delete without ORM as otherwise an UPDATE statement is emitted.
stmt = sqla.delete(db.Pixel).where(db.Pixel.id == assoc.pixel.id)
with pytest.raises(
sa_exc.IntegrityError,
match='fk_addresses_pixels_to_pixels_via_pixel_id_grid_id',
):
db_session.execute(stmt)
def test_put_an_address_on_a_grid_twice(self, db_session, address, assoc, pixel):
"""Insert a record that violates a unique constraint."""
db_session.add(assoc)
db_session.commit()
# Create a neighboring `Pixel` and put the same `address` as in `pixel` in it.
neighbor = db.Pixel(grid=pixel.grid, n_x=pixel.n_x, n_y=pixel.n_y + 1)
another_assoc = db.AddressPixelAssociation(address=address, pixel=neighbor)
db_session.add(another_assoc)
with pytest.raises(sa_exc.IntegrityError, match='duplicate key value'):
db_session.commit()

96
tests/db/test_cities.py
View file

@ -1,96 +0,0 @@
"""Test the ORM's `City` model."""
import pytest
from urban_meal_delivery import db
from urban_meal_delivery.db import utils
class TestSpecialMethods:
"""Test special methods in `City`."""
def test_create_city(self, city):
"""Test instantiation of a new `City` object."""
assert city is not None
def test_text_representation(self, city):
"""`City` has a non-literal text representation."""
result = repr(city)
assert result == f'<City({city.name})>'
@pytest.mark.db
@pytest.mark.no_cover
class TestConstraints:
"""Test the database constraints defined in `City`."""
def test_insert_into_database(self, db_session, city):
"""Insert an instance into the (empty) database."""
assert db_session.query(db.City).count() == 0
db_session.add(city)
db_session.commit()
assert db_session.query(db.City).count() == 1
class TestProperties:
"""Test properties in `City`."""
def test_center(self, city, city_data):
"""Test `City.center` property."""
result = city.center
assert isinstance(result, utils.Location)
assert result.latitude == pytest.approx(city_data['center_latitude'])
assert result.longitude == pytest.approx(city_data['center_longitude'])
def test_center_is_cached(self, city):
"""Test `City.center` property."""
result1 = city.center
result2 = city.center
assert result1 is result2
def test_northeast(self, city, city_data):
"""Test `City.northeast` property."""
result = city.northeast
assert isinstance(result, utils.Location)
assert result.latitude == pytest.approx(city_data['northeast_latitude'])
assert result.longitude == pytest.approx(city_data['northeast_longitude'])
def test_northeast_is_cached(self, city):
"""Test `City.northeast` property."""
result1 = city.northeast
result2 = city.northeast
assert result1 is result2
def test_southwest(self, city, city_data):
"""Test `City.southwest` property."""
result = city.southwest
assert isinstance(result, utils.Location)
assert result.latitude == pytest.approx(city_data['southwest_latitude'])
assert result.longitude == pytest.approx(city_data['southwest_longitude'])
def test_southwest_is_cached(self, city):
"""Test `City.southwest` property."""
result1 = city.southwest
result2 = city.southwest
assert result1 is result2
def test_total_x(self, city):
"""Test `City.total_x` property."""
result = city.total_x
assert result > 18_000
def test_total_y(self, city):
"""Test `City.total_y` property."""
result = city.total_y
assert result > 9_000

107
tests/db/test_couriers.py
View file

@ -1,107 +0,0 @@
"""Test the ORM's `Courier` model."""
import pytest
from sqlalchemy import exc as sa_exc
from urban_meal_delivery import db
class TestSpecialMethods:
"""Test special methods in `Courier`."""
def test_create_courier(self, courier):
"""Test instantiation of a new `Courier` object."""
assert courier is not None
def test_text_representation(self, courier):
"""`Courier` has a non-literal text representation."""
result = repr(courier)
assert result == f'<Courier(#{courier.id})>'
@pytest.mark.db
@pytest.mark.no_cover
class TestConstraints:
"""Test the database constraints defined in `Courier`."""
def test_insert_into_database(self, db_session, courier):
"""Insert an instance into the (empty) database."""
assert db_session.query(db.Courier).count() == 0
db_session.add(courier)
db_session.commit()
assert db_session.query(db.Courier).count() == 1
def test_invalid_vehicle(self, db_session, courier):
"""Insert an instance with invalid data."""
courier.vehicle = 'invalid'
db_session.add(courier)
with pytest.raises(sa_exc.IntegrityError, match='available_vehicle_types'):
db_session.commit()
def test_negative_speed(self, db_session, courier):
"""Insert an instance with invalid data."""
courier.historic_speed = -1
db_session.add(courier)
with pytest.raises(sa_exc.IntegrityError, match='realistic_speed'):
db_session.commit()
def test_unrealistic_speed(self, db_session, courier):
"""Insert an instance with invalid data."""
courier.historic_speed = 999
db_session.add(courier)
with pytest.raises(sa_exc.IntegrityError, match='realistic_speed'):
db_session.commit()
def test_negative_capacity(self, db_session, courier):
"""Insert an instance with invalid data."""
courier.capacity = -1
db_session.add(courier)
with pytest.raises(sa_exc.IntegrityError, match='capacity_under_200_liters'):
db_session.commit()
def test_too_much_capacity(self, db_session, courier):
"""Insert an instance with invalid data."""
courier.capacity = 999
db_session.add(courier)
with pytest.raises(sa_exc.IntegrityError, match='capacity_under_200_liters'):
db_session.commit()
def test_negative_pay_per_hour(self, db_session, courier):
"""Insert an instance with invalid data."""
courier.pay_per_hour = -1
db_session.add(courier)
with pytest.raises(sa_exc.IntegrityError, match='realistic_pay_per_hour'):
db_session.commit()
def test_too_much_pay_per_hour(self, db_session, courier):
"""Insert an instance with invalid data."""
courier.pay_per_hour = 9999
db_session.add(courier)
with pytest.raises(sa_exc.IntegrityError, match='realistic_pay_per_hour'):
db_session.commit()
def test_negative_pay_per_order(self, db_session, courier):
"""Insert an instance with invalid data."""
courier.pay_per_order = -1
db_session.add(courier)
with pytest.raises(sa_exc.IntegrityError, match='realistic_pay_per_order'):
db_session.commit()
def test_too_much_pay_per_order(self, db_session, courier):
"""Insert an instance with invalid data."""
courier.pay_per_order = 999
db_session.add(courier)
with pytest.raises(sa_exc.IntegrityError, match='realistic_pay_per_order'):
db_session.commit()

34
tests/db/test_customer.py
View file

@ -1,34 +0,0 @@
"""Test the ORM's `Customer` model."""
import pytest
from urban_meal_delivery import db
class TestSpecialMethods:
"""Test special methods in `Customer`."""
def test_create_customer(self, customer):
"""Test instantiation of a new `Customer` object."""
assert customer is not None
def test_text_representation(self, customer):
"""`Customer` has a non-literal text representation."""
result = repr(customer)
assert result == f'<Customer(#{customer.id})>'
@pytest.mark.db
@pytest.mark.no_cover
class TestConstraints:
"""Test the database constraints defined in `Customer`."""
def test_insert_into_database(self, db_session, customer):
"""Insert an instance into the (empty) database."""
assert db_session.query(db.Customer).count() == 0
db_session.add(customer)
db_session.commit()
assert db_session.query(db.Customer).count() == 1

505
tests/db/test_forecasts.py
View file

@ -1,505 +0,0 @@
"""Test the ORM's `Forecast` model."""
import datetime as dt
import pandas as pd
import pytest
import sqlalchemy as sqla
from sqlalchemy import exc as sa_exc
from tests import config as test_config
from urban_meal_delivery import db
MODEL = 'hets'
@pytest.fixture
def forecast(pixel):
"""A `forecast` made in the `pixel` at `NOON`."""
start_at = dt.datetime(
test_config.END.year,
test_config.END.month,
test_config.END.day,
test_config.NOON,
)
return db.Forecast(
pixel=pixel,
start_at=start_at,
time_step=test_config.LONG_TIME_STEP,
train_horizon=test_config.LONG_TRAIN_HORIZON,
model=MODEL,
actual=12,
prediction=12.3,
low80=1.23,
high80=123.4,
low95=0.123,
high95=1234.5,
)
class TestSpecialMethods:
"""Test special methods in `Forecast`."""
def test_create_forecast(self, forecast):
"""Test instantiation of a new `Forecast` object."""
assert forecast is not None
def test_text_representation(self, forecast):
"""`Forecast` has a non-literal text representation."""
result = repr(forecast)
assert (
result
== f'<Forecast: {forecast.prediction} for pixel ({forecast.pixel.n_x}|{forecast.pixel.n_y}) at {forecast.start_at}>' # noqa:E501
)
@pytest.mark.db
@pytest.mark.no_cover
class TestConstraints:
"""Test the database constraints defined in `Forecast`."""
def test_insert_into_database(self, db_session, forecast):
"""Insert an instance into the (empty) database."""
assert db_session.query(db.Forecast).count() == 0
db_session.add(forecast)
db_session.commit()
assert db_session.query(db.Forecast).count() == 1
def test_delete_a_referenced_pixel(self, db_session, forecast):
"""Remove a record that is referenced with a FK."""
db_session.add(forecast)
db_session.commit()
# Must delete without ORM as otherwise an UPDATE statement is emitted.
stmt = sqla.delete(db.Pixel).where(db.Pixel.id == forecast.pixel.id)
with pytest.raises(
sa_exc.IntegrityError, match='fk_forecasts_to_pixels_via_pixel_id',
):
db_session.execute(stmt)
@pytest.mark.parametrize('hour', [10, 23])
def test_invalid_start_at_outside_operating_hours(
self, db_session, forecast, hour,
):
"""Insert an instance with invalid data."""
forecast.start_at = dt.datetime(
forecast.start_at.year,
forecast.start_at.month,
forecast.start_at.day,
hour,
)
db_session.add(forecast)
with pytest.raises(
sa_exc.IntegrityError, match='within_operating_hours',
):
db_session.commit()
def test_invalid_start_at_not_quarter_of_hour(self, db_session, forecast):
"""Insert an instance with invalid data."""
forecast.start_at += dt.timedelta(minutes=1)
db_session.add(forecast)
with pytest.raises(
sa_exc.IntegrityError, match='must_be_quarters_of_the_hour',
):
db_session.commit()
def test_invalid_start_at_seconds_set(self, db_session, forecast):
"""Insert an instance with invalid data."""
forecast.start_at += dt.timedelta(seconds=1)
db_session.add(forecast)
with pytest.raises(
sa_exc.IntegrityError, match='no_seconds',
):
db_session.commit()
def test_invalid_start_at_microseconds_set(self, db_session, forecast):
"""Insert an instance with invalid data."""
forecast.start_at += dt.timedelta(microseconds=1)
db_session.add(forecast)
with pytest.raises(
sa_exc.IntegrityError, match='no_microseconds',
):
db_session.commit()
@pytest.mark.parametrize('value', [-1, 0])
def test_positive_time_step(self, db_session, forecast, value):
"""Insert an instance with invalid data."""
forecast.time_step = value
db_session.add(forecast)
with pytest.raises(
sa_exc.IntegrityError, match='time_step_must_be_positive',
):
db_session.commit()
@pytest.mark.parametrize('value', [-1, 0])
def test_positive_train_horizon(self, db_session, forecast, value):
"""Insert an instance with invalid data."""
forecast.train_horizon = value
db_session.add(forecast)
with pytest.raises(
sa_exc.IntegrityError, match='training_horizon_must_be_positive',
):
db_session.commit()
def test_non_negative_actuals(self, db_session, forecast):
"""Insert an instance with invalid data."""
forecast.actual = -1
db_session.add(forecast)
with pytest.raises(
sa_exc.IntegrityError, match='actuals_must_be_non_negative',
):
db_session.commit()
def test_set_prediction_without_ci(self, db_session, forecast):
"""Sanity check to see that the check constraint ...
... "prediction_must_be_within_ci" is not triggered.
"""
forecast.low80 = None
forecast.high80 = None
forecast.low95 = None
forecast.high95 = None
db_session.add(forecast)
db_session.commit()
def test_ci80_with_missing_low(self, db_session, forecast):
"""Insert an instance with invalid data."""
assert forecast.high80 is not None
forecast.low80 = None
db_session.add(forecast)
with pytest.raises(
sa_exc.IntegrityError, match='ci_upper_and_lower_bounds',
):
db_session.commit()
def test_ci95_with_missing_low(self, db_session, forecast):
"""Insert an instance with invalid data."""
assert forecast.high95 is not None
forecast.low95 = None
db_session.add(forecast)
with pytest.raises(
sa_exc.IntegrityError, match='ci_upper_and_lower_bounds',
):
db_session.commit()
def test_ci80_with_missing_high(self, db_session, forecast):
"""Insert an instance with invalid data."""
assert forecast.low80 is not None
forecast.high80 = None
db_session.add(forecast)
with pytest.raises(
sa_exc.IntegrityError, match='ci_upper_and_lower_bounds',
):
db_session.commit()
def test_ci95_with_missing_high(self, db_session, forecast):
"""Insert an instance with invalid data."""
assert forecast.low95 is not None
forecast.high95 = None
db_session.add(forecast)
with pytest.raises(
sa_exc.IntegrityError, match='ci_upper_and_lower_bounds',
):
db_session.commit()
def test_prediction_smaller_than_low80_with_ci95_set(self, db_session, forecast):
"""Insert an instance with invalid data."""
assert forecast.low95 is not None
assert forecast.high95 is not None
forecast.prediction = forecast.low80 - 0.001
db_session.add(forecast)
with pytest.raises(
sa_exc.IntegrityError, match='prediction_must_be_within_ci',
):
db_session.commit()
def test_prediction_smaller_than_low80_without_ci95_set(
self, db_session, forecast,
):
"""Insert an instance with invalid data."""
forecast.low95 = None
forecast.high95 = None
forecast.prediction = forecast.low80 - 0.001
db_session.add(forecast)
with pytest.raises(
sa_exc.IntegrityError, match='prediction_must_be_within_ci',
):
db_session.commit()
def test_prediction_smaller_than_low95_with_ci80_set(self, db_session, forecast):
"""Insert an instance with invalid data."""
assert forecast.low80 is not None
assert forecast.high80 is not None
forecast.prediction = forecast.low95 - 0.001
db_session.add(forecast)
with pytest.raises(
sa_exc.IntegrityError, match='prediction_must_be_within_ci',
):
db_session.commit()
def test_prediction_smaller_than_low95_without_ci80_set(
self, db_session, forecast,
):
"""Insert an instance with invalid data."""
forecast.low80 = None
forecast.high80 = None
forecast.prediction = forecast.low95 - 0.001
db_session.add(forecast)
with pytest.raises(
sa_exc.IntegrityError, match='prediction_must_be_within_ci',
):
db_session.commit()
def test_prediction_greater_than_high80_with_ci95_set(self, db_session, forecast):
"""Insert an instance with invalid data."""
assert forecast.low95 is not None
assert forecast.high95 is not None
forecast.prediction = forecast.high80 + 0.001
db_session.add(forecast)
with pytest.raises(
sa_exc.IntegrityError, match='prediction_must_be_within_ci',
):
db_session.commit()
def test_prediction_greater_than_high80_without_ci95_set(
self, db_session, forecast,
):
"""Insert an instance with invalid data."""
forecast.low95 = None
forecast.high95 = None
forecast.prediction = forecast.high80 + 0.001
db_session.add(forecast)
with pytest.raises(
sa_exc.IntegrityError, match='prediction_must_be_within_ci',
):
db_session.commit()
def test_prediction_greater_than_high95_with_ci80_set(self, db_session, forecast):
"""Insert an instance with invalid data."""
assert forecast.low80 is not None
assert forecast.high80 is not None
forecast.prediction = forecast.high95 + 0.001
db_session.add(forecast)
with pytest.raises(
sa_exc.IntegrityError, match='prediction_must_be_within_ci',
):
db_session.commit()
def test_prediction_greater_than_high95_without_ci80_set(
self, db_session, forecast,
):
"""Insert an instance with invalid data."""
forecast.low80 = None
forecast.high80 = None
forecast.prediction = forecast.high95 + 0.001
db_session.add(forecast)
with pytest.raises(
sa_exc.IntegrityError, match='prediction_must_be_within_ci',
):
db_session.commit()
def test_ci80_upper_bound_greater_than_lower_bound(self, db_session, forecast):
"""Insert an instance with invalid data."""
assert forecast.low80 is not None
assert forecast.high80 is not None
# Do not trigger the "ci95_must_be_wider_than_ci80" constraint.
forecast.low95 = None
forecast.high95 = None
forecast.low80, forecast.high80 = ( # noqa:WPS414
forecast.high80,
forecast.low80,
)
db_session.add(forecast)
with pytest.raises(
sa_exc.IntegrityError, match='ci_upper_bound_greater_than_lower_bound',
):
db_session.commit()
def test_ci95_upper_bound_greater_than_lower_bound(self, db_session, forecast):
"""Insert an instance with invalid data."""
assert forecast.low95 is not None
assert forecast.high95 is not None
# Do not trigger the "ci95_must_be_wider_than_ci80" constraint.
forecast.low80 = None
forecast.high80 = None
forecast.low95, forecast.high95 = ( # noqa:WPS414
forecast.high95,
forecast.low95,
)
db_session.add(forecast)
with pytest.raises(
sa_exc.IntegrityError, match='ci_upper_bound_greater_than_lower_bound',
):
db_session.commit()
def test_ci95_is_wider_than_ci80_at_low_end(self, db_session, forecast):
"""Insert an instance with invalid data."""
assert forecast.low80 is not None
assert forecast.low95 is not None
forecast.low80, forecast.low95 = (forecast.low95, forecast.low80) # noqa:WPS414
db_session.add(forecast)
with pytest.raises(
sa_exc.IntegrityError, match='ci95_must_be_wider_than_ci80',
):
db_session.commit()
def test_ci95_is_wider_than_ci80_at_high_end(self, db_session, forecast):
"""Insert an instance with invalid data."""
assert forecast.high80 is not None
assert forecast.high95 is not None
forecast.high80, forecast.high95 = ( # noqa:WPS414
forecast.high95,
forecast.high80,
)
db_session.add(forecast)
with pytest.raises(
sa_exc.IntegrityError, match='ci95_must_be_wider_than_ci80',
):
db_session.commit()
def test_two_predictions_for_same_forecasting_setting(self, db_session, forecast):
"""Insert a record that violates a unique constraint."""
db_session.add(forecast)
db_session.commit()
another_forecast = db.Forecast(
pixel=forecast.pixel,
start_at=forecast.start_at,
time_step=forecast.time_step,
train_horizon=forecast.train_horizon,
model=forecast.model,
actual=forecast.actual,
prediction=2,
low80=1,
high80=3,
low95=0,
high95=4,
)
db_session.add(another_forecast)
with pytest.raises(sa_exc.IntegrityError, match='duplicate key value'):
db_session.commit()
class TestFromDataFrameConstructor:
"""Test the alternative `Forecast.from_dataframe()` constructor."""
@pytest.fixture
def prediction_data(self):
"""A `pd.DataFrame` as returned by `*Model.predict()` ...
... and used as the `data` argument to `Forecast.from_dataframe()`.
We assume the `data` come from some vertical forecasting `*Model`
and contain several rows (= `3` in this example) corresponding
to different time steps centered around `NOON`.
"""
noon_start_at = dt.datetime(
test_config.END.year,
test_config.END.month,
test_config.END.day,
test_config.NOON,
)
index = pd.Index(
[
noon_start_at - dt.timedelta(minutes=test_config.LONG_TIME_STEP),
noon_start_at,
noon_start_at + dt.timedelta(minutes=test_config.LONG_TIME_STEP),
],
)
index.name = 'start_at'
return pd.DataFrame(
data={
'actual': (11, 12, 13),
'prediction': (11.3, 12.3, 13.3),
'low80': (1.123, 1.23, 1.323),
'high80': (112.34, 123.4, 132.34),
'low95': (0.1123, 0.123, 0.1323),
'high95': (1123.45, 1234.5, 1323.45),
},
index=index,
)
def test_convert_dataframe_into_orm_objects(self, pixel, prediction_data):
"""Call `Forecast.from_dataframe()`."""
forecasts = db.Forecast.from_dataframe(
pixel=pixel,
time_step=test_config.LONG_TIME_STEP,
train_horizon=test_config.LONG_TRAIN_HORIZON,
model=MODEL,
data=prediction_data,
)
assert len(forecasts) == 3
for forecast in forecasts:
assert isinstance(forecast, db.Forecast)
@pytest.mark.db
def test_persist_predictions_into_database(
self, db_session, pixel, prediction_data,
):
"""Call `Forecast.from_dataframe()` and persist the results."""
forecasts = db.Forecast.from_dataframe(
pixel=pixel,
time_step=test_config.LONG_TIME_STEP,
train_horizon=test_config.LONG_TRAIN_HORIZON,
model=MODEL,
data=prediction_data,
)
db_session.add_all(forecasts)
db_session.commit()

239
tests/db/test_grids.py
View file

@ -1,239 +0,0 @@
"""Test the ORM's `Grid` model."""
import pytest
import sqlalchemy as sqla
from sqlalchemy import exc as sa_exc
from urban_meal_delivery import db
class TestSpecialMethods:
"""Test special methods in `Grid`."""
def test_create_grid(self, grid):
"""Test instantiation of a new `Grid` object."""
assert grid is not None
def test_text_representation(self, grid):
"""`Grid` has a non-literal text representation."""
result = repr(grid)
assert result == f'<Grid: {grid.pixel_area} sqr. km>'
@pytest.mark.db
@pytest.mark.no_cover
class TestConstraints:
"""Test the database constraints defined in `Grid`."""
def test_insert_into_database(self, db_session, grid):
"""Insert an instance into the (empty) database."""
assert db_session.query(db.Grid).count() == 0
db_session.add(grid)
db_session.commit()
assert db_session.query(db.Grid).count() == 1
def test_delete_a_referenced_city(self, db_session, grid):
"""Remove a record that is referenced with a FK."""
db_session.add(grid)
db_session.commit()
# Must delete without ORM as otherwise an UPDATE statement is emitted.
stmt = sqla.delete(db.City).where(db.City.id == grid.city.id)
with pytest.raises(
sa_exc.IntegrityError, match='fk_grids_to_cities_via_city_id',
):
db_session.execute(stmt)
def test_two_grids_with_identical_side_length(self, db_session, grid):
"""Insert a record that violates a unique constraint."""
db_session.add(grid)
db_session.commit()
# Create a `Grid` with the same `.side_length` in the same `.city`.
another_grid = db.Grid(city=grid.city, side_length=grid.side_length)
db_session.add(another_grid)
with pytest.raises(sa_exc.IntegrityError, match='duplicate key value'):
db_session.commit()
class TestProperties:
"""Test properties in `Grid`."""
def test_pixel_area(self, grid):
"""Test `Grid.pixel_area` property."""
result = grid.pixel_area
assert result == 1.0
class TestGridification:
"""Test the `Grid.gridify()` constructor."""
@pytest.fixture
def addresses_mock(self, mocker, monkeypatch):
"""A `Mock` whose `.return_value` are to be set ...
... to the addresses that are gridified. The addresses are
all considered `Order.pickup_address` attributes for some orders.
"""
mock = mocker.Mock()
query = ( # noqa:ECE001
mock.query.return_value.join.return_value.filter.return_value.all # noqa:E501,WPS219
)
monkeypatch.setattr(db, 'session', mock)
return query
@pytest.mark.no_cover
def test_no_pixel_without_addresses(self, city, addresses_mock):
"""Without orders, there are no `Pixel` objects on the `grid`.
This test case skips the `for`-loop inside `Grid.gridify()`.
"""
addresses_mock.return_value = []
# The chosen `side_length` would result in one `Pixel` if there were orders.
# `+1` as otherwise there would be a second pixel in one direction.
side_length = max(city.total_x, city.total_y) + 1
result = db.Grid.gridify(city=city, side_length=side_length)
assert isinstance(result, db.Grid)
assert len(result.pixels) == 0 # noqa:WPS507
def test_one_pixel_with_one_address(self, city, order, addresses_mock):
"""At the very least, there must be one `Pixel` ...
... if the `side_length` is greater than both the
horizontal and vertical distances of the viewport.
"""
addresses_mock.return_value = [order.pickup_address]
# `+1` as otherwise there would be a second pixel in one direction.
side_length = max(city.total_x, city.total_y) + 1
result = db.Grid.gridify(city=city, side_length=side_length)
assert isinstance(result, db.Grid)
assert len(result.pixels) == 1
def test_one_pixel_with_two_addresses(self, city, make_order, addresses_mock):
"""At the very least, there must be one `Pixel` ...
... if the `side_length` is greater than both the
horizontal and vertical distances of the viewport.
This test case is necessary as `test_one_pixel_with_one_address`
does not have to re-use an already created `Pixel` object internally.
"""
orders = [make_order(), make_order()]
addresses_mock.return_value = [order.pickup_address for order in orders]
# `+1` as otherwise there would be a second pixel in one direction.
side_length = max(city.total_x, city.total_y) + 1
result = db.Grid.gridify(city=city, side_length=side_length)
assert isinstance(result, db.Grid)
assert len(result.pixels) == 1
def test_no_pixel_with_one_address_too_far_south(self, city, order, addresses_mock):
"""An `address` outside the `city`'s viewport is discarded."""
# Move the `address` just below `city.southwest`.
order.pickup_address.latitude = city.southwest.latitude - 0.1
addresses_mock.return_value = [order.pickup_address]
# `+1` as otherwise there would be a second pixel in one direction.
side_length = max(city.total_x, city.total_y) + 1
result = db.Grid.gridify(city=city, side_length=side_length)
assert isinstance(result, db.Grid)
assert len(result.pixels) == 0 # noqa:WPS507
@pytest.mark.no_cover
def test_no_pixel_with_one_address_too_far_west(self, city, order, addresses_mock):
"""An `address` outside the `city`'s viewport is discarded.
This test is a logical sibling to
`test_no_pixel_with_one_address_too_far_south` and therefore redundant.
"""
# Move the `address` just left to `city.southwest`.
order.pickup_address.longitude = city.southwest.longitude - 0.1
addresses_mock.return_value = [order.pickup_address]
# `+1` as otherwise there would be a second pixel in one direction.
side_length = max(city.total_x, city.total_y) + 1
result = db.Grid.gridify(city=city, side_length=side_length)
assert isinstance(result, db.Grid)
assert len(result.pixels) == 0 # noqa:WPS507
@pytest.mark.no_cover
def test_two_pixels_with_two_addresses(self, city, make_address, addresses_mock):
"""Two `Address` objects in distinct `Pixel` objects.
This test is more of a sanity check.
"""
# Create two `Address` objects in distinct `Pixel`s.
addresses_mock.return_value = [
# One `Address` in the lower-left `Pixel`, ...
make_address(latitude=48.8357377, longitude=2.2517412),
# ... and another one in the upper-right one.
make_address(latitude=48.8898312, longitude=2.4357622),
]
side_length = max(city.total_x // 2, city.total_y // 2) + 1
# By assumption of the test data.
n_pixels_x = (city.total_x // side_length) + 1
n_pixels_y = (city.total_y // side_length) + 1
assert n_pixels_x * n_pixels_y == 4
# Create a `Grid` with at most four `Pixel`s.
result = db.Grid.gridify(city=city, side_length=side_length)
assert isinstance(result, db.Grid)
assert len(result.pixels) == 2
@pytest.mark.db
@pytest.mark.no_cover
@pytest.mark.parametrize('side_length', [250, 500, 1_000, 2_000, 4_000, 8_000])
def test_make_random_grids( # noqa:WPS211,WPS218
self, db_session, city, make_address, make_restaurant, make_order, side_length,
):
"""With 100 random `Address` objects, a grid must have ...
... between 1 and a deterministic upper bound of `Pixel` objects.
This test creates confidence that the created `Grid`
objects adhere to the database constraints.
"""
addresses = [make_address() for _ in range(100)]
restaurants = [make_restaurant(address=address) for address in addresses]
orders = [make_order(restaurant=restaurant) for restaurant in restaurants]
db_session.add_all(orders)
n_pixels_x = (city.total_x // side_length) + 1
n_pixels_y = (city.total_y // side_length) + 1
result = db.Grid.gridify(city=city, side_length=side_length)
assert isinstance(result, db.Grid)
assert 1 <= len(result.pixels) <= n_pixels_x * n_pixels_y
# Sanity checks for `Pixel.southwest` and `Pixel.northeast`.
for pixel in result.pixels:
assert abs(pixel.southwest.x - pixel.n_x * side_length) < 2
assert abs(pixel.southwest.y - pixel.n_y * side_length) < 2
assert abs(pixel.northeast.x - (pixel.n_x + 1) * side_length) < 2
assert abs(pixel.northeast.y - (pixel.n_y + 1) * side_length) < 2
db_session.add(result)
db_session.commit()

470
tests/db/test_orders.py
View file

@ -1,470 +0,0 @@
"""Test the ORM's `Order` model."""
import datetime
import random
import pytest
from urban_meal_delivery import db
class TestSpecialMethods:
"""Test special methods in `Order`."""
def test_create_order(self, order):
"""Test instantiation of a new `Order` object."""
assert order is not None
def test_text_representation(self, order):
"""`Order` has a non-literal text representation."""
result = repr(order)
assert result == f'<Order(#{order.id})>'
@pytest.mark.db
@pytest.mark.no_cover
class TestConstraints:
"""Test the database constraints defined in `Order`."""
def test_insert_into_database(self, db_session, order):
"""Insert an instance into the (empty) database."""
assert db_session.query(db.Order).count() == 0
db_session.add(order)
db_session.commit()
assert db_session.query(db.Order).count() == 1
# TODO (order-constraints): the various Foreign Key and Check Constraints
# should be tested eventually. This is not of highest importance as
# we have a lot of confidence from the data cleaning notebook.
class TestProperties:
"""Test properties in `Order`.
The `order` fixture uses the defaults specified in `factories.OrderFactory`
and provided by the `make_order` fixture.
"""
def test_is_ad_hoc(self, order):
"""Test `Order.scheduled` property."""
assert order.ad_hoc is True
result = order.scheduled
assert result is False
def test_is_scheduled(self, make_order):
"""Test `Order.scheduled` property."""
order = make_order(scheduled=True)
assert order.ad_hoc is False
result = order.scheduled
assert result is True
def test_is_completed(self, order):
"""Test `Order.completed` property."""
result = order.completed
assert result is True
def test_is_not_completed1(self, make_order):
"""Test `Order.completed` property."""
order = make_order(cancel_before_pickup=True)
assert order.cancelled is True
result = order.completed
assert result is False
def test_is_not_completed2(self, make_order):
"""Test `Order.completed` property."""
order = make_order(cancel_after_pickup=True)
assert order.cancelled is True
result = order.completed
assert result is False
def test_is_not_corrected(self, order):
"""Test `Order.corrected` property."""
# By default, the `OrderFactory` sets all `.*_corrected` attributes to `False`.
result = order.corrected
assert result is False
@pytest.mark.parametrize(
'column',
[
'scheduled_delivery_at',
'cancelled_at',
'restaurant_notified_at',
'restaurant_confirmed_at',
'dispatch_at',
'courier_notified_at',
'courier_accepted_at',
'pickup_at',
'left_pickup_at',
'delivery_at',
],
)
def test_is_corrected(self, order, column):
"""Test `Order.corrected` property."""
setattr(order, f'{column}_corrected', True)
result = order.corrected
assert result is True
def test_time_to_accept_no_dispatch_at(self, order):
"""Test `Order.time_to_accept` property."""
order.dispatch_at = None
with pytest.raises(RuntimeError, match='not set'):
int(order.time_to_accept)
def test_time_to_accept_no_courier_accepted(self, order):
"""Test `Order.time_to_accept` property."""
order.courier_accepted_at = None
with pytest.raises(RuntimeError, match='not set'):
int(order.time_to_accept)
def test_time_to_accept_success(self, order):
"""Test `Order.time_to_accept` property."""
result = order.time_to_accept
assert result > datetime.timedelta(0)
def test_time_to_react_no_courier_notified(self, order):
"""Test `Order.time_to_react` property."""
order.courier_notified_at = None
with pytest.raises(RuntimeError, match='not set'):
int(order.time_to_react)
def test_time_to_react_no_courier_accepted(self, order):
"""Test `Order.time_to_react` property."""
order.courier_accepted_at = None
with pytest.raises(RuntimeError, match='not set'):
int(order.time_to_react)
def test_time_to_react_success(self, order):
"""Test `Order.time_to_react` property."""
result = order.time_to_react
assert result > datetime.timedelta(0)
def test_time_to_pickup_no_reached_pickup_at(self, order):
"""Test `Order.time_to_pickup` property."""
order.reached_pickup_at = None
with pytest.raises(RuntimeError, match='not set'):
int(order.time_to_pickup)
def test_time_to_pickup_no_courier_accepted(self, order):
"""Test `Order.time_to_pickup` property."""
order.courier_accepted_at = None
with pytest.raises(RuntimeError, match='not set'):
int(order.time_to_pickup)
def test_time_to_pickup_success(self, order):
"""Test `Order.time_to_pickup` property."""
result = order.time_to_pickup
assert result > datetime.timedelta(0)
def test_time_at_pickup_no_reached_pickup_at(self, order):
"""Test `Order.time_at_pickup` property."""
order.reached_pickup_at = None
with pytest.raises(RuntimeError, match='not set'):
int(order.time_at_pickup)
def test_time_at_pickup_no_pickup_at(self, order):
"""Test `Order.time_at_pickup` property."""
order.pickup_at = None
with pytest.raises(RuntimeError, match='not set'):
int(order.time_at_pickup)
def test_time_at_pickup_success(self, order):
"""Test `Order.time_at_pickup` property."""
result = order.time_at_pickup
assert result > datetime.timedelta(0)
def test_scheduled_pickup_at_no_restaurant_notified(self, order): # noqa:WPS118
"""Test `Order.scheduled_pickup_at` property."""
order.restaurant_notified_at = None
with pytest.raises(RuntimeError, match='not set'):
int(order.scheduled_pickup_at)
def test_scheduled_pickup_at_no_est_prep_duration(self, order): # noqa:WPS118
"""Test `Order.scheduled_pickup_at` property."""
order.estimated_prep_duration = None
with pytest.raises(RuntimeError, match='not set'):
int(order.scheduled_pickup_at)
def test_scheduled_pickup_at_success(self, order):
"""Test `Order.scheduled_pickup_at` property."""
result = order.scheduled_pickup_at
assert order.placed_at < result < order.delivery_at
def test_courier_is_early_at_pickup(self, order):
"""Test `Order.courier_early` property."""
# Manipulate the attribute that determines `Order.scheduled_pickup_at`.
order.estimated_prep_duration = 999_999
result = order.courier_early
assert bool(result) is True
def test_courier_is_not_early_at_pickup(self, order):
"""Test `Order.courier_early` property."""
# Manipulate the attribute that determines `Order.scheduled_pickup_at`.
order.estimated_prep_duration = 1
result = order.courier_early
assert bool(result) is False
def test_courier_is_late_at_pickup(self, order):
"""Test `Order.courier_late` property."""
# Manipulate the attribute that determines `Order.scheduled_pickup_at`.
order.estimated_prep_duration = 1
result = order.courier_late
assert bool(result) is True
def test_courier_is_not_late_at_pickup(self, order):
"""Test `Order.courier_late` property."""
# Manipulate the attribute that determines `Order.scheduled_pickup_at`.
order.estimated_prep_duration = 999_999
result = order.courier_late
assert bool(result) is False
def test_restaurant_early_at_pickup(self, order):
"""Test `Order.restaurant_early` property."""
# Manipulate the attribute that determines `Order.scheduled_pickup_at`.
order.estimated_prep_duration = 999_999
result = order.restaurant_early
assert bool(result) is True
def test_restaurant_is_not_early_at_pickup(self, order):
"""Test `Order.restaurant_early` property."""
# Manipulate the attribute that determines `Order.scheduled_pickup_at`.
order.estimated_prep_duration = 1
result = order.restaurant_early
assert bool(result) is False
def test_restaurant_is_late_at_pickup(self, order):
"""Test `Order.restaurant_late` property."""
# Manipulate the attribute that determines `Order.scheduled_pickup_at`.
order.estimated_prep_duration = 1
result = order.restaurant_late
assert bool(result) is True
def test_restaurant_is_not_late_at_pickup(self, order):
"""Test `Order.restaurant_late` property."""
# Manipulate the attribute that determines `Order.scheduled_pickup_at`.
order.estimated_prep_duration = 999_999
result = order.restaurant_late
assert bool(result) is False
def test_time_to_delivery_no_reached_delivery_at(self, order): # noqa:WPS118
"""Test `Order.time_to_delivery` property."""
order.reached_delivery_at = None
with pytest.raises(RuntimeError, match='not set'):
int(order.time_to_delivery)
def test_time_to_delivery_no_pickup_at(self, order):
"""Test `Order.time_to_delivery` property."""
order.pickup_at = None
with pytest.raises(RuntimeError, match='not set'):
int(order.time_to_delivery)
def test_time_to_delivery_success(self, order):
"""Test `Order.time_to_delivery` property."""
result = order.time_to_delivery
assert result > datetime.timedelta(0)
def test_time_at_delivery_no_reached_delivery_at(self, order): # noqa:WPS118
"""Test `Order.time_at_delivery` property."""
order.reached_delivery_at = None
with pytest.raises(RuntimeError, match='not set'):
int(order.time_at_delivery)
def test_time_at_delivery_no_delivery_at(self, order):
"""Test `Order.time_at_delivery` property."""
order.delivery_at = None
with pytest.raises(RuntimeError, match='not set'):
int(order.time_at_delivery)
def test_time_at_delivery_success(self, order):
"""Test `Order.time_at_delivery` property."""
result = order.time_at_delivery
assert result > datetime.timedelta(0)
def test_courier_waited_at_delviery(self, order):
"""Test `Order.courier_waited_at_delivery` property."""
order._courier_waited_at_delivery = True
result = order.courier_waited_at_delivery.total_seconds()
assert result > 0
def test_courier_did_not_wait_at_delivery(self, order):
"""Test `Order.courier_waited_at_delivery` property."""
order._courier_waited_at_delivery = False
result = order.courier_waited_at_delivery.total_seconds()
assert result == 0
def test_ad_hoc_order_cannot_be_early(self, order):
"""Test `Order.delivery_early` property."""
# By default, the `OrderFactory` creates ad-hoc orders.
with pytest.raises(AttributeError, match='scheduled'):
int(order.delivery_early)
def test_scheduled_order_delivered_early(self, make_order):
"""Test `Order.delivery_early` property."""
order = make_order(scheduled=True)
# Schedule the order to a lot later.
order.scheduled_delivery_at += datetime.timedelta(hours=2)
result = order.delivery_early
assert bool(result) is True
def test_scheduled_order_not_delivered_early(self, make_order):
"""Test `Order.delivery_early` property."""
order = make_order(scheduled=True)
# Schedule the order to a lot earlier.
order.scheduled_delivery_at -= datetime.timedelta(hours=2)
result = order.delivery_early
assert bool(result) is False
def test_ad_hoc_order_cannot_be_late(self, order):
"""Test Order.delivery_late property."""
# By default, the `OrderFactory` creates ad-hoc orders.
with pytest.raises(AttributeError, match='scheduled'):
int(order.delivery_late)
def test_scheduled_order_delivered_late(self, make_order):
"""Test `Order.delivery_early` property."""
order = make_order(scheduled=True)
# Schedule the order to a lot earlier.
order.scheduled_delivery_at -= datetime.timedelta(hours=2)
result = order.delivery_late
assert bool(result) is True
def test_scheduled_order_not_delivered_late(self, make_order):
"""Test `Order.delivery_early` property."""
order = make_order(scheduled=True)
# Schedule the order to a lot later.
order.scheduled_delivery_at += datetime.timedelta(hours=2)
result = order.delivery_late
assert bool(result) is False
def test_no_total_time_for_scheduled_order(self, make_order):
"""Test `Order.total_time` property."""
order = make_order(scheduled=True)
with pytest.raises(AttributeError, match='Scheduled'):
int(order.total_time)
def test_no_total_time_for_cancelled_order(self, make_order):
"""Test `Order.total_time` property."""
order = make_order(cancel_before_pickup=True)
with pytest.raises(RuntimeError, match='Cancelled'):
int(order.total_time)
def test_total_time_success(self, order):
"""Test `Order.total_time` property."""
result = order.total_time
assert result > datetime.timedelta(0)
@pytest.mark.db
@pytest.mark.no_cover
def test_make_random_orders( # noqa:C901,WPS211,WPS213,WPS231
db_session, make_address, make_courier, make_restaurant, make_order,
):
"""Sanity check the all the `make_*` fixtures.
Ensure that all generated `Address`, `Courier`, `Customer`, `Restauarant`,
and `Order` objects adhere to the database constraints.
""" # noqa:D202
# Generate a large number of `Order`s to obtain a large variance of data.
for _ in range(1_000): # noqa:WPS122
# Ad-hoc `Order`s are far more common than pre-orders.
scheduled = random.choice([True, False, False, False, False])
# Randomly pass a `address` argument to `make_restaurant()` and
# a `restaurant` argument to `make_order()`.
if random.random() < 0.5:
address = random.choice([None, make_address()])
restaurant = make_restaurant(address=address)
else:
restaurant = None
# Randomly pass a `courier` argument to `make_order()`.
courier = random.choice([None, make_courier()])
# A tiny fraction of `Order`s get cancelled.
if random.random() < 0.05:
if random.random() < 0.5:
cancel_before_pickup, cancel_after_pickup = True, False
else:
cancel_before_pickup, cancel_after_pickup = False, True
else:
cancel_before_pickup, cancel_after_pickup = False, False
# Write all the generated objects to the database.
# This should already trigger an `IntegrityError` if the data are flawed.
order = make_order(
scheduled=scheduled,
restaurant=restaurant,
courier=courier,
cancel_before_pickup=cancel_before_pickup,
cancel_after_pickup=cancel_after_pickup,
)
db_session.add(order)
db_session.commit()

152
tests/db/test_pixels.py
View file

@ -1,152 +0,0 @@
"""Test the ORM's `Pixel` model."""
import pytest
import sqlalchemy as sqla
from sqlalchemy import exc as sa_exc
from urban_meal_delivery import db
class TestSpecialMethods:
"""Test special methods in `Pixel`."""
def test_create_pixel(self, pixel):
"""Test instantiation of a new `Pixel` object."""
assert pixel is not None
def test_text_representation(self, pixel):
"""`Pixel` has a non-literal text representation."""
result = repr(pixel)
assert result == f'<Pixel: ({pixel.n_x}|{pixel.n_y})>'
@pytest.mark.db
@pytest.mark.no_cover
class TestConstraints:
"""Test the database constraints defined in `Pixel`."""
def test_insert_into_database(self, db_session, pixel):
"""Insert an instance into the (empty) database."""
assert db_session.query(db.Pixel).count() == 0
db_session.add(pixel)
db_session.commit()
assert db_session.query(db.Pixel).count() == 1
def test_delete_a_referenced_grid(self, db_session, pixel):
"""Remove a record that is referenced with a FK."""
db_session.add(pixel)
db_session.commit()
# Must delete without ORM as otherwise an UPDATE statement is emitted.
stmt = sqla.delete(db.Grid).where(db.Grid.id == pixel.grid.id)
with pytest.raises(
sa_exc.IntegrityError, match='fk_pixels_to_grids_via_grid_id',
):
db_session.execute(stmt)
def test_negative_n_x(self, db_session, pixel):
"""Insert an instance with invalid data."""
pixel.n_x = -1
db_session.add(pixel)
with pytest.raises(sa_exc.IntegrityError, match='n_x_is_positive'):
db_session.commit()
def test_negative_n_y(self, db_session, pixel):
"""Insert an instance with invalid data."""
pixel.n_y = -1
db_session.add(pixel)
with pytest.raises(sa_exc.IntegrityError, match='n_y_is_positive'):
db_session.commit()
def test_non_unique_coordinates_within_a_grid(self, db_session, pixel):
"""Insert an instance with invalid data."""
another_pixel = db.Pixel(grid=pixel.grid, n_x=pixel.n_x, n_y=pixel.n_y)
db_session.add(another_pixel)
with pytest.raises(sa_exc.IntegrityError, match='duplicate key value'):
db_session.commit()
class TestProperties:
"""Test properties in `Pixel`."""
def test_side_length(self, pixel):
"""Test `Pixel.side_length` property."""
result = pixel.side_length
assert result == 1_000
def test_area(self, pixel):
"""Test `Pixel.area` property."""
result = pixel.area
assert result == 1.0
def test_northeast(self, pixel):
"""Test `Pixel.northeast` property."""
result = pixel.northeast
assert abs(result.x - pixel.side_length) < 2
assert abs(result.y - pixel.side_length) < 2
def test_northeast_is_cached(self, pixel):
"""Test `Pixel.northeast` property."""
result1 = pixel.northeast
result2 = pixel.northeast
assert result1 is result2
def test_southwest(self, pixel):
"""Test `Pixel.southwest` property."""
result = pixel.southwest
assert abs(result.x) < 2
assert abs(result.y) < 2
def test_southwest_is_cached(self, pixel):
"""Test `Pixel.southwest` property."""
result1 = pixel.southwest
result2 = pixel.southwest
assert result1 is result2
@pytest.fixture
def _restaurants_mock(self, mocker, monkeypatch, restaurant):
"""A `Mock` whose `.return_value` is `[restaurant]`."""
mock = mocker.Mock()
query = ( # noqa:ECE001
mock.query.return_value.join.return_value.filter.return_value.all # noqa:E501,WPS219
)
query.return_value = [restaurant]
monkeypatch.setattr(db, 'session', mock)
@pytest.mark.usefixtures('_restaurants_mock')
def test_restaurants(self, pixel, restaurant):
"""Test `Pixel.restaurants` property."""
result = pixel.restaurants
assert result == [restaurant]
@pytest.mark.usefixtures('_restaurants_mock')
def test_restaurants_is_cached(self, pixel):
"""Test `Pixel.restaurants` property."""
result1 = pixel.restaurants
result2 = pixel.restaurants
assert result1 is result2
@pytest.mark.db
def test_restaurants_with_db(self, pixel):
"""Test `Pixel.restaurants` property.
This is a trivial integration test.
"""
result = pixel.restaurants
assert not result # = empty `list`

Some files were not shown because too many files have changed in this diff Show more