- the purpose of this constructor method is to generate all `Pixel`s
for a `Grid` that have at least one `Address` assigned to them
- fix missing `UniqueConstraint` in `Grid` class => it was not possible
to create two `Grid`s with the same `.side_length` in different cities
- change the `City.viewport` property into two separate `City.southwest`
and `City.northeast` properties; also add `City.total_x` and
`City.total_y` properties for convenience
- the old `UTMCoordinate` class becomes the new `Location` class
- its main purpose is to represent locations in both lat-long
coordinates as well as in the UTM system
- remove `Address.__init__()` and `City.__init__()` methods as they
are not executed for entries retrieved from the database
- simplfiy the `Location.__init__()` => remove `relative_to` argument
- remove the factory functions for creating engines and sessions
- define global engine, connection, and session objects to be used
everywhere in the urban_meal_delivery package
- add Grid, Pixel, and AddressPixelAssociation ORM models
- each Grid belongs to a City an is characterized by the side_length
of all the square Pixels contained in it
- Pixels aggregate Addresses => many-to-many relationship (that is
modeled with SQLAlchemy's Association Pattern to implement a couple
of constraints)
- the Address.x and Address.y properties use the UTMCoordinate class
behind the scenes
- x and y are simple coordinates in an x-y plane
- the (0, 0) origin is the southwest corner of Address.city.viewport
- the class is a utility to abstract working with latitude-longitude
coordinates in their UTM representation (~ "cartesian plane")
- the class's .x and .y properties enable working with simple x-y
coordinates where the (0, 0) origin is the lower-left of a city's
viewport
- create `*Factory` classes with fakerboy and faker that generate
randomized instances of the ORM models
- add new pytest marker: "db" are the integration tests involving the
database whereas "e2e" will be all other integration tests
- streamline the docstrings in the ORM models
As a lot of the integration tests populate the database with test data,
it is deemed safer to run the tests in random order to uncover potential
dependencies between distinct test cases.
Because of how the `db_session` fixture is designed, this should already
be taken care of.
- clean the raw data given by the undisclosed meal delivery platform:
+ keep data only for the three target citis:
* Bordeaux
* Lyon
* Paris
+ merge duplicates
* it appears as redundant addresses were created
for each order by the same customer
=> significant reduction in the number of addresses
* propagate the merges to the other tables
that reference records merged away
+ cast data types and keep their scopes narrow
+ normalize the data
+ remove obvious outliers
+ adjust/discard unplausible values
- map the cleaned data onto the ORM models
- store the cleaned data in a new database schema
- change references to temporary branches (e.g., "release-*" and
"publish") to point to the 'main' branch
- add --branch=BRANCH_NAME option to the nox session so that
one can pass in a target branch to make all references point to
- run "fix-branch-references" as the first pre-commit hook
as it fails the fastest
- bug fix: allow dots in branch references (e.g., "release-0.1.0")
- dependencies used to run the Jupyter Lab environment that are not
required by the `urban-meal-delivery` package itself are put into
an installation extra called "research"
- this allows to NOT install the requirements, for example, when
testing the package in an isolated environment
- run "format" and "lint" separately
=> remove the nox session "pre-commit"
- execute the entire "test-suite" before merges
+ rename "pre-merge" into "test-suite"
+ do not "format" and "lint" here any more
+ do not execute this before pushes
* allow branches with <100% test coverage to exist on GitHub
(they cannot be merged into 'main' until 100% coverage)
* GitHub Actions executes the test suite
- many *.py and *.ipynb files will contain links to resources on
GitHub or nbviewer that have branch references in them
- add a pre-commit hook implemented as the nox session
"fix-branch-references" that goes through these files and
changes all the branch labels to the current one
- use Alembic to migrate the PostgreSQL database
+ create initial migration script to set up the database,
as an alternative to db.Base.metadata.create_all()
+ integrate Alembic into the test suite; the db_engine fixture
now has two modes:
* create the latest version of tables all at once
* invoke `alembic upgrade head`
=> the "e2e" tests are all run twice, once in each mode; this
ensures that the migration scripts re-create the same database
schema as db.Base.metadata.create_all() would
* in both modes, a temporary PostgreSQL schema is used to create the
tables in
=> could now run "e2e" tests against production database and still
have isolation
- make the configuration module public (to be used by Alembic)
- adjust linting rules for Alembic
- use SQLAlchemy (and PostgreSQL) to model the ORM layer
- add the following models:
+ Address => modelling all kinds of addresses
+ City => model the three target cities
+ Courier => model the UDP's couriers
+ Customer => model the UDP's customers
+ Order => model the orders received by the UDP
+ Restaurant => model the restaurants active on the UDP
- so far, the emphasis lies on expression the Foreign Key
and Check Constraints that are used to validate the assumptions
inherent to the cleanded data
- provide database-independent unit tests with 100% coverage
- provide additional integration tests ("e2e") that commit data to
a PostgreSQL instance to validate that the constraints work
- adapt linting rules a bit
- add the following file:
+ src/urban_meal_delivery/_config.py
- a config module is created holding two sets of configurations:
+ production => against the real database
+ testing => against a database with test data
- the module is "protected" (i.e., underscore) and imported at the
top level via a proxy-like object `config` that detects in which of
the two environments the package is being run
