alexander/urban-meal-delivery-demand-forecasting
1
0
Fork 0
Code Activity

Add Model section

Browse source
This commit is contained in:
Alexander Hess 2020-10-04 23:39:20 +02:00
commit 91bd4ba083
Signed by: alexander
GPG key ID: 344EA5AB10D868E0
25 changed files with 1354 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

54
tex/apx/case_study.tex Normal file
View file

@ -0,0 +1,54 @@
\section{Raw Order Data in the Case Study}
\label{dataset}
The raw data for the empirical study in Section \ref{stu} was provided by a
meal delivery platform operating in five cities in France in 2016.
The platform received a total of 686,385 orders distributed as follows:
\
\begin{center}
\begin{tabular}{llr}
\hline
\thead{City} & \thead{Launch Day} & \thead{Orders} \\
\hline
Bordeaux & July 18 & 64,012 \\
Lille & October 30 & 14,362 \\
Lyon & February 21 & 214,635 \\
Nantes & October 31 & 12,900 \\
Paris & March 7 & 380,476 \\
\end{tabular}
\end{center}
\
The part of the database relevant for forecasting can be thought of as one
table per city, where each row represents one order and consists of the
following groups of columns:
\begin{enumerate}
\item \textbf{Restaurant Data}
\begin{enumerate}
\item unique ID and name
\item pickup location as latitude-longitude pair
\end{enumerate}
\item \textbf{Customer Data}
\begin{enumerate}
\item unique ID, name, and phone number
\item delivery location as latitude-longitude pair (mostly physical
addresses but also public spots)
\end{enumerate}
\item \textbf{Timestamps}
\begin{enumerate}
\item placement via the smartphone app
\item fulfillment workflow (pickup, delivery, cancellation, re-deliveries)
\end{enumerate}
\item \textbf{Courier Data}
\begin{enumerate}
\item unique ID, name, and phone number
\item shift data (begin, breaks, end)
\item average speed
\end{enumerate}
\item \textbf{Order Details}
\begin{enumerate}
\item meals and drinks
\item prices and discounts granted
\end{enumerate}
\end{enumerate}

121
tex/apx/enhanced_feats.tex Normal file
View file

@ -0,0 +1,121 @@
\section{Enhancing Forecasting Models with External Data}
\label{enhanced_feats}
In this appendix, we show how the feature matrix in Sub-section
\ref{ml_models} can be extended with features other than historical order
data.
Then, we provide an overview of what external data we tried out as predictors
in our empirical study.
\subsection{Enhanced Feature Matrices}
Feature matrices can naturally be extended by appending new feature columns
$x_{t,f}$ or $x_f$ on the right where the former represent predictors
changing throughout a day and the latter being static either within a
pixel or across a city.
$f$ refers to an external predictor variable, such as one of the examples
listed below.
In the SVR case, the columns should be standardized before fitting as external
predictors are most likely on a different scale than the historic order
data.
Thus, for a matrix with seasonally-adjusted order data $a_t$ in it, an
enhanced matrix looks as follows:
$$
\vec{y}
=
\begin{pmatrix}
a_T \\
a_{T-1} \\
\dots \\
a_{H+1}
\end{pmatrix}
~~~~~
\mat{X}
=
\begin{bmatrix}
a_{T-1} & a_{T-2} & \dots & a_{T-H} & ~~~
& x_{T,A} & \dots & x_{B} & \dots \\
a_{T-2} & a_{T-3} & \dots & a_{T-(H+1)} & ~~~
& x_{T-1,A} & \dots & x_{B} & \dots \\
\dots & \dots & \dots & \dots & ~~~
& \dots & \dots & \dots & \dots \\
a_H & a_{H-1} & \dots & a_1 & ~~~
& x_{H+1,A} & \dots & x_{B} & \dots
\end{bmatrix}
$$
\
Similarly, we can also enhance the tabular matrices from
\ref{tabular_ml_models}.
The same comments as for their pure equivalents in Sub-section \ref{ml_models}
apply, in particular, that ML models trained with an enhanced matrix can
process real-time data without being retrained.
\subsection{External Data in the Empirical Study}
\label{external_data}
In the empirical study, we tested four groups of external features that we
briefly describe here.
\vskip 0.1in
\textbf{Calendar Features}:
\begin{itemize}
\item Time of day (as synthesized integers: e.g., 1,050 for 10:30 am,
or 1,600 for 4 pm)
\item Day of week (as one-hot encoded booleans)
\item Work day or not (as booleans)
\end{itemize}
\vskip 0.1in
\textbf{Features derived from the historical Order Data}:
\begin{itemize}
\item Number of pre-orders for a time step (as integers)
\item 7-day SMA of the percentages of discounted orders (as percentages):
The platform is known for running marketing campaigns aimed at
first-time customers at irregular intervals. Consequently, the
order data show a wave-like pattern of coupons redeemed when looking
at the relative share of discounted orders per day.
\end{itemize}
\vskip 0.1in
\textbf{Neighborhood Features}:
\begin{itemize}
\item Ambient population (as integers) as obtained from the ORNL LandScan
database
\item Number of active platform restaurants (as integers)
\item Number of overall restaurants, food outlets, retailers, and other
businesses (as integers) as obtained from the Google Maps and Yelp
web services
\end{itemize}
\vskip 0.1in
\textbf{Real-time Weather} (raw data obtained from IBM's
Wunderground database):
\begin{itemize}
\item Absolute temperature, wind speed, and humidity
(as decimals and percentages)
\item Relative temperature with respect to 3-day and 7-day historical
means (as decimals)
\item Day vs. night defined by sunset (as booleans)
\item Summarized description (as indicators $-1$, $0$, and $+1$)
\item Lags of the absolute temperature and the summaries covering the
previous three hours
\end{itemize}
\vskip 0.1in
Unfortunately, we must report that none of the mentioned external data
improved the accuracy of the forecasts.
Some led to models overfitting the data, which could not be regulated.
Manual tests revealed that real-time weather data are the most promising
external source.
Nevertheless, the data provided by IBM's Wunderground database originate from
weather stations close to airports, which implies that we only have the
same aggregate weather data for the entire city.
If weather data is available on a more granular basis in the future, we see
some potential for exploitation.

261
tex/apx/peak_results.tex Normal file
View file

@ -0,0 +1,261 @@
\section{Forecasting Accuracies during Peak Times}
\label{peak_results}
This appendix shows all result tables from the main text with the MASE
averages calculated from time steps within peak times.
Peaks are the times of the day where the typical customer has a lunch or
dinner meal and defined to be either from 12 pm to 2 pm or from 6 pm to
8 pm.
While the exact decimals of the MASEs differ from the ones in the main
text, the relative ranks of the forecasting methods are the same except in
rare cases.
\begin{center}
\captionof{table}{Top-3 models by training weeks and average demand
($1~\text{km}^2$ pixel size, 60-minute time steps)}
\label{t:results:a}
\begin{tabular}{|c|c|*{12}{c|}}
\hline
\multirow{3}{*}{\rotatebox{90}{\thead{Training}}}
& \multirow{3}{*}{\rotatebox{90}{\thead{Rank}}}
& \multicolumn{3}{c|}{\thead{No Demand}}
& \multicolumn{3}{c|}{\thead{Low Demand}}
& \multicolumn{3}{c|}{\thead{Medium Demand}}
& \multicolumn{3}{c|}{\thead{High Demand}} \\
~ & ~
& \multicolumn{3}{c|}{(0 - 2.5)}
& \multicolumn{3}{c|}{(2.5 - 10)}
& \multicolumn{3}{c|}{(10 - 25)}
& \multicolumn{3}{c|}{(25 - $\infty$)} \\
\cline{3-14}
~ & ~
& Method & MASE & $n$
& Method & MASE & $n$
& Method & MASE & $n$
& Method & MASE & $n$ \\
\hline \hline
\multirow{3}{*}{3} & 1
& \textbf{\textit{trivial}}
& 0.794 & \multirow{3}{*}{\rotatebox{90}{4586}}
& \textbf{\textit{hsma}}
& 0.817 & \multirow{3}{*}{\rotatebox{90}{2975}}
& \textbf{\textit{hsma}}
& 0.838 & \multirow{3}{*}{\rotatebox{90}{2743}}
& \textbf{\textit{rtarima}}
& 0.871 & \multirow{3}{*}{\rotatebox{90}{2018}} \\
~ & 2
& \textit{hsma} & 0.808 & ~
& \textit{hses} & 0.847 & ~
& \textit{hses} & 0.851 & ~
& \textit{rtses} & 0.872 & ~ \\
~ & 3
& \textit{pnaive} & 0.938 & ~
& \textit{hets} & 0.848 & ~
& \textit{hets} & 0.853 & ~
& \textit{rtets} & 0.874 & ~ \\
\hline
\multirow{3}{*}{4} & 1
& \textbf{\textit{trivial}}
& 0.791 & \multirow{3}{*}{\rotatebox{90}{4532}}
& \textbf{\textit{hsma}}
& 0.833 & \multirow{3}{*}{\rotatebox{90}{3033}}
& \textbf{\textit{hsma}}
& 0.839 & \multirow{3}{*}{\rotatebox{90}{2687}}
& \textbf{\textit{vrfr}}
& 0.848 & \multirow{3}{*}{\rotatebox{90}{2016}} \\
~ & 2
& \textit{hsma} & 0.794 & ~
& \textit{hses} & 0.838 & ~
& \textit{hses} & 0.847 & ~
& \textbf{\textit{rtarima}} & 0.851 & ~ \\
~ & 3
& \textit{pnaive} & 0.907 & ~
& \textit{hets} & 0.841 & ~
& \textit{hets} & 0.851 & ~
& \textit{rtses} & 0.857 & ~ \\
\hline
\multirow{3}{*}{5} & 1
& \textbf{\textit{trivial}}
& 0.782 & \multirow{3}{*}{\rotatebox{90}{4527}}
& \textbf{\textit{hsma}}
& 0.844 & \multirow{3}{*}{\rotatebox{90}{3055}}
& \textbf{\textit{hsma}}
& 0.841 & \multirow{3}{*}{\rotatebox{90}{2662}}
& \textbf{\textit{vrfr}}
& 0.849 & \multirow{3}{*}{\rotatebox{90}{2019}} \\
~ & 2
& \textit{hsma} & 0.802 & ~
& \textit{hses} & 0.851 & ~
& \textit{hets} & 0.844 & ~
& \textbf{\textit{rtarima}} & 0.851 & ~ \\
~ & 3
& \textit{pnaive} & 0.888 & ~
& \textit{hets} & 0.863 & ~
& \textit{hses} & 0.845 & ~
& \textit{vsvr} & 0.853 & ~ \\
\hline
\multirow{3}{*}{6} & 1
& \textbf{\textit{trivial}}
& 0.743 & \multirow{3}{*}{\rotatebox{90}{4470}}
& \textbf{\textit{hsma}}
& 0.843 & \multirow{3}{*}{\rotatebox{90}{3086}}
& \textbf{\textit{hsma}}
& 0.841 & \multirow{3}{*}{\rotatebox{90}{2625}}
& \textbf{\textit{vrfr}}
& 0.844 & \multirow{3}{*}{\rotatebox{90}{2025}} \\
~ & 2
& \textit{hsma} & 0.765 & ~
& \textit{hses} & 0.853 & ~
& \textit{hses} & 0.844 & ~
& \textbf{\textit{hets}} & 0.847 & ~ \\
~ & 3
& \textit{pnaive} & 0.836 & ~
& \textit{hets} & 0.861 & ~
& \textit{hets} & 0.844 & ~
& \textit{vsvr} & 0.849 & ~ \\
\hline
\multirow{3}{*}{7} & 1
& \textbf{\textit{trivial}}
& 0.728 & \multirow{3}{*}{\rotatebox{90}{4454}}
& \textbf{\textit{hsma}}
& 0.855 & \multirow{3}{*}{\rotatebox{90}{3132}}
& \textbf{\textit{hets}}
& 0.843 & \multirow{3}{*}{\rotatebox{90}{2597}}
& \textbf{\textit{hets}}
& 0.839 & \multirow{3}{*}{\rotatebox{90}{2007}} \\
~ & 2
& \textit{hsma} & 0.744 & ~
& \textit{hses} & 0.862 & ~
& \textit{hsma} & 0.845 & ~
& \textbf{\textit{vrfr}} & 0.842 & ~ \\
~ & 3
& \textit{pnaive} & 0.812 & ~
& \textit{hets} & 0.868 & ~
& \textbf{\textit{vsvr}} & 0.849 & ~
& \textit{vsvr} & 0.846 & ~ \\
\hline
\multirow{3}{*}{8} & 1
& \textbf{\textit{trivial}}
& 0.736 & \multirow{3}{*}{\rotatebox{90}{4402}}
& \textbf{\textit{hsma}}
& 0.865 & \multirow{3}{*}{\rotatebox{90}{3159}}
& \textbf{\textit{hets}}
& 0.843 & \multirow{3}{*}{\rotatebox{90}{2575}}
& \textbf{\textit{hets}}
& 0.837 & \multirow{3}{*}{\rotatebox{90}{2002}} \\
~ & 2
& \textit{hsma} & 0.759 & ~
& \textit{hets} & 0.874 & ~
& \textbf{\textit{vsvr}} & 0.848 & ~
& \textbf{\textit{vrfr}} & 0.841 & ~ \\
~ & 3
& \textit{pnaive} & 0.820 & ~
& \textit{hses} & 0.879 & ~
& \textit{hsma} & 0.850 & ~
& \textit{vsvr} & 0.847 & ~ \\
\hline
\end{tabular}
\end{center}
\begin{center}
\captionof{table}{Ranking of benchmark and horizontal models
($1~\text{km}^2$ pixel size, 60-minute time steps):
the table shows the ranks for cases with $2.5 < ADD < 25$
(and $25 < ADD < \infty$ in parentheses if they differ)}
\label{t:hori:a}
\begin{tabular}{|c|ccc|cccccccc|}
\hline
\multirow{2}{*}{\rotatebox{90}{\thead{\scriptsize{Training}}}}
& \multicolumn{3}{c|}{\thead{Benchmarks}}
& \multicolumn{8}{c|}{\thead{Horizontal (whole-day-ahead)}} \\
\cline{2-12}
~ & \textit{naive} & \textit{fnaive} & \textit{paive}
& \textit{harima} & \textit{hcroston} & \textit{hets} & \textit{hholt}
& \textit{hhwinters} & \textit{hses} & \textit{hsma} & \textit{htheta} \\
\hline \hline
3 & 11 & 7 (2) & 8 (5) & 5 (7) & 4 & 3
& 9 (10) & 10 (9) & 2 (6) & 1 & 6 (8) \\
4 & 11 & 7 (2) & 8 (3) & 5 (6) & 4 (5) & 3 (1)
& 9 (10) & 10 (9) & 2 (8) & 1 (4) & 6 (7) \\
5 & 11 & 7 (2) & 8 (4) & 5 (3) & 4 (9) & 3 (1)
& 9 (10) & 10 (5) & 2 (8) & 1 (6) & 6 (7) \\
6 & 11 & 8 (5) & 9 (6) & 5 (4) & 4 (7) & 2 (1)
& 10 & 7 (2) & 3 (8) & 1 (9) & 6 (3) \\
7 & 11 & 8 (5) & 10 (6) & 5 (4) & 4 (7) & 2 (1)
& 9 (10) & 7 (2) & 3 (8) & 1 (9) & 6 (3) \\
8 & 11 & 9 (5) & 10 (6) & 5 (4) & 4 (7) & 2 (1)
& 8 (10) & 7 (2) & 3 (8) & 1 (9) & 6 (3) \\
\hline
\end{tabular}
\end{center}
\
\begin{center}
\captionof{table}{Ranking of classical models on vertical time series
($1~\text{km}^2$ pixel size, 60-minute time steps):
the table shows the ranks for cases with $2.5 < ADD < 25$
(and $25 < ADD < \infty$ in parentheses if they differ)}
\label{t:vert:a}
\begin{tabular}{|c|cc|ccccc|ccccc|}
\hline
\multirow{2}{*}{\rotatebox{90}{\thead{\scriptsize{Training}}}}
& \multicolumn{2}{c|}{\thead{Benchmarks}}
& \multicolumn{5}{c|}{\thead{Vertical (whole-day-ahead)}}
& \multicolumn{5}{c|}{\thead{Vertical (real-time)}} \\
\cline{2-13}
~ & \textit{hets} & \textit{hsma} & \textit{varima} & \textit{vets}
& \textit{vholt} & \textit{vses} & \textit{vtheta} & \textit{rtarima}
& \textit{rtets} & \textit{rtholt} & \textit{rtses} & \textit{rttheta} \\
\hline \hline
3 & 2 (10) & 1 (7) & 6 (4) & 8 (6) & 10 (9)
& 7 (5) & 11 (12) & 4 (1) & 5 (3) & 9 (8) & 3 (2) & 12 (11) \\
4 & 2 (7) & 1 (10) & 6 (4) & 8 (6) & 10 (9)
& 7 (5) & 12 (11) & 3 (1) & 5 (3) & 9 (8) & 4 (2) & 11 (12) \\
5 & 2 (3) & 1 (10) & 7 (5) & 8 (7) & 10 (9)
& 6 & 11 & 4 (1) & 5 (4) & 9 (8) & 3 (2) & 12 \\
6 & 2 (1) & 1 (10) & 6 (5) & 8 (7) & 10 (9)
& 7 (6) & 11 (12) & 3 (2) & 5 (4) & 9 (8) & 4 (3) & 12 (11) \\
7 & 2 (1) & 1 (10) & 8 (5) & 7 & 10 (9)
& 6 & 11 (12) & 5 (2) & 4 & 9 (8) & 3 & 12 (11) \\
8 & 2 (1) & 1 (9) & 8 (5) & 7 & 10 (8)
& 6 & 12 (10) & 5 (2) & 4 & 9 (6) & 3 & 11 \\
\hline
\end{tabular}
\end{center}
\
\pagebreak
\begin{center}
\captionof{table}{Ranking of ML models on vertical time series
($1~\text{km}^2$ pixel size, 60-minute time steps):
the table shows the ranks for cases with $2.5 < ADD < 25$
(and $25 < ADD < \infty$ in parentheses if they differ)}
\label{t:ml:a}
\begin{tabular}{|c|cccc|cc|}
\hline
\multirow{2}{*}{\rotatebox{90}{\thead{\scriptsize{Training}}}}
& \multicolumn{4}{c|}{\thead{Benchmarks}}
& \multicolumn{2}{c|}{\thead{ML}} \\
\cline{2-7}
~ & \textit{fnaive} & \textit{hets} & \textit{hsma}
& \textit{rtarima} & \textit{vrfr} & \textit{vsvr} \\
\hline \hline
3 & 6 & 2 (5) & 1 (3) & 3 (1) & 5 (2) & 4 \\
4 & 6 (5) & 2 (3) & 1 (6) & 3 (2) & 5 (1) & 4 \\
5 & 6 (5) & 2 (4) & 1 (6) & 4 (2) & 5 (1) & 3 \\
6 & 6 (5) & 2 & 1 (6) & 4 & 5 (1) & 3 \\
7 & 6 (5) & 2 (1) & 1 (6) & 4 & 5 (2) & 3 \\
8 & 6 (5) & 2 (1) & 1 (6) & 4 & 5 (2) & 3 \\
\hline
\end{tabular}
\end{center}
\

58
tex/apx/tabular_ml_models.tex Normal file
View file

@ -0,0 +1,58 @@
\section{Tabular and Real-time Forecasts without Retraining}
\label{tabular_ml_models}
Regarding the structure of the feature matrix for the ML models in Sub-section
\ref{ml_models}, we provide an alternative approach that works without
the STL method.
Instead of decomposing a time series and arranging the resulting
seasonally-adjusted time series $a_t$ into a matrix $\mat{X}$, one can
create a matrix with two types of feature columns mapped to the raw
observations in $\vec{y}$:
While the first group of columns takes all observations of the same time of
day over a horizon of, for example, one week ($n_h=7$), the second group
takes all observations covering a pre-defined time horizon, for example
$3$ hours ($n_r=3$ for 60-minute time steps), preceding the time step to
be fitted.
Thus, we exploit the two-dimensional structure of time tables as well, and
conceptually model historical and recent demand.
The alternative feature matrix appears as follows where the first three
columns are the historical and the last three the recent demand features:
$$
\vec{y}
=
\begin{pmatrix}
y_T \\
y_{T-1} \\
\dots \\
y_{1+n_hH}
\end{pmatrix}
~~~~~
\mat{X}
=
\begin{bmatrix}
y_{T-H} & y_{T-2H} & \dots & y_{T-n_hH}
& y_{T-1} & y_{T-2} & \dots & y_{T-n_r} \\
y_{T-1-H} & y_{T-1-2H} & \dots & y_{T-1-n_hH}
& y_{T-2} & y_{T-3} & \dots & y_{T-n_r-1} \\
\dots & \dots & \dots & \dots
& \dots & \dots & \dots & \dots \\
y_{1+(n_h-1)H} & y_{1+(n_h-2)H} & \dots & y_1
& y^*_{1+n_hH-1} & y^*_{1+n_hH-2} & \dots & y^*_{1+n_hH-n_r}
\end{bmatrix}
$$
\
Being a detail, we note that the recent demand features lying on the end of
the previous day are set to $0$, which is shown with the $^*$ notation
above.
This alignment of the undecomposed order data $y_t$ ensures that the ML
models learn the two seasonal patterns independently.
The parameters $n_h$ and $n_r$ must be adapted to the data, but we found the
above values to work well.
As such matrices resemble time tables, we refer to them as tabular.
However, we found the ML models with vertical time series to outperform the
tabular ML models, which is why we disregarded them in the study.
This tabular form could be beneficial for UDPs with a demand that exhibits
a weaker seasonality such as a meal delivery platform.