diff --git a/paper.pdf b/paper.pdf index 7d6ed23..4bdce7d 100644 Binary files a/paper.pdf and b/paper.pdf differ diff --git a/paper.tex b/paper.tex index 333f2f1..4659df3 100644 --- a/paper.tex +++ b/paper.tex @@ -1,4 +1,4 @@ -\documentclass[preprint,review,12pt]{static/elsarticle} +\documentclass[preprint,review,12pt,authoryear]{static/elsarticle} \input{tex/preamble} @@ -42,9 +42,6 @@ \input{tex/5_con/4_further_research} \newpage -\input{tex/glossary} -\newpage - \appendix \newpage \input{tex/apx/tabular_ml_models} @@ -55,6 +52,8 @@ \newpage \input{tex/apx/peak_results} \newpage +\input{tex/apx/glossary} +\newpage \bibliographystyle{static/elsarticle-harv} \bibliography{tex/references} diff --git a/tex/1_intro.tex b/tex/1_intro.tex index 87d0171..c35cf91 100644 --- a/tex/1_intro.tex +++ b/tex/1_intro.tex @@ -11,8 +11,9 @@ A common feature of these platforms is that they do not operate kitchens but related processes in simple smartphone apps, and managing the delivery via a fleet of either employees or crowd-sourced sub-contractors. -Various kind of urban delivery platforms (UDP) have received attention in - recent scholarly publications. +Various kinds of urban delivery platforms + (\gls{udp}; \ref{glossary} provides a glossary with all abbreviations) + have received attention in recent scholarly publications. \cite{hou2018} look into heuristics to simultaneously optimize courier scheduling and routing in general, while \cite{masmoudi2018} do so for the popular dial-a-ride problem and \cite{wang2018} investigate @@ -63,8 +64,8 @@ Moreover, many of the established forecasting methods can only handle a single and often low seasonality (i.e., repeated regular pattern), if at all. In this paper, we develop a rigorous methodology as to how to build and - evaluate a robust forecasting system for an urban delivery platform - (\gls{udp}) that offers ad-hoc point-to-point transportation of any kind. + evaluate a robust forecasting system for an UDP + that offers ad-hoc point-to-point transportation of any kind. We implement such a system with a broad set of commonly used forecasting methods. We not only apply established (i.e., "classical") time series methods but also diff --git a/tex/2_lit/2_class/2_ets.tex b/tex/2_lit/2_class/2_ets.tex index 6db9781..53537da 100644 --- a/tex/2_lit/2_class/2_ets.tex +++ b/tex/2_lit/2_class/2_ets.tex @@ -1,4 +1,4 @@ -\subsubsection{Na\"{i}ve Methods, Moving Averages, and Exponential Smoothing.} +\subsubsection{Na\"{i}ve Methods, Moving Averages, and Exponential Smoothing} \label{ets} Simple forecasting methods are often employed as a benchmark for more diff --git a/tex/2_lit/2_class/3_arima.tex b/tex/2_lit/2_class/3_arima.tex index d55ffd8..3432b7e 100644 --- a/tex/2_lit/2_class/3_arima.tex +++ b/tex/2_lit/2_class/3_arima.tex @@ -1,4 +1,4 @@ -\subsubsection{Autoregressive Integrated Moving Averages.} +\subsubsection{Autoregressive Integrated Moving Averages} \label{arima} \cite{box1962}, \cite{box1968}, and more papers by the same authors in the diff --git a/tex/2_lit/2_class/4_stl.tex b/tex/2_lit/2_class/4_stl.tex index 127123c..39c987a 100644 --- a/tex/2_lit/2_class/4_stl.tex +++ b/tex/2_lit/2_class/4_stl.tex @@ -1,4 +1,4 @@ -\subsubsection{Seasonal and Trend Decomposition using Loess.} +\subsubsection{Seasonal and Trend Decomposition using Loess} \label{stl} A time series $y_t$ may exhibit different types of patterns; to fully capture diff --git a/tex/2_lit/3_ml/2_learning.tex b/tex/2_lit/3_ml/2_learning.tex index 86d157a..5e7c2bc 100644 --- a/tex/2_lit/3_ml/2_learning.tex +++ b/tex/2_lit/3_ml/2_learning.tex @@ -1,4 +1,4 @@ -\subsubsection{Supervised Learning.} +\subsubsection{Supervised Learning} \label{learning} A conceptual difference between classical and ML methods is the format diff --git a/tex/2_lit/3_ml/3_cv.tex b/tex/2_lit/3_ml/3_cv.tex index 1d5186b..abfdebc 100644 --- a/tex/2_lit/3_ml/3_cv.tex +++ b/tex/2_lit/3_ml/3_cv.tex @@ -1,4 +1,4 @@ -\subsubsection{Cross-Validation.} +\subsubsection{Cross-Validation} \label{cv} Because ML models are trained by minimizing a loss function $L$, the diff --git a/tex/2_lit/3_ml/4_rf.tex b/tex/2_lit/3_ml/4_rf.tex index 784d2a7..eeb7161 100644 --- a/tex/2_lit/3_ml/4_rf.tex +++ b/tex/2_lit/3_ml/4_rf.tex @@ -1,4 +1,4 @@ -\subsubsection{Random Forest Regression.} +\subsubsection{Random Forest Regression} \label{rf} \cite{breiman1984} introduce the classification and regression tree diff --git a/tex/2_lit/3_ml/5_svm.tex b/tex/2_lit/3_ml/5_svm.tex index 1c12af5..528fe9e 100644 --- a/tex/2_lit/3_ml/5_svm.tex +++ b/tex/2_lit/3_ml/5_svm.tex @@ -1,4 +1,4 @@ -\subsubsection{Support Vector Regression.} +\subsubsection{Support Vector Regression} \label{svm} \cite{vapnik1963} and \cite{vapnik1964} introduce the so-called support vector diff --git a/tex/3_mod/3_grid.tex b/tex/3_mod/3_grid.tex index baa0828..222b1ec 100644 --- a/tex/3_mod/3_grid.tex +++ b/tex/3_mod/3_grid.tex @@ -9,9 +9,11 @@ Figure \ref{f:grid} shows how the orders' delivery locations are each covering the entire service area within a city. This gridification step is also applied to the pickup locations separately. The lower-left corner is chosen at random. -\cite{winkenbach2015} apply the same gridification idea and slice an urban - area to model a location-routing problem, and \cite{singleton2017} portray - it as a standard method in the field of urban analytics. +Applications of this gridification idea to model location-routing problems + can be viewed, for example, in \cite{winkenbach2015}, \cite{bergmann2020}, + \cite{janjevic2019}, \cite{snoeck2020}, and \cite{janjevic2020} + while \cite{singleton2017} portray it as a standard method in the field of + urban analytics. With increasing pixel sizes, the time series exhibit more order aggregation with a possibly stronger demand pattern. On the other hand, the larger the pixels, the less valuable become the diff --git a/tex/3_mod/7_models/2_hori.tex b/tex/3_mod/7_models/2_hori.tex index 5d5959a..21cc627 100644 --- a/tex/3_mod/7_models/2_hori.tex +++ b/tex/3_mod/7_models/2_hori.tex @@ -1,4 +1,4 @@ -\subsubsection{Horizontal and Whole-day-ahead Forecasts.} +\subsubsection{Horizontal and Whole-day-ahead Forecasts} \label{hori} The upper-left in Figure \ref{f:inputs} illustrates the simplest way to diff --git a/tex/3_mod/7_models/3_vert.tex b/tex/3_mod/7_models/3_vert.tex index 5459b81..43aaaf1 100644 --- a/tex/3_mod/7_models/3_vert.tex +++ b/tex/3_mod/7_models/3_vert.tex @@ -1,4 +1,4 @@ -\subsubsection{Vertical and Whole-day-ahead Forecasts without Retraining.} +\subsubsection{Vertical and Whole-day-ahead Forecasts without Retraining} \label{vert} The upper-right in Figure \ref{f:inputs} shows an alternative way to diff --git a/tex/3_mod/7_models/4_rt.tex b/tex/3_mod/7_models/4_rt.tex index 75122cc..6fa038d 100644 --- a/tex/3_mod/7_models/4_rt.tex +++ b/tex/3_mod/7_models/4_rt.tex @@ -1,4 +1,4 @@ -\subsubsection{Vertical and Real-time Forecasts with Retraining.} +\subsubsection{Vertical and Real-time Forecasts with Retraining} \label{rt} The lower-left in Figure \ref{f:inputs} shows how models trained on vertical diff --git a/tex/3_mod/7_models/5_ml.tex b/tex/3_mod/7_models/5_ml.tex index 26f22a6..7ca00c4 100644 --- a/tex/3_mod/7_models/5_ml.tex +++ b/tex/3_mod/7_models/5_ml.tex @@ -1,4 +1,4 @@ -\subsubsection{Vertical and Real-time Forecasts without Retraining.} +\subsubsection{Vertical and Real-time Forecasts without Retraining} \label{ml_models} The lower-right in Figure \ref{f:inputs} shows how ML models take diff --git a/tex/4_stu/4_overall.tex b/tex/4_stu/4_overall.tex index 4d11718..c36c876 100644 --- a/tex/4_stu/4_overall.tex +++ b/tex/4_stu/4_overall.tex @@ -13,36 +13,6 @@ We labeled them "no", "low", "medium", and "high" demand pixels with increasing ADD, and present the average MASE per cluster. The $n$ do not vary significantly across the training horizons, which confirms that the platform did not grow area-wise and is indeed in a steady-state. -We use this table to answer \textbf{Q1} regarding the overall best methods - under different ADDs. -All result tables in the main text report MASEs calculated with all time - steps of a day. -In contrast, \ref{peak_results} shows the same tables with MASEs calculated - with time steps within peak times only (i.e., lunch from 12 pm to 2 pm and - dinner from 6 pm to 8 pm). -The differences lie mainly in the decimals of the individual MASE - averages while the ranks of the forecasting methods do not change except - in rare cases. -That shows that the presented accuracies are driven by the forecasting methods' - accuracies at peak times. -Intuitively, they all correctly predict zero demand for non-peak times. - -Unsurprisingly, the best model for pixels without demand (i.e., - $0 < \text{ADD} < 2.5$) is \textit{trivial}. -Whereas \textit{hsma} also adapts well, its performance is worse. -None of the more sophisticated models reaches a similar accuracy. -The intuition behind is that \textit{trivial} is the least distorted by the - relatively large proportion of noise given the low-count nature of the - time series. - -For low demand (i.e., $2.5 < \text{ADD} < 10$), there is also a clear - best-performing model, namely \textit{hsma}. -As the non-seasonal \textit{hses} reaches a similar accuracy as its - potentially seasonal generalization, the \textit{hets}, we conclude that - the seasonal pattern from weekdays is not yet strong enough to be - recognized in low demand pixels. -So, in the absence of seasonality, models that only model a trend part are - the least susceptible to the noise. \begin{center} \captionof{table}{Top-3 models by training weeks and average demand @@ -198,6 +168,38 @@ So, in the absence of seasonality, models that only model a trend part are \hline \end{tabular} \end{center} +\ + +We use this table to answer \textbf{Q1} regarding the overall best methods + under different ADDs. +All result tables in the main text report MASEs calculated with all time + steps of a day. +In contrast, \ref{peak_results} shows the same tables with MASEs calculated + with time steps within peak times only (i.e., lunch from 12 pm to 2 pm and + dinner from 6 pm to 8 pm). +The differences lie mainly in the decimals of the individual MASE + averages while the ranks of the forecasting methods do not change except + in rare cases. +That shows that the presented accuracies are driven by the forecasting methods' + accuracies at peak times. +Intuitively, they all correctly predict zero demand for non-peak times. + +Unsurprisingly, the best model for pixels without demand (i.e., + $0 < \text{ADD} < 2.5$) is \textit{trivial}. +Whereas \textit{hsma} also adapts well, its performance is worse. +None of the more sophisticated models reaches a similar accuracy. +The intuition behind is that \textit{trivial} is the least distorted by the + relatively large proportion of noise given the low-count nature of the + time series. + +For low demand (i.e., $2.5 < \text{ADD} < 10$), there is also a clear + best-performing model, namely \textit{hsma}. +As the non-seasonal \textit{hses} reaches a similar accuracy as its + potentially seasonal generalization, the \textit{hets}, we conclude that + the seasonal pattern from weekdays is not yet strong enough to be + recognized in low demand pixels. +So, in the absence of seasonality, models that only model a trend part are + the least susceptible to the noise. For medium demand (i.e., $10 < \text{ADD} < 25$) and training horizons up to six weeks, the best-performing models are the same as for low demand. diff --git a/tex/5_con/3_implications.tex b/tex/5_con/3_implications.tex index 6fe91a4..ce9c194 100644 --- a/tex/5_con/3_implications.tex +++ b/tex/5_con/3_implications.tex @@ -10,8 +10,9 @@ Somewhat surprisingly, despite ML-based methods` popularity in both business and academia in recent years, we must conclude that classical forecasting methods suffice to reach the best accuracy in our study. There is one case where ML-based methods are competitive in our case study: - In a high demand pixel, if only about four to six weeks of past data is - available, the \textit{vrfr} model outperformed the classical ones. + In a high demand pixel (defined as more than 25 orders per day on average), + if only about four to six weeks of past data is available, + the \textit{vrfr} model outperformed the classical ones. So, we recommend trying out ML-based methods in such scenarios. In addition, with the \textit{hsma} and \textit{hets} models being the overall winners, incorporating real-time data is not beneficial, in particular, @@ -53,6 +54,8 @@ We emphasize that for the most part, our proposed forecasting system is calibrated automatically and no manual work by a data scientist is required. The only two parameters where assumptions need to be made are the pixel size and the time step. -While they can only be optimized by the data scientist over time, the results in our - empirical study suggest that a pixel size of $1~\text{km}^2$ and a time step of - one hour are ideal. \ No newline at end of file +The results in our empirical study suggest + that a pixel size of $1~\text{km}^2$ and a time step of one hour are ideal, + which results in the optimal trade-off + between signal strength and spatial-temporal resolution. +Future research may explore adaptive grid-sizing depending on, for instance, demand density. \ No newline at end of file diff --git a/tex/glossary.tex b/tex/apx/glossary.tex similarity index 98% rename from tex/glossary.tex rename to tex/apx/glossary.tex index 63cb402..77eb8b3 100644 --- a/tex/glossary.tex +++ b/tex/apx/glossary.tex @@ -1,3 +1,6 @@ +\section{Glossary} +\label{glossary} + % Abbreviations for technical terms. \newglossaryentry{add}{ name=ADD, description={Average Daily Demand} @@ -138,4 +141,4 @@ trained on vertical time series} } -\printglossaries \ No newline at end of file +\printglossary[title=] \ No newline at end of file diff --git a/tex/meta.tex b/tex/meta.tex index d944b33..941fc1f 100644 --- a/tex/meta.tex +++ b/tex/meta.tex @@ -3,7 +3,7 @@ \journal{Transportation Research Part E} \title{Real-time Demand Forecasting for an Urban Delivery Platform} -\author[WHU]{Alexander Hess\fnref{emails}} +\author[WHU]{Alexander Hess\fnref{emails}\fnref{corresponding}} \author[WHU]{Stefan Spinler\fnref{emails}} \author[MIT]{Matthias Winkenbach\fnref{emails}} \address[WHU]{ @@ -21,20 +21,26 @@ Emails: mwinkenb@mit.edu } +\fntext[corresponding]{ +The corresponding author is Alexander Hess. +Use the provided email. +} + \begin{abstract} Meal delivery platforms like Uber Eats shape the landscape in cities around the world. -This paper addresses forecasting demand into the short-term future. +This paper addresses forecasting demand on a grid into the short-term future, + enabling, for example, predictive routing applications. We propose an approach incorporating - both classical forecasting - and machine learning methods. -Model evaluation and selection is adapted to demand typical for such a platform - (i.e., intermittent with a double-seasonal pattern). -The results of an empirical study with a European meal delivery service show - that machine learning models become competitive - once the average daily demand passes a threshold. -As a main contribution, the paper explains - how a forecasting system must be set up - to enable predictive routing. + both classical forecasting and machine learning methods + and adapt model evaluation and selection to typical demand: + intermittent with a double-seasonal pattern. +An empirical study shows that + an exponential smoothing based method trained on past demand data alone + achieves optimal accuracy, + if at least two months are on record. +With a more limited demand history, + machine learning is shown + to yield more accurate prediction results than classical methods. \end{abstract} \begin{keyword} diff --git a/tex/references.bib b/tex/references.bib index 8926f3f..3dca6ea 100644 --- a/tex/references.bib +++ b/tex/references.bib @@ -56,6 +56,17 @@ pages={8--15}, publisher={Elsevier} } +@article{bergmann2020, +title={Integrating first-mile pickup and last-mile delivery + on shared vehicle routes for efficient urban e-commerce distribution}, +author={Bergmann, Felix M and Wagner, Stephan M and Winkenbach, Matthias}, +year={2020}, +journal={Transportation Research Part B: Methodological}, +volume={131}, +pages={26--62}, +publisher={Elsevier} +} + @article{box1962, title={Some statistical Aspects of adaptive Optimization and Control}, author={Box, George and Jenkins, Gwilym}, @@ -376,6 +387,26 @@ pages={211--225}, publisher={INFORMS} } +@article{janjevic2019, +title={Integrating collection-and-delivery points + in the strategic design of urban last-mile e-commerce distribution networks}, +author={Janjevic, Milena and Winkenbach, Matthias and Merch{\'a}n, Daniel}, +year={2019}, +journal={Transportation Research Part E: Logistics and Transportation Review}, +volume={131}, +pages={37--67}, +publisher={Elsevier} +} + +@article{janjevic2020, +title={Designing Multi-tier, Multi-service-level, and Multi-modal + Last-Mile Distribution Networks for Omni-Channel Operations}, +author={Janjevic, Milena and Merchan, Daniel and Winkenbach, Matthias}, +year={2020}, +journal={European Journal of Operational Research}, +publisher={Elsevier} +} + @article{kim2016, title={A new Metric of Absolute Percentage Error for Intermittent Demand Forecasts}, @@ -559,6 +590,17 @@ number={3}, pages={199--222} } +@article{snoeck2020, +title={The value of physical distribution flexibility + in serving dense and uncertain urban markets}, +author={Snoeck, Andr{\'e} and Winkenbach, Matthias}, +year={2020}, +journal={Transportation Research Part A: Policy and Practice}, +volume={136}, +pages={151--177}, +publisher={Elsevier} +} + @article{stitson1999, title={Support Vector Regression with ANOVA Decomposition Kernels}, author={Stitson, Mark and Gammerman, Alex and Vapnik, Vladimir