A comparative study on predicting influenza outbreaks using different feature spaces: application of influenza-like illness data from Early Warning Alert and Response System in Syria

Objective An accurate forecasting of outbreaks of influenza-like illness (ILI) could support public health officials to suggest public health actions earlier. We investigated the performance of three different feature spaces in different models to forecast the weekly ILI rate in Syria using EWARS data from World Health Organization (WHO). Time series feature space was first used and we applied the seven models which are Naïve, Average, Seasonal naïve, drift, dynamic harmonic regression (Dhr), seasonal and trend decomposition using loess (STL) and TBATS. The Second feature space is like some state-of-the-art, which we named \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$53-weeks-before\_52-first-order-difference$$\end{document}53-weeks-before_52-first-order-difference feature space. The third one, we proposed and named \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$n-years-before\_m-weeks-around$$\end{document}n-years-before_m-weeks-around (YnWm) feature space. Machine learning (ML) and deep learning (DL) model were applied to the second and third feature spaces (generalized linear model (GLM), support vector regression (SVR), gradient boosting (GB), random forest (RF) and long short term memory (LSTM)). Results It was indicated that the LSTM model of four layers with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1-year-before\_4-weeks-around$$\end{document}1-year-before_4-weeks-around feature space gave more accurate results than other models and reached the lowest MAPE of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$3.52\%$$\end{document}3.52% and the lowest RMSE of 0.01662. I hope that this modelling methodology can be applied in other countries and therefore help prevent and control influenza worldwide.


Introduction
Influenza epidemic results in three to five million cases of severe illnesses and approximately 290,000 to 650,000 deaths worldwide each year [1]. WHO's Early Warning, Alert and Response System (EWARS) is designed to improve disease outbreak detection in emergency settings. The system has been built in SYRIA since 2012 to collect and detect near real-time information on several outbreaks including influenza [2]. The Existing researches on modeling influenza epidemic falls into two categories: Mechanistic and Statistical models. They are summarized in literature reviews [3][4][5][6] and in the CDC comparisons [7,8]. Researches under statistical category vary according to the different Features and methods used. Some researchers used the number of patients in the past as features [9][10][11][12][13], while others integrated other data sources to predict the number of patients in the future. Examples of these sources are climatological data [14,15], search engine queries [16][17][18][19], public comments on social media like Twitter [20,21], online informationseeking behavior on websites like Wikipedia [22,23] and a combination of multiple data streams [15,[24][25][26]. Different methods on these features were applied. Some researches treated the problem as an instance of more general time series forecasting using time series methods (ARIMA, ARIMA-STL, GARMA) [9,10,17,27], while others used ML methods including Stacked linear regression [24,26], AdaBoost regression with decision trees [26], GB [12], SVR [26,28], elastic net [28] RF [11,12,28], Artificial Neural Network (ANN) [12,20]. Recently, a DL method Called LSTM has attracted much interest in ILI prediction and gave excellent results, which are more accurate than those of other methods [12,13,15,29]. In addition to investigating the performance of the three different feature spaces with multiple time-series, ML and DL based methods to predict the weekly ILI rate in Syria; we proposed novel future spaces n − years − before_m − weeks − around that integrate into state-of-the-art ML and DL methods. There are two important contributions of this paper. First, the use of n − years − before_m − weeks − around future spaces to predict ILI rate. Second, analyzing multiple models performance over the EWARS data from WHO in Syria.

Materials and methods Data
We collected SYRIA flu data from the EWARS reports published by WHO on the website [30]. We only used the Flu Data from the first week of 2014 to the 42nd week of 2018. To avoid any possible population variations, we adopted the ILI rates as predictors (x) and responses (y) of our models. Figure 1a illustrates the raw data. The Y-axis represents the weekly ILI rate, and the X-axis represents the time series. The seasonality is obvious, as shown in Fig 1b. We split the data into two parts: the first 90% was the training set and the last 10% was the testing set.

Feature space
In this study, we reviewed three types of feature space. We named them as (time series feature space, 53 − weeks − before_52 − first − order − difference feature space and n − years − before_m − weeks − around feature space). Tables S1 and S2 illustrate the data set, response, predictors and the pretreatment of the source data. An additional pdf file shows this in more detail (see Additional file 1).

ILI rate = ILI number/Total number of illnesses
Time series feature space (fs1) We treated the ILI weekly data as time series with seasonal value 52.
53 − weeks − before_52 − first − order − differences feature space (fs2) Some previous studies found that using the ILI rate of the past 53 weeks and the 52 firstorder differences helped improve the results of the prediction models for influenza data [11][12][13].
n − years − before_m − weeks − around(YnWm) feature spaces (fs3) We reviewed a maximum of 3 years before and 5 weeks around. To predict the ILI rate of week WX of year YX, We used the ILI rate of the past m weeks before WX in YX, the ILI rate of past n seasonal weeks and the ILI rate of the m weeks around seasonal weeks. In the case of 1 − year − before_4 − weeks − around , the response y is the ILI rate of week 30 of year 2017 then the predictors are the ILI rates of weeks 29, 28, 27, 26 from year 2017 and the ILI rates of weeks 26, 27, 28, 29, 30, 31, 32, 33, 34 from year 2016. The number of predictors and rows changed depending on the value of n and m and we had to drop some data rows so that we get the same training data length for any value of n and m. Figure

Models
The models, programming languages, and libraries, which were used in this study were illustrated in Additional file 1. We trained all models in R Programming Language (version 3.4.4). For time series models, we used the "forecast" package (version 8.4). For ML models, we applied the caret package (Version 6.0-8). For DL models, we used the Keras package (Version 2.2.4) based on Tensorflow (Version 1.10). A personal computer with Intel I7-8550U processor, 8 GB of RAM and an NVIDIA 130 MX GPU was used for the experiments. Each experiment takes approximately 1 to 30 min to train the model. The prediction takes less than 5 s on the same hardware.

Metrics
We compared different models with different feature spaces using the mean absolute percentage error (MAPE) and root mean squared error (RMSE) as key performance indicators (KPIs).
where At is the actual value and Ft is the forecasted value.

fs1 with time series models
Seven methods (Average, Naïve, Seasonal Naïve, Drift, STL, DHR and TBATS) were applied in a recursive way. We trained the model on the training data set then predicted the ILI rate of week number x then we repeated the process by combining the actual ILI rate of week number x with the training data set. The metrics of all models are presented in Table 1(a), (b). it was indicated that the TBATS model outperformed other six models in predicting weekly ILI rate (MAPE = 4.66%, RMSE = 0.03096). Our explanation of the success of TBATS model is that the seasonality is allowed to change slowly over time in a TBATS model, while DHR, STL, Seasonal Naïve models force the seasonal patterns to repeat periodically without changing and Average, Naïve, Drift models do not depend on the seasonal patterns. We considered TBATS model a baseline in comparing the results of the following models.

fs2 with ML and DL models
We applied four ML methods (GLM, SVR, GB and RF) and two DL methods (LSTM with 3 layers and LSTM with 4 layers). Table 1(a), (b) shows that none of the models achieved better results than the baseline model in MAPE metric but in RMSE metric, the LSTM model    can learn the seasonality by n years before and can recognize the trend of the data by m weeks around. The temporal variation of the real weekly ILI rate and the predicted values obtained from the three models (the best result in each feature space) for the test period were plotted in Fig 2b. As can be seen, the predicted values of weekly ILI rate were in a good agreement with their related observed values and the used models could be used to model the weekly ILI rate. Moreover, LSTM model of 4 layers with fs3 resulted in better predicted values and trend between them for the observed values of ILI rate than the other models especially when there is a change in trend between observations.

Discussion
The accurate forecast of an outbreak of ILI could support public health officials in taking public health actions, such as allocating or temporarily readjusting medical resources for hospitals and medical centers. The ILI rate varies from year to year depending on multiple factors. Despite this difference, the ILI rate plot in each year takes a semi-constant form with an increase or decrease compared to previous years and could be predicted by using specific observations of past years with statistical models.
Performance of statistical models is data dependent and there is no model that performs well in all situations. Therefore, evaluating the performance of different models is of great importance as they provide useful and important information regarding strengths and weaknesses of the models and gives an insight to use better models for forecasting purposes. Some sate of the arts [11,12] utilize 53 weeks before with 52 first order difference with different statistical models and found that both the recent observations and the later observations with the difference were interesting and had significant influence on the predication. In this study, we proposed novel future spaces, namely n − years − before_m − weeks − around , and compared to some existing future spaces that utilize historical observations in different ways by integrating it into state-of-the-art ML and DL models. Our results revealed the success of our future space for some values of n and m and its failure for other values to outperform other future spaces in prediction ILI rate in Syria. This fact suggests that combining carefully selected number of Nearby Historical Observations with Carefully selected number of seasonal Historical Observations is advantageous over simply choosing all Historical Observations. While the results presented here are for ILI within  Syria, our novel future spaces shows promise to be easily extended to accurately track not only influenza in other countries but also other infectious diseases Through careful tuning of the values of n and m. We believe that it is up to policymakers in organizations concerned with EWARS to decide whether these forecasts are ready for use in decision support at the current level of accuracy.
In conclusion, we performed Naïve, Average, Seasonal naïve, Drift, DHR, STL, TBATS, GLM, SVR, GB, RF and LSTM methods with different feature spaces to predict the weekly ILI rate using EWARS data from WHO in Syria. We found that the TBATS method with time series feature space gave better results than those resulted from all methods in 53 − weeks − before_52 − first − order − differences feature space. We also found that the GLM, SVR, GB, RF and LSTM methods with a good choice to n and m in n − years − before_m − weeks − around feature space gave better results than those resulted in 53 − weeks − before_52 − first − order − differences feature space. In all the models, the LSTM model of 4 layers reached the lowest MAPE (3.52%) and the lowest RMSE (0.01662).

Limitations
Climatological data and Pharmaceutical Sales could be used to achieve better performance of the used models. We would like to investigate the impact of these parameters in future work.