Skip to main content

Forecasting AIDS prevalence in the United States using online search traffic data

Abstract

Over the past decade and with the increasing use of the Internet, the assessment of health issues using online search traffic data has become an integral part of Health Informatics. Internet data in general and from Google Trends in particular have been shown to be valid and valuable in predictions, forecastings, and nowcastings; and in detecting, tracking, and monitoring diseases’ outbreaks and epidemics. Empirical relationships have been shown to exist between Google Trends’ data and official data in several health topics, with the science of infodemiology using the vast amount of information available online for the assessment of public health and policy matters. The aim of this study is to provide a method of forecasting AIDS prevalence in the US using online search traffic data from Google Trends on AIDS related terms. The results at first show that significant correlations between Google Trends’ data and official health data on AIDS prevalence (2004–2015) exist in several States, while the estimated forecasting models for AIDS prevalence show that official health data and Google Trends data on AIDS follow a logarithmic relationship. Overall, the results of this study support previous work on the subject suggesting that Google data are valid and valuable for the analysis and forecasting of human behavior towards health topics, and could further assist with Health Assessment in the US and in other countries and regions with valid available official health data.

Introduction

Big Data, characterized by large volumes, high processing speed, and wide variety of datasets [1,2,3], have been shown to be very valuable in health care research, with Health Informatics being the field in which big data analytics have been extensively applied [4]. A popular way of addressing the challenge of Big Data is the analysis of online search traffic data [5, 6], mainly with data from Google Trends [7]. Over the past decade, this field of research, i.e., analyzing online search traffic data, has been widely used and is growing in popularity for assessing various topics, though it has mostly focused on the fields of Health and Medicine [8].

Many studies on the subject have empirically shown that Google Trends’ data are related to public health data. Topics that have been explored up to this point include the analysis, assessment, and prediction of epidemics and outbreaks, as, for example, Ebola [9, 10], Measles [11], the Bed-Bug epidemic [12], and Tuberculosis [13]. A much studied topic is that of influenza like illness (the flu), which is a seasonal disease and has shown well performing results in the past [14,15,16,17].

Recently, more topics on relating Google data with official health data have been visited, as in the case of suicide rates, where it has been show that Google queries can be used to monitor the risk of suicide [18, 19]. On a different direction, there has been shown that correlations exist between Google Trends data and prescription drugs issuing [20, 21] and revenues [22]. Apart from prescription drugs, focus has been given to illegal drugs as well, with notable examples including the tracking of dabbing in the US [23], Krokodil in Russia [24], and Methamphetamines in Central Europe [25].

According to Infodemiology [26], data available on the Internet can be used to inform public health and policy by monitoring the public’s behavior towards diseases, selecting the relevant available information, as well as monitoring how the public reacts to health marketing campaigns. Though it is widely supported and evident that official health data and online search traffic data are correlated, the most important step towards health assessment using Google Trends is that of finding methods of predicting and nowcasting diseases’ occurrence and outbreaks, as well as forecasting seasonal diseases’ prevalence.

Though seasonality has been assessed in various cases, such as, for information on tobacco and lung cancer [27], the restless legs syndrome [28], and in sleep-disordered breathing [29], studies developing methods towards the direction of forecasting and nowcasting exhibit significantly lower numbers. Despite that, recent research has exhibited promising results in the forecasting of various diseases and outbreaks, as, for example, Tuberculosis [13], influenza like illness [17], pertussis [30], suicide risk [18], and dementia [31].

As Infodemiology data can be retrieved in real time and thus allow the nowcasting of human behavior based on Internet data, the detection, monitoring, and prediction of epidemics and outbreaks can be much assisted by the analysis of Google queries. A topic that is of high significance and interest is that of AIDS (Acquired Immune Deficiency Syndrome) and HIV (Human Immunodeficiency Virus). HIV is a virus that is mainly transmitted via sexual intercourse and needle/syringe use [32]. The treatment for HIV consists of the antiretroviral therapy, which controls the HIV virus. If the HIV remains without treatment, it affects the immune system, which worsens as time passes. The HIV infection consists of 3 stages: (1) acute HIV infection, (2) clinical latency, and (3) AIDS; the latter being the most severe stage of the HIV infection [33], which leads to an increased number of ‘opportunistic infections’ [32].

People would more easily search for information online than consult a doctor in general. In the case of AIDS, as it is a sensitive subject, the anonymity provided by the Internet allows people to search for information online. Thus the monitoring of Internet data is essential in the overall assessment of AIDS prevalence in regions where Internet penetration is high, as in the case of the United States. Novel methods of assessment are needed, as data on ‘AIDS Prevalence’, ‘AIDS Diagnoses’, and ‘AIDS Deaths’ provided by the Centers for Disease Control and Prevention (CDC) are not available in real time, as gathering, analyzing and making these data available is a long process that takes over a year.

AIDS is categorized as an epidemic [34], and as such it needs constant assessment. The aim of this paper is to analyze the online interest in AIDS related terms and estimate forecasting models for AIDS prevalence in the US using data from Google Trends. The rest of this paper is structured as follows: the “Research methodology” section consists of the procedure of the data collection and methodology followed to analyze and forecast AIDS prevalence, in the “Results” section the results of the analysis are presented, the “Discussion” section consists of the discussion of the analysis, while the “Conclusions” section consists of the overall conclusions and future research suggestions.

Research methodology

Data

Data from Google Trends are downloaded online in ‘.csv’ format and are normalized over the selected time-frame as follows: “Search results are proportionate to the time and location of a query: Each data point is divided by the total searches of the geography and time range it represents, to compare relative popularity. Otherwise places with the most search volume would always be ranked highest. The resulting numbers are then scaled on a range of 0–100 based on a topic’s proportion to all searches on all topics. Different regions that show the same number of searches for a term will not always have the same total search volumes.” [35]. Google Trends is not case-sensitive, though takes into account spelling errors and accents. In this study, this effect is minimized, as the examined term, i.e. AIDS, is universal, not translated, and difficult to misspell. Note that data may slightly vary when retrieved at different time points.

Methods

The choice of terms is crucial for the robustness of the results when using online data [36]. In Google Trends, the four options below are available when retrieving data for the examined disease. The term’s online interest can be retrieved in the ‘Search Term’ form, i.e. include all queries that had the respective term, thereafter referred to as ‘AIDS (Search Term)’. In addition, Google Trends groups related queries under other search terms as well, which in this case are ‘AIDS (Illness)’. Finally, Google Trends also gives the option of including terms related to the topics of ‘Management of AIDS/HIV (Topic)’, and ‘Diagnosis of HIV/AIDS (Topic)’.

Analysis stages

At first, an overall assessment of all four available terms and topics’ variations in online interest is provided, so as to identify the option that would increase the validity of further analysis on the subject. The next step towards examining the possibility of forecasting AIDS prevalence and incidence, is to identify any existing correlations between Google data on related terms and topics and official health data for AIDS. In this study, data on 'AIDS Prevalence' (2004–2015) are retrieved by the CDC website [37]. Depending on the significance of the calculated Pearson correlations, the possibility of forecasting AIDS prevalence in the US will be assessed. Finally, forecasting models of AIDS prevalence based on Google Trends’ data for the US as well as for each 50 States plus DC are estimated.

Results

At first, an overall assessment of the online interest towards AIDS in the US is performed, followed by the exploring of the correlations between AIDS prevalence and Google Trends data in the US and each US State individually. Finally, forecasting models for AIDS prevalence in the US are estimated, at both national and State level, so as to elaborate on the usefulness of the tool in health assessment in the US.

AIDS online interest in the US

Figure 1 consists of the changes in the online interest in ‘AIDS (Search Terms)’ and ‘AIDS (Illness)’ from January 2004 to December 2015, while Fig. 2 depicts the monthly normalized online interest in the ‘Google Trends’ topics of ‘Diagnosis of HIV/AIDS’ and ‘Management of HIV/AIDS’ from January 2004 to December 2015.

Fig. 1
figure 1

Monthly Normalized Google Trends’ Data for ‘AIDS (Search Term)’ and ‘AIDS (Illness)’ from January 2004 to December 2015

Fig. 2
figure 2

Monthly normalized Google Trends’ Data for ‘Diagnosis of HIV/AIDS (Topic)’ and ‘Management of HIV/AIDS (Topic)’ from January 2004 to December 2015

The top related queries for ‘AIDS (Search Term)’ include ‘aids hiv’ (100), ‘hearing aids’ (99), ‘hiv’ (97), ‘aids symptoms’ (33), ‘aids and hiv’ (25), ‘aids day’ (24), ‘africa aids’ (22), ‘aids cure’ (16), ‘aids test’ (11), ‘aids statistics’ (11), and ‘aids virus’ (10). For ‘AIDS (Illness)’, the top related queries include ‘aids’ (100), ‘hiv’ (26), ‘aids hiv’ (14), ‘hiv/aids’ (6), ‘aids symptoms’ (5), ‘africa’ (4), ‘aids day’ (4), ‘hiv symptoms’ (3), ‘aids cure’ (2), ‘hiv infection’ (2), ‘hiv transmission’ (2), and ‘aids statistics’ (2).

For the topic of ‘Diagnosis of HIV/AIDS’, the top related queries include ‘hiv’ (100), ‘hiv test’ (53), ‘hiv testing’ (50), ‘free hiv testing’ (13), ‘test for hiv’ (11), ‘hiv symptoms’ (9), ‘hiv home test’ (7), ‘aids’ (6), ‘hiv aids’ (6), ‘hiv rapid test’ (4), ‘free hiv test’ (4), ‘hiv positive’ (4), ‘hiv test results’ (4), ‘positive hiv test’ (3), ‘rapid hiv testing’ (3), ‘hiv test kit’ (3), and ‘oraquick hiv test’ (2). For the topic ‘Management of HIV/AIDS’, the top related queries include ‘antiretroviral’ (100), ‘hiv’ (86), ‘aids’ (59), ‘antiretroviral therapy’ (58), ‘aids drugs’ (38), ‘antiretrovirals’ (28), ‘hiv treatment’ (23), ‘antiretroviral treatment’ (22), ‘hiv aids’ (20), ‘antiretroviral drugs’ (16), ‘hiv management’ (12), ‘highly active antiretroviral therapy’ (7), and ‘hiv medications’ (4).

Figure 3 consists of the heat maps of the online interest by US State from January 2004 to December 2015 for ‘AIDS (Search Term)’, ‘AIDS (Illness)’, ‘Diagnosis of HIV/AIDS (Topic)’, and ‘Management of HIV/AIDS (Topic)’.

Fig. 3
figure 3

Online Interest by State for ‘AIDS (Search Term)’, ‘AIDS (Illness)’, ‘Diagnosis of HIV/AIDS (Topic)’, and ‘Management of HIV/AIDS (Topic)’ from January 2004 to December 2015

It is evident that the terms related to AIDS exhibit high and constant interest from 2004 to 2015. The topics of ‘Diagnosis of HIV/AIDS (Topic)’ and ‘Management of HIV/AIDS (Topic)’ cover a narrow range of AIDS related terms and will thus not be included in further analysis.

AIDS prevalence vs. Google Trends

In order to examine the possibility of forecasting AIDS prevalence in the US, the relationships between online search traffic data from Google and official health data on AIDS prevalence are at first examined, by calculating the respective correlations at both national and State level. Depending on the significance of the correlations, the possibility of forecasting AIDS prevalence in the US will be examined. For the analysis of AIDS related queries, both Google Trends categories, i.e. ‘AIDS (Search Term)’ and ‘AIDS (Illness)’, are analyzed. Data for the categories ‘AIDS Deaths’, ‘AIDS Diagnoses’, and ‘AIDS Prevalence’ are available for 12 years, i.e. from January 2004 to December 2015.

Statistically significant correlations are observed between ‘AIDS Prevalence’ with both ‘AIDS (Search Term)’ (r = − 0.9508, p < 0.01) and with ‘AIDS (Illness)’ (r = − 0.9615, p < 0.01) in the US. For ‘AIDS (Search Term)’, statistically significant correlations are observed with ‘AIDS Diagnoses’ (r = 0.8743, p < 0.01), and with ‘AIDS Deaths’ (r = 0.9343, p < 0.01). Significant correlations are also identified for ‘AIDS Diagnoses’ with ‘AIDS (Illness)’ (r = 0.8945, p < 0.01), and for ‘AIDS Deaths’ with ‘AIDS (Illness)’ (r = 0.9423, p < 0.01). Therefore, we proceed to the next step of identifying correlations between online and health data in each US State.

Table 1 consists of the Pearson correlation coefficients (r) between ‘AIDS Prevalence’ and (a) ‘AIDS (Search Term)’ and (b) ‘AIDS (Illness)’ from January 2004 to December 2015, while Table 2 consists of the Pearson correlation coefficients (r) between ‘AIDS Diagnoses’ and a) ‘AIDS (Search Term)’ and b) ‘AIDS (Illness)’ from January 2004 to December 2015. Table 3 consists of the Pearson correlation coefficients (r) between ‘AIDS Deaths’, and a) ‘AIDS (Search Term)’ and b) ‘AIDS (Illness)’ from January 2004 to December 2015.

Table 1 Pearson correlation coefficients between ‘AIDS Prevalence’ and ‘AIDS (Search Term)’ and ‘AIDS (Illness)’ from January 2004 to December 2015
Table 2 Pearson correlation coefficients between ‘AIDS Diagnoses’ and (a) AIDS (Search Term)’ and (b) ‘AIDS (Illness)’ from January 2004 to December 2015
Table 3 Pearson correlation coefficients between ‘AIDS Deaths’ and (a) AIDS (Search Term)’ and (b) ‘AIDS (Illness)’ from January 2004 to December 2015

For ‘AIDS Prevalence’, all correlations are statistically significant. Therefore it is evident that the online behavior towards AIDS follows that of ‘AIDS Prevalence’. Thus the States that exhibit statistically significant correlations are further selected for the forecasting of AIDS in the US.

For ‘AIDS Diagnoses’, the States with significance of correlation of p < 0.01 in both examined terms are Arkansas, California, Connecticut, Delaware, DC, Florida, Illinois, Indiana, Maine, Maryland, Massachusetts, Michigan, Minnesota, Missouri, Nevada, New Hampshire, New Jersey, New York, Oklahoma, Oregon, Pennsylvania, Rhode Island, South Carolina, Washington, and West Virginia. For ‘AIDS Deaths’, the respective States are Arizona, California, Connecticut, Delaware, DC, Florida, Georgia, Illinois, Louisiana, Maryland, Massachusetts, Michigan, Mississippi, Missouri, New Jersey, New York, Pennsylvania, Tennessee, Texas, Utah, and Washington.

Forecasting AIDS prevalence in USA

As ‘AIDS Prevalence’ data are highly correlated with both ‘AIDS (Search Term)’ and with ‘AIDS (Illness)’ in all 50 States (plus DC), the next step is to examine the relationships between Google data and AIDS data and estimate the forecasting models. The relationship is logarithmic and of the form \(y\, = \,\alpha { \ln }\left( x \right)\, + \,\beta\), where y (y-axis-dependent variable) denotes the ‘AIDS Prevalence’, x (x-axis-independent variable) denotes the respective Google Trends’ data, namely ‘AIDS (Search Term)’ and ‘AIDS (Illness)’, and α and β are constants. To elaborate on the robustness of the estimated models, the R2 is selected, as it is the statistical measure by which the variable variation is explained. R2 takes values between 0 and 1 (i.e. 0% to 100%), and the higher the percentage, the better the fit.

Table 4 consists of the coefficients for the estimated logarithmic models for ‘AIDS Prevalence’ for both the examined Google Trends’ terms, i.e. ‘AIDS (Search Term)’ and ‘AIDS (Illness)’, while Figs. 4, 5, 6 and 7 depict the respective relationships in the US and in each individual State.

Table 4 Regression coefficients and R2 for the estimated forecasting models for ‘AIDS Prevalence’
Fig. 4
figure 4

‘AIDS Prevalence’ vs. Google Trends ‘AIDS (Search Term)’ and ‘AIDS (Illness)’ from January 2004 to December 2015 (USA; Alabama–Idaho)

Fig. 5
figure 5

‘AIDS Prevalence’ vs. Google Trends ‘AIDS (Search Term)’ and ‘AIDS (Illness)’ from January 2004 to December 2015 (Illinois–Montana)

Fig. 6
figure 6

‘AIDS Prevalence’ vs. Google Trends ‘AIDS (Search Term)’ and ‘AIDS (Illness)’ from January 2004 to December 2015 (Nebraska–South Carolina)

Fig. 7
figure 7

‘AIDS Prevalence’ vs. Google Trends ‘AIDS (Search Term)’ and ‘AIDS (Illness)’ from January 2004 to December 2015 (South Dakota–Wyoming)

In the US, the estimated models for ‘AIDS Prevalence’ based on the two examined terms have an R2 of 0.9695 and 0.9844, which shows that the relationship between AIDS prevalence and Google Trends data is well described using the estimated equations and that AIDS prevalence can be predicted based on online search traffic data from Google. Furthermore, most States’ models exhibit high R2 in at least one Google Trends’ category, which is indicative of the significance of the estimated forecasting models of AIDS prevalence in the US States.

Though in several States the R2 is higher for the respective linear or polynomial forecasting model, the relationship is overall logarithmic as clearly shown in the case of the US. Therefore, all estimated models for all categories and all individual States are calculated based on a logarithmic relationship independent of the value of R2, as this will be more evident when more years’ data are available.

The categories ‘AIDS Diagnoses’ and ‘AIDS Deaths’, though significant correlations with Google data are identified, are not included in further analysis, as the results are not significant for all States, though the respective analyses on said categories can be found in Appendix 1 and Appendix 2.

Discussion

The AIDS epidemic is a serious health issue and needs immediate and constant attention. In the Internet age, new methods for the monitoring and assessment of AIDS are required, so as to decrease the numbers of AIDS prevalence and deaths around the globe, and especially in developing countries. In this study, we provide a novel approach of monitoring online search traffic data retrieved from Google Trends in order to develop forecasting models for AIDS prevalence in the US.

Both examined Google terms, i.e. ‘AIDS (Search Term)’ and ‘AIDS (Illness)’, exhibited significant correlations with official data on ‘AIDS Prevalence’, ‘AIDS Diagnoses’, and ‘AIDS Deaths’, especially in the States where the AIDS rates are higher. Despite previous concerns on the reliability of Google Trends data [38], our results support research over the last decade showing that empirical relationships widely exist between Google Trends’ data and public health data records [5, 6, 9, 11, 20,21,22, 26, 39,40,41,42]. Therefore, the forecasting of AIDS prevalence is possible, as the estimated models for several States are robust despite the limitation of data being available for only 12 years. For ‘HIV (Search Term)’ and ‘HIV (Illness)’, though search volumes are high throughout the examined period, the correlations with official HIV data were not as statistically significant as in the case of AIDS, and were identified in fewer US States, which is an interesting topic to be examined in future research on the subject.

Table 5 consists of the coefficients and the R2 for the estimated forecasting logarithmic forecasting models of the form \(y\, = \,\alpha { \ln }\left( x \right)\, + \,\beta\) for States that exhibit high significance in all three categories, i.e. ‘AIDS Prevalence’, ‘AIDS Diagnoses’, and ‘AIDS Deaths’.

Table 5 Estimated Logarithmic forecasting models for USA and selected states

This study has some limitations. The estimated forecasting models are based on only 12 years’ data, thus the robustness of the models will increase when more years or smaller interval data are made officially available. In addition, we do not argue that each hit on the AIDS related keywords corresponds to an AIDS case and vise versa, as hits can also be attributed to general or academic interest, or increased interest due to an event, incident, or public figure that announces something related to the disease. Overall, the online interest towards AIDS increases according to the rates of AIDS prevalence (Appendix 3), thus it is expected for the forecasting models to be robust in the States for which the rates—and the online interest—are increased. Therefore, when more data are available, the significance will most probably increase.

Overall, this study highlights the importance of the analysis of online queries in order to better and more timely assess various issues in the US Health Care System. The estimated forecasting models on AIDS prevalence have very good performance, indicating that Google data can be of value in dealing with this sensitive subject, as we can this way have access to data that would not easily or at all been accessed with conventional methods.

Conclusions

This study aimed at introducing a novel approach in forecasting AIDS prevalence in the US using data from Google Trends on related terms. The results, exhibiting significant correlations between Google Trends’ data and official health data on AIDS (2004–2015) and high significance of the estimated forecasting models in several US States, support previous work on the subject suggesting that Google Trends’ data have been shown to be empirically related to health data and that they can assist with the analysis, monitoring, and forecasting of several health topics. This study, however, also addresses a more important issue; that of anonymity. A Google Trends important advantage is that it uses the revealed and not the stated data [37] in general, but in the case of AIDS the latter is even more important. As HIV and AIDS testing, diagnosis, and treatment is a sensitive subject, people may less easily go to the hospital or consult a doctor, health official, especially before testing and diagnosis.

Therefore, the monitoring of the interest towards States with increased rates of AIDS prevalence is essential, so that health officials can a) make relative information available on the Internet at time point e.g. with advertisements, b) take preventive measures, e.g. organizing event etc., and c) prepare the Health Care System accordingly, e.g. organize free testing outside of the hospitals. AIDS and HIV are terms that are not translated, not easily misspelled, and do not include accents or special characters. Thus, future research can include the application of this method in other countries and regions, as well as taking into consideration data retrieved by other online sources.

References

  1. Hilbert M, Lopez P. The World’s technological capacity to store, communicate, and compute information. Science. 2011;332:60–5.

    Article  Google Scholar 

  2. Chen CLP, Zhang CY. Data-intensive applications, challenges, techniques and technologies: a survey on big data. Inform Sci. 2014;275:314–47.

    Article  Google Scholar 

  3. Al Nuaimi E, Al Neyadi H, Mohamed N, Al-Jaroodi J. Applications of big data to smart cities. J Int Serv App. 2015;6:25.

    Article  Google Scholar 

  4. Matthew Herland M, Khoshgoftaar TM, Wald R. A review of data mining using big data in health informatics. J Big Data. 2014;1:2.

    Article  Google Scholar 

  5. Preis T, Moat HS, Stanley HE, Bishop SR. Quantifying the advantage of looking forward. Sci Rep. 2012;2:350.

    Article  Google Scholar 

  6. Preis T, Moat HS, Stanley HE. Quantifying trading behavior in financial markets using Google Trends. Sci Rep. 2013;3:1684.

    Article  Google Scholar 

  7. Google Trends. https://trends.google.com/trends/explore. Accessed 7 Feb 2018.

  8. Nuti SV, Wayda B, Ranasinghei I, Wang S, Dreyer RP, Chen SI, Murugiah K. The use of Google Trends in health care research: a systematic review. PLoS ONE. 2014;9:e109583.

    Article  Google Scholar 

  9. Alicino C, Bragazzi NL, Faccio V, Amicizia D, Panatto D, Gasparini R, Icardi G, Orsi A. Assessing Ebola-related web search behaviour: insights and implications from an analytical study of Google Trends-based query volumes. Infect Dis Poverty. 2015;4(1):54.

    Article  Google Scholar 

  10. Hossain L, Kam D, Kong F, Wigand RT, Bossomaier T. Social media in Ebola outbreak. Epidemiol Infect. 2016;144:2136–43.

    Article  Google Scholar 

  11. Mavragani A, Ochoa G. The internet and the anti-vaccine movement: tracking the 2017 EU measles outbreak. Big Data Cogn Comput. 2018;2(1):2.

    Article  Google Scholar 

  12. Sentana-Lledo D, Barbu CM, Ngo MN, Wu Y, Sethuraman K, Levy MZ. Seasons, searches, and intentions: what the internet can tell us about the bed bug (Hemiptera: Cimicidae) epidemic. J Med Entomol. 2016;53(1):116–21.

    Article  Google Scholar 

  13. Zhou X, Ye J, Feng Y. Tuberculosis surveillance by analyzing Google Trends. IEEE Trans Biomed Eng. 2011;58:2247–54.

    Article  Google Scholar 

  14. Kang M, Zhong H, He J, Rutherford S, Yang F. Using Google Trends for influenza surveillance in South China. PLoS ONE. 2013;8(1):e55205.

    Article  Google Scholar 

  15. Davidson MW, Haim DA, Radin JM. Using networks to combine big data and traditional surveillance to improve influenza predictions. Sci Rep. 2015;5:8154.

    Article  Google Scholar 

  16. Cho S, Sohn CH, Jo MW, Shin SY, Lee JH, Ryoo SM, Kim WY, Seo DW. Correlation between national influenza surveillance data and Google Trends in South Korea. PLoS ONE. 2013;8:e81422.

    Article  Google Scholar 

  17. Domnich A, Panatto D, Signori A, Lai PL, Gasparini R, Amicizia D. Age-related differences in the accuracy of web query-based predictions of influenza-like illness. PLoS ONE. 2015;10:0127754.

    Article  Google Scholar 

  18. Solano P, Ustulin M, Pizzorno E, Vichi M, Pompili M, Serafini G, Amore M. A Google-based approach for monitoring suicide risk. Psychiatry Res. 2016;246:581–6.

    Article  Google Scholar 

  19. Arora VS, Stuckler D, McKee M. Tracking search engine queries for suicide in the United Kingdom, 2004–2013. Public Health. 2016;137:147–53.

    Article  Google Scholar 

  20. Mavragani A, Sypsa K, Sampri A, Tsagarakis KP. Quantifying the UK online interest in substances of the EU watch list for water monitoring: diclofenac, estradiol, and the macrolide antibiotics. Water. 2016;8:542.

    Article  Google Scholar 

  21. Gahr M, Uzelac Z, Zeiss R, Connemann BJ, Lang D, Schönfeldt-Lecuona C. Linking annual prescription volume of antidepressants to corresponding web search query data: a possible proxy for medical prescription behavior? J Clin Psychopharmacol. 2015;235:681–5.

    Article  Google Scholar 

  22. Schuster NM, Rogers MA, McMahon LF Jr. Using search engine query data to track pharmaceutical utilization: a study of statins. Am J Manag Care. 2010;16:e215–9.

    Google Scholar 

  23. Zhang Z, Zheng X, Zeng DD, Leischow SJ. Tracking dabbing using search query surveillance: a case study in the United States. J Med Internet Res. 2016;18(9):e252.

    Article  Google Scholar 

  24. Zheluk A, Quinn C, Meylakhs P. Internet search and Krokodil in the Russian Federation: an infoveillance study. J Med Internet Res. 2014;16(9):e212.

    Article  Google Scholar 

  25. Gamma A, Schleifer R, Weinmann W, Buadze A, Liebren M. Could Google Trends be used to predict methamphetamine-related crime? An analysis of search volume data in Switzerland, Germany, and Austria. PLoS ONE. 2016;11(11):e0166566.

    Article  Google Scholar 

  26. Eysenbach G. Infodemiology and Infoveillance: framework for an emerging set of public health informatics methods to analyze search, communication and publication behavior on the internet. J Med Internet Res. 2009;11(1):e11.

    Article  Google Scholar 

  27. Zhang Z, Zheng X, Zeng DD, Leischow SJ. Information seeking regarding tobacco and lung cancer: effects of seasonality. PLoS ONE. 2015;10(3):e0117938.

    Article  Google Scholar 

  28. Ingram DG, Plante DT. Seasonal trends in restless legs symptomatology: evidence from internet search query data. Sleep Med. 2013;14(12):1364–8.

    Article  Google Scholar 

  29. Ingram DG, Matthews CK, Plante DT. Seasonal trends in sleep-disordered breathing: evidence from Internet search engine query data. Sleep Breath. 2015;19(1):79–84.

    Article  Google Scholar 

  30. Pollett S, Wood N, Boscardin WJ, Bengtsson H, Schwarcz S, Harriman K, Winter K, Rutherford G. Validating the use of Google Trends to enhance pertussis surveillance in California. PLoS Curr. 2015;19:7.

    Google Scholar 

  31. Wang HW, Chen DR, Yu HW, Chen YM. Forecasting the incidence of dementia and dementia-related outpatient visits with Google Trends: evidence from Taiwan. J Med Internet Res. 2015;17(11):e264.

    Article  Google Scholar 

  32. Centers for Disease Control and Prevention: HIV/AIDS. https://www.cdc.gov/hiv/basics.html/. Accessed 7 Feb 2018.

  33. What are HIV and AIDS? https://www.hiv.gov/hiv-basics/overview/about-hiv-and-aids/what-are-hiv-and-aids. Accessed 7 Feb 2018.

  34. UNAIDS. Fact sheet—latest statistics on the status of the AIDS epidemic. http://www.unaids.org/en/resources/fact-sheet. Accessed 7 Feb 2018.

  35. Google. Trends help. how trends data is adjusted. https://support.google.com/trends/answer/4365533?hl=en. Accessed 7 Feb 2018.

  36. Scharkow M, Vogelgesang J. Measuring the public agenda using search engine queries. Int J Public Opin Res. 2011;23:104–13.

    Article  Google Scholar 

  37. Atlas Plus. Centers for disease control and prevention. https://gis.cdc.gov/grasp/nchhstpatlas/main.html. Accessed 7 Feb 2018.

  38. Cervellin Gianfranco, Comelli Ivan, Lippi Giuseppe. Is Google Trends a reliable tool for digital epidemiology? Insights from different clinical settings. J Epidemiol Global Health. 2017;7:185–9.

    Article  Google Scholar 

  39. Mavragani A, Sampri A, Sypsa K, Tsagarakis KP. Integrating ‘Smart Health’ in the US Health Care System: asthma Monitoring in the Google Era. JMIR Public Health Surveill. 2018;4(1):e24.

    Article  Google Scholar 

  40. Jun SP, Park DH. Consumer information search behavior and purchasing decisions: empirical evidence from Korea. Technol Forecast Soc Change. 2016;31:97–111.

    Article  Google Scholar 

  41. Jun SP, Park DH, Yeom J. The possibility of using search traffic information to explore consumer product attitudes and forecast consumer preference. Technol Forecast Soc Change. 2014;86:237–53.

    Article  Google Scholar 

  42. Mavragani A, Tsagarakis KP. YES or NO: predicting the 2015 Greferendum results using Google Trends. Technol Forecast Soc. 2016;109:1–5.

    Article  Google Scholar 

Download references

Authors’ contributions

AM collected the data, performed the analysis, and wrote the paper. GO had the overall supervision. Both authors read and approved the final manuscript.

Authors’ information

Amaryllis Mavragani is a Ph.D. Candidate at the Department of Computing Science and Mathematics, University of Stirling. She holds a B.Sc in Mathematics from the University of Crete and an M.Sc from Democritus University of Thrace. Her research interests include Data Analysis, Mathematical Modeling, Online Behavior, Big Data, Public Health, Environmental Economics and Legislation, and Statistical Analysis.

Gabriela Ochoa is a Senior Lecturer at the Department of Computing Science and Mathematics, University of Stirling. She holds a Ph.D. in Computer Science and Artificial Intelligence from the University of Sussex, UK. Her research interests lie in the foundations and application of evolutionary algorithms and heuristic search methods, data science and visualization. She is an associated editor of both the IEEE Transactions on Evolutionary Computation and the Evolutionary Computation Journal, MIT Press.

Acknowledgements

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Availability of data and materials

All data used in this study are publicly available and accessible in the cited sources.

Consent for publication

The authors consent to the publication of this work.

Ethics approval and consent to participate

Not applicable.

Funding

Not applicable.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Amaryllis Mavragani.

Appendices

Appendix 1

AIDS diagnoses vs. Google Trends

Figure 8 depicts the scatterplots between ‘AIDS Diagnoses’ and both the examined Google terms, i.e. ‘AIDS (Search Term)’ and ‘AIDS (Illness)’, in the US and in the 25 States for which significant correlations with p < 0.01 were observed between AIDS and Google data. The States are Arkansas, California, Connecticut, Delaware, DC, Florida, Illinois, Indiana, Maine, Maryland, Massachusetts, Michigan, Minnesota, Missouri, Nevada, New Hampshire, New Jersey, New York, Oklahoma, Oregon, Pennsylvania, Rhode Island, South Carolina, Washington, and West Virginia.

Fig. 8
figure 8

‘AIDS Diagnoses’ vs. Google Trends ‘AIDS (Search Term)’ and ‘AIDS (Illness)’ from January 2004 to December 2015

Table 6 consists of the coefficients for the estimated logarithmic models for ‘AIDS Diagnoses’ for both the examined terms, namely ‘AIDS (Search Term)’ and ‘AIDS (Illness)’. As in ‘AIDS Prevalence’, the relationship between Google Trends and health data is logarithmic and of the form \(y\, = \,\alpha { \ln }\left( x \right)\, + \,\beta\).

Table 6 Coefficients α and β, and R2 for the estimated forecasting models for ‘AIDS Diagnoses’

For ‘AIDS Diagnoses’, the estimated forecasting models for ‘AIDS (Search Term)’ and ‘AIDS (Illness)’ in the US have an R2 of 0.8548 and 0.8982, respectively. It is therefore evident that the forecasting model for ‘AIDS Diagnoses’ in the US performs well, though not as well as in the ‘AIDS Prevalence’ category, which could be attributed to the more narrow –compared to AIDS prevalence—field that said category covers, which is also supported by the correlations in Table 2, which show that the ‘AIDS Diagnoses’ are not as significantly and in less States correlated with Google Trends’ data.

Appendix 2

AIDS Deaths vs. Google Trends

Figure 9 depicts the relationship between ‘AIDS Deaths’ and both the examined Google terms, i.e. ‘AIDS (Search Term)’ and ‘AIDS (Illness)’, in the US and in the 21 States for which significant correlations with p < 0.01 between AIDS data and Google Trends’ data were observed. These States are Arizona, California, Connecticut, Delaware, DC, Florida, Georgia, Illinois, Louisiana, Maryland, Massachusetts, Michigan, Mississippi, Missouri, New Jersey, New York, Pennsylvania, Tennessee, Texas, Utah, and Washington.

Fig. 9
figure 9

‘AIDS Deaths’ vs. Google Trends ‘AIDS (Search Term)’ and ‘AIDS (Illness)’ from January 2004 to December 2015

Table 7 consists of the coefficients for the estimated logarithmic models for ‘AIDS Deaths’ for both the examined Google Trends’ terms, i.e. ‘AIDS (Search Term)’ and ‘AIDS (Illness)’ for the aforementioned States.

Table 7 Coefficients α and β, and R2 for the estimated forecasting models for ‘AIDS Deaths’

Thus, as in the case of ‘AIDS Diagnoses’, when the AIDS data category is narrow, the forecasting results are robust in less States. Despite this, the forecasting models for the ‘AIDS Prevalence’ category exhibit significant results. Therefore, as more years’ data become available, the forecasting of AIDS Diagnoses and Deaths will be possible in more States.

Appendix 3

Forecasting model significance vs. AIDS rates in the US

Figure 10a maps the following five groups of significance of modeling by State: the first level—denoted by NC-consists of the States for which the correlations between health and Google data were not significant in all pairs of categories and thus not included for further analysis. The second group—denoted by C(0)-consists of the States for which significant correlations were identified in all categories, but the forecasting models had an R2 lower than 0.85 in all AIDS data categories. The third, forth, and fifth groups—denoted by C(1), C(2), and C(3), respectively- consist of the States for which significant correlations were identified in all categories, and the forecasting models’ R2 was above 0.85 in one (1), two (2), and three (3) AIDS data categories, respectively.

Fig. 10
figure 10

US states categorized by (a) correlations-estimated models’ significance and (b) AIDS rates

In order to elaborate on why some States exhibit low correlations and not significant forecasting models and why some others show very high correlations in addition to very significant forecasting models, we calculate the average of the AIDS prevalence yearly Rates for all US States excluding DC from 2004 to 2015 and divide them into 5 classes of equal intervals. Figure 10b maps said 5 classes of AIDS prevalence Rates’ in each US State. As is evident, a correspondence exists between the 1st class of AIDS prevalence rates, i.e. the group with the States that do not exhibit significant correlations between Google data in AIDS related terms with official data on AIDS prevalence, Diagnoses, and Deaths. In particular, the 1st class, i.e. with average yearly rates on AIDS prevalence from 2004 to 2015 of 16.81 to 99.10, consists of 29 out of the 51 States, namely Oregon, New Mexico, Arkansas, Indiana, Michigan, Ohio, Kentucky, Minnesota, Kansas, Utah, Alaska, Nebraska, West Virginia, Maine, New Hampshire, Wisconsin, Vermont, Iowa, Idaho, Montana, Wyoming, South Dakota, and North Dakota. Of those, only two exhibit significant correlations between public health and Google data, namely Michigan and Ohio. It is thus evident that the online interest towards AIDS increases according to the rates of AIDS prevalence, thus it is expected for the forecasting models to be robust in the States for which the rates are increased.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mavragani, A., Ochoa, G. Forecasting AIDS prevalence in the United States using online search traffic data. J Big Data 5, 17 (2018). https://doi.org/10.1186/s40537-018-0126-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40537-018-0126-7

Keywords