Lessons learned from applying a deep neural network to Twitter posts in order to estimate potentials of using big data to monitor the United Nations Sustainable Development Goals.
With the emergence of the Internet of Things (IoT) and the extensive amount of data produced by it, science's desire to investigate this vast amount of untapped data is growing, resulting in the paradigm of big data: data sets of exceedingly large volumes, growing at exceptional rates, consisting of enormous amounts of structured and unstructured data. At the same time, artificial intelligence (AI) techniques needed to analyze data sets of these proportions continue to improve.
The potentials attributed to big data analyses are extensive, particularly in the context of efficiently generating reliable, up-to-date data to measure progress towards the Agenda 2030's Sustainable Development Goals (SDGs). However, many scientific contributions in this domain, focusing on unexploited capacities, rely on future technological progress and therefore project prospective potentials. Yet, the SDGs were designed to tackle current global challenges.
For some of the indicators of sustainability introduced with the SDGs, it is still unclear how reliable data can efficiently be generated. Therefore, this study examines current technological capabilities and their potential contribution to overcoming a lack of data. It does so with an example of a big data analysis: applying an image classification algorithm (deep neural network) to geolocated media content posted to Twitter, in order to both illustrate the current potentials of such an approach, as well as challenges left to overcome if big data is to be used to generate useful information for measuring progress towards the SDGs.
The findings of this study show that current technological capabilities already facilitate real-time analyses of big data from social media on a global scale. Yet, biases within the data, resulting from uncertainties regarding the accuracy of geolocated social media posts, along with low internet penetration rates and a consequent lack of data - coupled with an unavailability of data from prime sources like Facebook and Instagram - render such analyses incomplete, thus diminishing the significance of information gained this way.
Better access to more data from diverse sources is needed to improve on our current capacities to generate reliable data to monitor progress towards improving sustainability. However, especially analyses of data from social media are embedded in a debate over privacy and data protection. This debate is here to stay. Nevertheless, some of the reservations against artificial intelligence and big data analyses can be alleviated by a high degree of transparency (i.e. by making big data projects open source).