Big Data entails innovative technological progress to the intelligence cycle as it strengthens the collection stage, introduces the correlational analysis method, and facilitates the dissemination of data to the final consumers. However, Big Data also presents some challenges and risks as human consciousness and expert participation remains essential to ensure the intelligence cycle’s effectiveness.
by Alejandra Bringas Colmenarejo
The inclusion of Big Data (BD) in the intelligence cycle has entailed a great advance since it introduced objective and quantitative methods in a discipline highly characterised by its subjectivity. In this sense, BD attempts to reduce intelligence uncertainty through the collection of a huge volume of data and the identification of hidden correlations unobservable in smaller samples. However, while BD is a beneficial technological advance of the intelligence cycle, it also leads to deep controversy given that policymakers may be tempted to replace the expert knowledge and the intelligence analysis with raw BD assets and correlations [1].
BD “represents the Information assets characterized by such a High Volume, Velocity and Variety to require specific Technology and Analytical Methods for its transformation into value” [2]. Consequently, BD is defined by the extremely large quantity of information collected in real-time and in continuous flows. Such information includes structured and unstructured data, traditional processed numeric and text databases, as well as unprocessed formats like images, audios, videos, tweets, emails and more [3]. Furthermore, BD also entails the necessary technologies to collect, manipulate, compare and analyse the collected bulk data and transform it into a reasoned intelligence assessment [4].
The inclusion of BD in the intelligence cycle has several challenges since it surpassed information, knowledge, casualty and context to centre the focus of attention on correlations [5]. Once its veracity and validity have been determined, the data collected from different sources is analysed to predict, determine or even prevent future scenarios, actions and behaviours [6]. Consequently, BD intelligence analysis is “the process of examining and interrogating Big Data assets to derive insights of value for decision making in a quasi-immediate response” [7]. However, this intelligence progress entails some risks and challenges since the increasing dependence on gathering technologies, as well as the enormous quantity of data collected, could result in a sense of overconfidence in technologies and a refusal of human capabilities.
Regarding intelligence collection, BD improves the inductive approach that attempts to recognize long-term trends, patterns and anomalies [8]. Different algorithms and informatics tools enable the automatization of collection, storage, management and transmission of data. This automatization decreases the dependence from manual processes and facilitates the continuous flows of data, [9] which strengthens the analysts’ capabilities to discover intelligence gaps or unusual behaviours. However, to avoid a paralysation of the intelligence process it is essential that the algorithms used are effective in selecting valid and useful data from the vast raw data collected [10].
BD also allows intelligence analysts to generate and refute hypotheses. BD analysis appears to be quite inductive since it refers to past events and historical patterns to causally respond to the question of ‘what is happening’. However, the value of BD lies in the correlation and the identification of hidden events and circumstances so that realities which may not be evident or observable become available to the intelligence analyst. Consequently, filtering valid information from the massive quantity of data allows analysts to support their speculations with facts or to deny a previously confirmed hypothesis [11]. The quick and real-time collection, as well as the long-term storage of data, provides analysts with the necessary evidence to develop informed and predictive intelligence hypotheses. In spite of that, the BD correlation process could also result in the identification of patterns and realities that extrapolated from their specific context are completely useless or coincidental. Consequently, intelligence agents should carefully use BD correlations as without the appropriate expertise analysis they could lead to irrelevant events or unconnected behaviours [12].
Despite the massive volume of data gathered by the intelligence actors, some information remains unknown and excluded from the correlation process because of its secrecy or its restricted access. In this context, non-state data collectors, such as social media platforms, marketing agencies or companies collect and store information that can be bought by the intelligence actors to fulfil the information gap. Nevertheless, the veracity and accuracy of this information remains dependent on the initial collectors [13]. As a result, data provided by private actors could involuntarily impact the effectiveness of the intelligence process or maliciously corrupt, manipulate and counterfeit the reality to deliberately influence the final intelligence assessment [14].
In this manner, BD remains dependent on human capabilities because it still lacks creativity, consciousness and judgement to contextualize new correlations within a broader analytical framework [15]. The limitations of BD should be understood completely in order to avoid misinterpretations and misunderstandings of reality. BD needs expert analysts who are able to identify mere coincidences and consider the unpredictable behaviour of human beings.
Concerning the relation between intelligence analysts and consumers, BD could play different roles. It could help disseminate relevant intelligent assessments to their effective consumers facilitating well-informed analysis and decision-making. Despite this progress in the dissemination stage, intelligence consumers may be sceptical about the veracity and validity of BD’s correlations. Consequently, they could ask for in-depth pattern’ explanations or even become reluctant to authorise action or enact policies supported by BD’s analysis [16]. Otherwise, consumers may be tempted to use raw data without the necessary subsequent analysis to support their own interest and purposes, contrary to the effectiveness of the intelligence cycle [17].
The challenges introduced by Big Data in the intelligence cycle are part of the existential debate between humans and technology and a logical consequence of the very speed of technological advances. Nevertheless, an even greater intelligence revolution could result from the next technological progress – the autonomy of artificial intelligence (AI). AI would collect BD in real-time, develop the consequent intelligence analysis and finally disseminate a reasoned assessment. Future BD analysis and AI would be able to reduce uncertainty and solve intelligence puzzles. However, the challenges and risks associated with this kind of technology are also undeniable since the human element in the intelligence cycle is reduced to the mere intelligence consumer. In the present time, BD does not possess human consciousness, however, full autonomy could be a reality in the near future [18].
Sources:
[1] Van Puyvelde, Damien, Stephen Coulthart, and M. Shahriar Hossain. “Beyond the buzzword: big data and national security decision-making.” International Affairs, 2017: 1397-1416.
[2] De Mauro, Andrea, Michele Grimaldi, and Marco Greco. (2014) “What is Big Data? A Consensual Definition and a Review of Key Research Topics.” 4th International Conference on Integrated Information. AIP Proceedings, pp. 1-11.
[3] Normandeau, K. (2013, September 12). Beyond Volume, Variety and Velocity is the Issue of Big Data Veracity. Available at https://insidebigdata.com/2013/09/12/beyond-volume-variety-velocity-issue-big-data-veracity/
[4] Boyd D. & Crawford. K. Critical Questions for Big Data. (Information, Communication and Society, 2012), p. 662-678
[5] Landon-Murray, M. (2016). Big Data and Intelligence: Applications, Human Capital, and Education. Journal of Strategic Security, 9(2), p.92-121.
[6] Lyon, D. (2014, July-December). Surveillance, Snowden, and Big Data: Capacities, consequences, critique. Big Data & Society, p.1-13. doi: 10.1177/2053951714541861
[7] Couch, N., & Robins, B. (2013). Big Data for Defence and Security. Royal United Services Institute for Defence and Security Studies, p.6.
[8] Lim, K. (2015). Big Data and Strategic Studies. Intelligence and National Security, p.619-635.
[9] Symon, P. B., & Tarapore, A. (2015). Defense Intelligence Analysis in the Age of Big Data. Joint Force Quarterly 79, p. 4-12
[10] Couch & Robins, p.9
[11] Lim, p. 636
[12] Landon-Murray, p.94
[13] Zwitter, A. (2015) Big Data and International Relations. Ethics & International Affairs, 29, no 4, pp. 377-389.
[14] Symon & Tarapore, p. 9.
[15] Dyndal, G. L., Berntsen, T. A., & Redse-Johansen, S. (2017, 28 July). Autonomous military drones: no longer science fiction. Available at NATO Review Magazine: https://www.nato.int/docu/review/2017/also-in-2017/autonomous-military-drones-no-longer-science-fiction/en/index.htm
[16] Landon-Murray, p.101.
[17] Jani, K. (2016). The Promise and Prejudice of Big Data in Intelligence Community. Sam Nunn School of International Affairs, p.14.
[18] Dyndal; Berntsen & Redse-Johansen.