<p id="p0007" style="margin-bottom: 16px; padding: 0px; color: rgb(46, 46, 46); font-family: NexusSerif, Georgia, &quot;Times New Roman&quot;, Times, STIXGeneral, &quot;Cambria Math&quot;, &quot;Lucida Sans Unicode&quot;, &quot;Microsoft Sans Serif&quot;, &quot;Segoe UI Symbol&quot;, &quot;Arial Unicode MS&quot;, serif; font-size: 18px;"><span style="font-family: Helvetica; font-size: 14px;">An intelligence analyst’s job is to construct coherent hypotheses despite significant gaps and inconsistencies in gathering evidence, and present them clearly to decision-makers to inform their interventions</span><span style="font-family: Helvetica; font-size: 14px;">.</span></p><p id="p0008" style="margin-bottom: 16px; padding: 0px; color: rgb(46, 46, 46); font-family: NexusSerif, Georgia, &quot;Times New Roman&quot;, Times, STIXGeneral, &quot;Cambria Math&quot;, &quot;Lucida Sans Unicode&quot;, &quot;Microsoft Sans Serif&quot;, &quot;Segoe UI Symbol&quot;, &quot;Arial Unicode MS&quot;, serif; font-size: 18px;"><span style="font-family: Helvetica; font-size: 14px;">Current automated systems to support the day-to-day practice of analysts are (almost) exclusively focused on two aspects of the problem. The first is data collection, aggregation, and visualization</span><span style="font-family: Helvetica; font-size: 14px;">. Such tools help analysts collate, inspect and interact with a large dataset, and support the identification of relationships, for example through link analysis</span><span style="font-family: Helvetica; font-size: 14px;">. Recently, crowdsourcing tools that enable the public to contribute information have also been introduced to integrate more traditional intelligence collection approaches.</span><span style="font-family: Helvetica; font-size: 14px;">&nbsp;The second problem on which tools focus involves listing and weighing up alternative hypotheses</span><span style="font-family: Helvetica; font-size: 14px;">, through automated analysis of competing hypotheses</span><span style="font-family: Helvetica; font-size: 14px;">. This analysis requires that all alternative hypotheses be identified from available evidence, and, if aided by automated inferential reasoners such as Bayesian networks</span><span style="font-family: Helvetica; font-size: 14px;">, the tools also require that each (aggregated) piece of evidence is given a weight or a degree of certainty.</span></p><p id="p0009" style="margin-bottom: 16px; padding: 0px;"><font color="#2e2e2e" face="Helvetica"><span style="font-family: Helvetica; font-size: 14px;">We observe, however, that there is a gap in technology that supports the process that analysts perform after inspecting the data, and before the identification of hypotheses. The task of the analyst here consistently involves the structuring of evidence&nbsp;to select plausible hypotheses. This is currently done manually, supported only by generic spreadsheets and text-processing tools. The challenge we seek to address in this research is to understand how automated reasoning can best complement human expertise in this evidential reasoning process.</span></font></p><p id="p0010" style="margin-bottom: 16px; padding: 0px; color: rgb(46, 46, 46); font-family: NexusSerif, Georgia, &quot;Times New Roman&quot;, Times, STIXGeneral, &quot;Cambria Math&quot;, &quot;Lucida Sans Unicode&quot;, &quot;Microsoft Sans Serif&quot;, &quot;Segoe UI Symbol&quot;, &quot;Arial Unicode MS&quot;, serif; font-size: 18px;"><span style="font-family: Helvetica; font-size: 14px;">Experienced analysts currently identify plausible hypotheses using a combination of manual approaches to assess available evidence, establish what information is credible, and understand what additional evidence may be required or what questions to ask to determine plausibility. This activity may be time critical to enable effective situational understanding, which poses significant challenges for individual analysts. The volume and variety of information that analysts must consider are significant, and, evidence may be unreliable or conflicting, with important information missing. Collaboration may be used to provide peer review, share the burden of analysis, and help in the validation of conclusions</span><span style="font-family: Helvetica; font-size: 14px;">. Such collaboration, however, requires analysts to work with a common model and a consistent worldview, which is hard to achieve in the real world.</span></p><p id="p0011" style="margin-bottom: 16px; padding: 0px; color: rgb(46, 46, 46); font-family: NexusSerif, Georgia, &quot;Times New Roman&quot;, Times, STIXGeneral, &quot;Cambria Math&quot;, &quot;Lucida Sans Unicode&quot;, &quot;Microsoft Sans Serif&quot;, &quot;Segoe UI Symbol&quot;, &quot;Arial Unicode MS&quot;, serif; font-size: 18px;"><span style="font-family: Helvetica; font-size: 14px;">When data is diverse and comes from different sources, analysts must reason about the reliability of the evidence leading to claims from information such as how, where, when, and by whom the evidence was gathered and analyzed. Cognitive biases may inadvertently be introduced in the process, preventing an analyst from drawing accurate conclusions. This process of interpreting evidence relies heavily on the expertise and training of analysts, and there is a distinct lack of methods to ease the high cognitive burden involved in forming hypotheses. Furthermore, there is a general lack of understanding of how the hypothesis formation process works, as it is not normally recorded, making it difficult for senior analysts to pass on their analytical skills to trainees. The analytical process is also resistant to automation due to the significant knowledge engineering effort required to process data and express reasoning patterns</span><span style="font-family: Helvetica; font-size: 14px;">.</span></p><p id="p0012" style="margin-bottom: 16px; padding: 0px; color: rgb(46, 46, 46); font-family: NexusSerif, Georgia, &quot;Times New Roman&quot;, Times, STIXGeneral, &quot;Cambria Math&quot;, &quot;Lucida Sans Unicode&quot;, &quot;Microsoft Sans Serif&quot;, &quot;Segoe UI Symbol&quot;, &quot;Arial Unicode MS&quot;, serif; font-size: 18px;"><span style="font-family: Helvetica; font-size: 14px;">In this paper, we have illustrated how novel AI methods, based on a combination of argumentation theory, crowdsourcing, and provenance reasoning can contribute to improved performance in intelligence analysis. While existing systems care mostly about information presentation and collection</span><span style="font-family: Helvetica; font-size: 14px;">&nbsp;we co-designed our software tool — CISpaces — with intelligence analysts to focus on the sensemaking activities around forming hypotheses from available evidence using patterns of defeasible inferences, or&nbsp;</span><em style="margin: 0px; padding: 0px;"><span style="font-family: Helvetica; font-size: 14px;">argumentation schemes</span></em><span style="font-family: Helvetica; font-size: 14px;">. Our formal evaluation of CISpaces using the Technology Acceptance Model (TAM)&nbsp;</span><span style="font-family: Helvetica; font-size: 14px;">&nbsp;provides evidence that intelligence analysts benefit from the support they receive from the tool in their sensemaking activities.</span></p><p id="p0013" style="margin-bottom: 16px; padding: 0px; color: rgb(46, 46, 46); font-family: NexusSerif, Georgia, &quot;Times New Roman&quot;, Times, STIXGeneral, &quot;Cambria Math&quot;, &quot;Lucida Sans Unicode&quot;, &quot;Microsoft Sans Serif&quot;, &quot;Segoe UI Symbol&quot;, &quot;Arial Unicode MS&quot;, serif; font-size: 18px;"><span style="font-family: Helvetica; font-size: 14px;"><b>The contributions of this paper are thus manyfold:</b></span></p><dl class="list" style="margin-right: 0px; margin-bottom: 24px; margin-left: 0px; padding: 0px;"><dt class="list-label" style="margin: 0px 2px 0px 0px; padding: 0px; float: left; clear: left;"><span style="font-family: Helvetica; font-size: 14px;">•</span></dt><dd class="list-description" style="margin: 0px 0px 0px 24px; padding: 0px;"><p id="p0014" style="margin-bottom: 16px; padding: 0px;"><span style="font-family: Helvetica; font-size: 14px;">In 3rd section</span><span style="font-family: Helvetica; font-size: 14px;">, we provide a blueprint for further co-design of artificial intelligence-driven tools, by showing how to govern the process for a successful outcome.</span></p></dd><dt class="list-label" style="margin: 0px 2px 0px 0px; padding: 0px; float: left; clear: left;"><span style="font-family: Helvetica; font-size: 14px;">•</span></dt><dd class="list-description" style="margin: 0px 0px 0px 24px; padding: 0px;"><p id="p0015" style="margin-bottom: 16px; padding: 0px;"><span style="font-family: Helvetica; font-size: 14px;">In&nbsp;</span><a name="bsec0010" href="https://www.sciencedirect.com/science/article/pii/S2667305322000886#sec0010" class="workspace-trigger" style="margin: 0px; padding: 0px; word-break: break-word;"><span style="font-family: Helvetica; font-size: 14px;"><font color="#000000">Section&nbsp;4</font></span></a><span style="font-family: Helvetica; font-size: 14px;">, we expand on our preliminary conference paper&nbsp;</span><span style="font-family: Helvetica; font-size: 14px;">to illustrate the delicate interconnection between the various artificial intelligence techniques utilized and extended to achieve the co-designed objectives. In particular:</span></p><dl class="list" style="margin-right: 0px; margin-bottom: 24px; margin-left: 0px; padding: 0px;"><dt class="list-label" style="margin: 0px 2px 0px 0px; padding: 0px; float: left; clear: left;"><span style="font-family: Helvetica; font-size: 14px;">•</span></dt><dd class="list-description" style="margin: 0px 0px 0px 24px; padding: 0px;"><p id="p0016" style="margin-bottom: 16px; padding: 0px;"><span style="font-family: Helvetica; font-size: 14px;">we advance the engineering of&nbsp;</span><em style="margin: 0px; padding: 0px;"><span style="font-family: Helvetica; font-size: 14px;">argumentation-based reasoning</span></em><span style="font-family: Helvetica; font-size: 14px;">&nbsp;</span><span style="font-family: Helvetica; font-size: 14px;">&nbsp;to identify plausible hypotheses as sets of acceptable arguments;</span></p></dd><dt class="list-label" style="margin: 0px 2px 0px 0px; padding: 0px; float: left; clear: left;"><span style="font-family: Helvetica; font-size: 14px;">•</span></dt><dd class="list-description" style="margin: 0px 0px 0px 24px; padding: 0px;"><p id="p0017" style="margin-bottom: 16px; padding: 0px;"><span style="font-family: Helvetica; font-size: 14px;">we show how to argue with, and about,&nbsp;</span><em style="margin: 0px; padding: 0px;"><span style="font-family: Helvetica; font-size: 14px;">crowd-sourced</span></em><span style="font-family: Helvetica; font-size: 14px;">&nbsp;information&nbsp;</span><span style="font-family: Helvetica; font-size: 14px;">&nbsp;pre-analyzed using&nbsp;</span><em style="margin: 0px; padding: 0px;"><span style="font-family: Helvetica; font-size: 14px;">Bayesian analysis</span></em><span style="font-family: Helvetica; font-size: 14px;">;</span></p></dd><dt class="list-label" style="margin: 0px 2px 0px 0px; padding: 0px; float: left; clear: left;"><span style="font-family: Helvetica; font-size: 14px;">•</span></dt><dd class="list-description" style="margin: 0px 0px 0px 24px; padding: 0px;"><p id="p0018" style="margin-bottom: 16px; padding: 0px;"><span style="font-family: Helvetica; font-size: 14px;">we embed&nbsp;</span><em style="margin: 0px; padding: 0px;"><span style="font-family: Helvetica; font-size: 14px;">provenance</span></em><span style="font-family: Helvetica; font-size: 14px;">&nbsp;analysis&nbsp;</span><span style="font-family: Helvetica; font-size: 14px;">in the argumentative process to establish the credibility of hypotheses.</span></p></dd></dl><p></p></dd><dt class="list-label" style="margin: 0px 2px 0px 0px; padding: 0px; float: left; clear: left;"><span style="font-family: Helvetica; font-size: 14px;">•</span></dt><dd class="list-description" style="margin: 0px 0px 0px 24px; padding: 0px;"><p id="p0019" style="margin-bottom: 16px; padding: 0px;"><span style="font-family: Helvetica; font-size: 14px;">In&nbsp;</span><a name="bsec0021" href="https://www.sciencedirect.com/science/article/pii/S2667305322000886#sec0021" class="workspace-trigger" style="margin: 0px; padding: 0px; word-break: break-word;"><span style="font-family: Helvetica; font-size: 14px;"><font color="#000000">Section&nbsp;5</font></span></a><span style="font-family: Helvetica; font-size: 14px;"><font color="#000000"><span style="font-family: Helvetica; font-size: 14px;">&nbsp;</span></font><span style="font-family: Helvetica; font-size: 14px;">we provide empirical evidence that intelligence analysts benefit from the unique mixture of formal argumentation theory, crowdsourcing support, and provenance recording provided in spaces, for the first time, using the Technology Acceptance Model (TAM)&nbsp;</span></span><span style="font-family: Helvetica; font-size: 14px;">&nbsp;in an argumentation-based system.</span></p></dd></dl><p></p><p id="p0020" style="margin-bottom: 16px; padding: 0px; color: rgb(46, 46, 46); font-family: NexusSerif, Georgia, &quot;Times New Roman&quot;, Times, STIXGeneral, &quot;Cambria Math&quot;, &quot;Lucida Sans Unicode&quot;, &quot;Microsoft Sans Serif&quot;, &quot;Segoe UI Symbol&quot;, &quot;Arial Unicode MS&quot;, serif; font-size: 18px;"><span style="font-family: Helvetica; font-size: 14px;">Our results suggest that the novel, principled AI methods implemented in CISpaces may advance performance in intelligence analysis. Despite having designed CISpaces as a basic research prototype (TRL 3), during their evaluation, analysts benchmarked the quality of its features against commercial systems they use every day: we compare them in Section 6.</span></p><p id="p0021" style="margin-bottom: 16px; padding: 0px; color: rgb(46, 46, 46); font-family: NexusSerif, Georgia, &quot;Times New Roman&quot;, Times, STIXGeneral, &quot;Cambria Math&quot;, &quot;Lucida Sans Unicode&quot;, &quot;Microsoft Sans Serif&quot;, &quot;Segoe UI Symbol&quot;, &quot;Arial Unicode MS&quot;, serif; font-size: 18px;"><span style="font-family: Helvetica; font-size: 14px;">We collected evidence suggesting that the AI methods implemented in CISpaces can have a behavioral effect on end users' intention to adopt spaces. The analysts’ evaluation highlights drawbacks in CISpaces that predominantly result from the interface between the tool and data sources and aspects of the user interface (rather than the underlying AI methods). We, therefore, conclude that for successful adoption by the intelligence analysis community, CISpaces will need data integration with existing organizational standards both for the input and the output of information. These and other engineering and usability aspects, while being essential for commercialization, are beyond the scope of this paper.</span></p><h2 id="sctt0004" class="u-h3 u-margin-l-top u-margin-xs-bottom" style="margin-right: 0px; margin-left: 0px; padding: 0px; color: rgb(80, 80, 80); font-family: NexusSerif, Georgia, &quot;Times New Roman&quot;, Times, STIXGeneral, &quot;Cambria Math&quot;, &quot;Lucida Sans Unicode&quot;, &quot;Microsoft Sans Serif&quot;, &quot;Segoe UI Symbol&quot;, &quot;Arial Unicode MS&quot;, serif; margin-top: 32px !important; margin-bottom: 8px !important; font-size: 1.2rem !important; line-height: 1.333 !important;"><span style="font-family: Helvetica; font-size: 14px;"><b>Challenges of Intelligence Analysis:</b></span></h2><p id="p0022" style="margin-bottom: 16px; padding: 0px; color: rgb(46, 46, 46); font-family: NexusSerif, Georgia, &quot;Times New Roman&quot;, Times, STIXGeneral, &quot;Cambria Math&quot;, &quot;Lucida Sans Unicode&quot;, &quot;Microsoft Sans Serif&quot;, &quot;Segoe UI Symbol&quot;, &quot;Arial Unicode MS&quot;, serif; font-size: 18px;"><span style="font-family: Helvetica; font-size: 14px;">Intelligence analysis is the application of individual and collective cognitive methods to evaluate, integrate, and interpret information about situations and events, aiming to provide a warning regarding potential threats or to identify opportunities</span><span style="font-family: Helvetica; font-size: 14px;">. Various types of intelligence can be distinguished based on the source. HUMINT (human intelligence), for example, is intelligence gathered from human sources. IMINT (imagery intelligence) is derived from image or video sources. OSINT (open source intelligence) is acquired from sources such as social media&nbsp;</span><span style="font-family: Helvetica; font-size: 14px;">and more recent types of intelligence include for example crowdsourced intelligence, made up of structured information acquired from or volunteered by the general public</span><span style="font-family: Helvetica; font-size: 14px;">. Analysts often specialize in a specific type of intelligence and may be focused on particular objectives (e.g., tracking activities of a criminal organization). In the military context, strategic analysts focus on studying the long-term objectives and intentions of foreign actors, while operational and tactical analysts are focused on supporting specific actions or providing timely responses to emerging situations.&nbsp;</span></p><p id="p0022" style="margin-bottom: 16px; padding: 0px; color: rgb(46, 46, 46); font-family: NexusSerif, Georgia, &quot;Times New Roman&quot;, Times, STIXGeneral, &quot;Cambria Math&quot;, &quot;Lucida Sans Unicode&quot;, &quot;Microsoft Sans Serif&quot;, &quot;Segoe UI Symbol&quot;, &quot;Arial Unicode MS&quot;, serif; font-size: 18px;"><span style="font-family: Helvetica; font-size: 14px;"><br></span><span style="font-family: Helvetica; font-size: 14px;">model of intelligence analysis is one of the most influential in training and practice. It consists of two high-level iterative loops:&nbsp;</span><em style="margin: 0px; padding: 0px;"><span style="font-family: Helvetica; font-size: 14px;">foraging</span></em><span style="font-family: Helvetica; font-size: 14px;">&nbsp;for&nbsp;</span><em style="margin: 0px; padding: 0px;"><span style="font-family: Helvetica; font-size: 14px;">information</span></em><span style="font-family: Helvetica; font-size: 14px;">&nbsp;which is collected, filtered, and collated into&nbsp;</span><em style="margin: 0px; padding: 0px;"><span style="font-family: Helvetica; font-size: 14px;">evidence files</span></em><span style="font-family: Helvetica; font-size: 14px;">; and&nbsp;</span><em style="margin: 0px; padding: 0px;"><span style="font-family: Helvetica; font-size: 14px;">sensemaking</span></em><span style="font-family: Helvetica; font-size: 14px;">, where the evidence files are interpreted through logical reasoning by drawing inferences and identifying&nbsp;</span><em style="margin: 0px; padding: 0px;"><span style="font-family: Helvetica; font-size: 14px;">hypotheses</span></em><span style="font-family: Helvetica; font-size: 14px;">, which are then brought together to form a coherent explanation of the situation. A sketch of this process is shown within the box at the top of below figure no.1</span><span style="font-family: Helvetica; font-size: 14px;">. The top row represents general features that characterize or influence the reasoning process during analysis. The other rows in this figure&nbsp;represent concepts related to different dimensions of intelligence analysis corresponding to a specific phase of analysis represented by the curly bracket in the column. More generally, this figure&nbsp;provides a reference for the components and challenges which inform the remainder of our discussion and are ordered according to the topics covered in this Section.</span></p><p id="p0022" style="margin-bottom: 16px; padding: 0px; color: rgb(46, 46, 46); font-family: NexusSerif, Georgia, &quot;Times New Roman&quot;, Times, STIXGeneral, &quot;Cambria Math&quot;, &quot;Lucida Sans Unicode&quot;, &quot;Microsoft Sans Serif&quot;, &quot;Segoe UI Symbol&quot;, &quot;Arial Unicode MS&quot;, serif; font-size: 18px;"><img src="/media/django-summernote/2022-11-15/b0e4535f-d926-4de2-aac6-9f0c9ad54fb7.png" style="width: 100%;"><span style="font-family: Helvetica; font-size: 14px;"><br></span></p>