Research Algorithm Detects Analyst Judgments
University researchers are creating a tool that improves the way security intelligence is analysed. It will help those organisations tackling security threats in the UK and beyond.
The team of academics at the University of Portsmouth are working on software that will help security analysts identify the critical information from the huge volume of data they receive and enable them to better detect potential security threats. This software reduces the reliance on individual judgement and creates a more systemised approach. The project is one of eight to be announced today by the Centre for Research and Evidence on Security Threats (CREST).
Professor Ashraf Labib is leading the research by the Portsmouth team. He said the innovative new software will deliver an analytic approach to address errors in judgement that can hinder the decision making process.
He said: “Intelligence analysts need to process large volumes of data quickly, extracting crucial information to detect potential security threats. This could be identifying certain key words used to denote people, places or objects that need to be highlighted to security services for further investigation.
“This kind of analysis relies on consistent judgments, but research and historical evidence shows us that analysts’ judgments are often inconsistent due to the sheer mass of data, the variation in types and nature of intelligence information and the time pressures in which they are operating. This means decisions can be made that deviate significantly from those of their colleagues, from their own prior decisions and from the guidelines and rules they are trained to follow.
“This disparity is mainly due to two types of errors. ‘The ‘noise’ or sheer volume of data, and bias, which can occur over time with individuals and also groups. Both can complicate the intelligence analysis process and can result in key pieces of data being misclassified or overlooked with potential security threat implications. We are developing an innovative analytic approach to address these errors and enable analysts to achieve better judgments and to test it in a group decision making context.”
The software uses what the researchers call the Dominance-based Rough Set Approach (DRSA) algorithm that captures the patterns and behavior of the analyst. This is used to evaluate the consistency of analysts’ judgments at individual and group levels, as well as identifying key factors or biases which influence an analyst’s decision-making. The findings will be used to inform analyst training and as a decision aide within the tool to ensure more robust judgments are made.
The University of Portsmouth researchers are experts in root-cause decision analysis and operational research techniques, information elicitation and criminal intelligence. They are
Professor Ashraf Labib and Dr Salem Chakhar from the University’s Business School, Professor Lorraine Hope from the Department of Psychology and Dr Adrian James from the University’s Institute of Criminal Justice Studies. They are working with consulting company, Polaris, who specialise in operational research, experimental design and software development.
The research has already developed an approach for individuals and the team are now working on decision analysis for groups, which should be available later this year.
The project is one of several to be selected (subject to contract) through a rigorous and independent review process, from more than 80 applications to CREST’s recent commissioning call. CREST offered £900,000 to fund innovative economic, behavioural, and social science research relevant to understanding and mitigating contemporary security threats.
Speaking about the announcement, the Director of CREST, Professor Paul Taylor, said: “We continue to be impressed by the outstanding quality of responses to our call. The successful applicants promise to deliver theoretical innovation that will make a real difference to the work of the security and intelligence agencies. I am looking forward to working with them.”