Detecting Danger

For many people, conversations over coffee are for small talk—the weather, dinner plans, the humdrum details of daily life. For Drs. Nikolay Sirakov and Salvatore Attardo, oftentimes conversations over coffee are for exploring how computers can recognize weapons and assess threats to security in schools, banks and local favorite locations for lattes.

“It actually started out with Dr. Sirakov and I having coffee at the Cowhill. He and I would go there and always have coffee and talk about ideas,” Attardo said. “He is a mathematician, and I am a linguist. But we had this idea that we could do something that brought together our two fields.”

That idea was to have video monitoring that operates in conjunction with a computer program that can recognize whether a weapon is in a scene. If a weapon were detected, the system would automatically notify the police. But how does one give a computer the sophisticated, humanlike capacity to determine if what it has spotted is a deadly weapon?

A possible answer lies in the combination of mathematics and computational linguistics—a field that has given society closed captioning, digital spellcheckers and voice recognition. In the research that began four years ago as a conversational topic at Cowhill and led to the 2013 paper “From Shape to Threat: Exploiting the Convergence Between Visual and Conceptual Organization for Weapon Identification and Threat Assessment,” Dr. Attardo teamed with Dr. Christian Hempelmann and graduate student Carlo Di Ferrante to develop an ontology of weapons—that is, a repository of information that specifies each kind of weapon and its characteristics similar to the way the human brain stores data. Meanwhile, students working in tandem with Dr. Sirakov, joined by Dr. Abdullah Arslan, have begun to develop the program that moves an object from simple pixels to potential peril.

“The ontology tells us how many rounds are in a magazine, how long a barrel is, how far one can shoot with it,” said Hempelmann. “And it groups them by the way they look, by the contours. Based on this, you can identify the gun and ask, ‘How dangerous is this guy?'”

The computer code responds to that question through a process Attardo describes as a target shrinking until it zeroes in on an area. Then, like an individual using an Etch-a-Sketch, with the right instructions, the computer can outline a particular object. By pulling from a series of numbers that represent the shapes of each of the 73 weapons currently listed in the team's ontology, the computer could then identify which weapon has been captured on video and alert the appropriate authorities for the level of threat, which would be a revolutionary innovation for active shooter situations.

“Our current system shows that weapon identification and smart threat assessment can be done using visual analysis. We hope that in the future, our ontology and software system will be fully implemented, integrated with different surveillance platforms and deployed in places where security is very important,” said Arslan. “We hope that lives will be saved by early detection of weapons in such places.”

As the team refines the computer code, extends the ontology and applies for additional grants in their continued work to make this concept an actuality, national organizations in the fields of defense and security have predictably taken notice.

In 2011, Sirakov submitted a report to the Defense Advanced Research Projects Agency (DARPA), which funds research that assists in maintaining the “technological superiority of the U.S. military.” The submission led to an invitation to the Society of Photographic Instrumentation Engineers' (SPIE) 2013 Defense, Security and Sensing Symposium in Baltimore. There, in the Automatic Target Recognition conference, Sirakov and Arslan presented the “From Shape to Threat” paper and found it in the running for the best paper of the conference.

“I joked with Abdullah, ‘Maybe we will get this.' But when we did not hear from them for two or three weeks after the symposium, we decided this is not us,” Sirakov said. “We were very happy to hear at the beginning of July that we won the best paper award.”

The award, sponsored by the Lockheed Martin Corporation, came with a $2,000 prize and a certain sense of accomplishment. Researchers from universities such as Carnegie Mellon and Johns Hopkins were among their competitors.

Carlo Di Ferrante can feel especially proud. He built the computational structure and folder hierarchy for the weapons ontology and is the only student to appear as an author on the award-winning “From Shape to Threat” publication.

“I always felt like a part of the research team, and they also made me feel like my opinion was as valuable as theirs,” Di Ferrante said of his co-authors. “The work with professors has been the thing that I think has most contributed to preparing me as a professional. I learned from them by observing how they approached the work and following their advice. The confidence in myself that they inspired allowed me to trust my instincts and work hard on them.”

Newfound confidence in tow, Di Ferrante says that at A&M-Commerce he has also found the experience and knowledge he had been seeking. After completing his first round of higher education in his native country of Italy, Di Ferrante discovered few work opportunities at home. However, coming to A&M-Commerce to pursue his master's degree in computer science paved the way for his current opportunity: a six-month engineering internship with Google London. He is working on “text-to-speech” systems, and he credits the university and his professor colleagues for giving him the skills to succeed as a Googler.

“Abdullah Arslan taught me about data structure and algorithm design. He is one of the people I respect the most because he inspired me. I saw in him someone that really loves what he teaches,” Di Ferrante said. “I also enjoyed the work breakfasts with Christian Hempelmann and the humor of Nikolay Sirakov. All these things together with the wise guidance of Dr. Attardo and the warmth of all the people I met made me love both working and studying at A&M-Commerce. A&M-Commerce gave to me, during one year, more than I could have ever imagined to get from studying at a university.”

It is no surprise that Di Ferrante says that he looks forward to returning to the university to “continue enjoying the greatness of this challenging and warm place.” After all, he recognizes that the team is just at the genesis of cutting edge research with added implications for airport security scanning and image processing in the healthcare field.

“Research shows that new trends in computer science are about how to make computers process and resolve problems like the human brain does. It requires the science of computers to be contaminated with the science of humanity,” Di Ferrante said. “I think A&M-Commerce's progress in this direction is beyond most of the universities in the world. We wrote a paper on weapon recognition, but the techniques we used could be applied in a number of ways that are limited only by our imagination.”

Listen to our conversation with Drs. Attardo, Sirakov, Hempelmann and Arslan here.