SINTESYS therefore intends to create an integrated software system offering services based on innovative techniques and models such as text mining, audio analysis, video and image recognition, human computer interaction, cognitive psychology, information fusion and artificial intelligence.
The combined use of such techniques allows situations of potential danger to emerge for public safety that the separate assessment of each source would not show up, such as, for example, the reporting of:
- threatening phrases and text content on forums, blogs and chats
- dangerous subjects in multimedia content published on social media
- particular impulsive sounds (shouts, shots, broken glass...) in audio files
- dangerous items (weapons, abandoned suitcases...) in images and videos
- Relations between suspicious individuals.
The system is planned to be flexible and to adapt to the various needs of the security experts, who can navigate through the data using intelligent graphic interfaces, including 3D ones, that can adapt to the various types of information sources, the actual investigative needs and the specific investigation paths followed by analysts using a specific recognition of their habits and based on a psychological study of interaction models.
Platform for the extraction and aggregated analysis of information from open web sources
One of the main results of the SINTESYS project is represented by the integrated platform for the extraction and aggregated analysis of information from open sources on the web. The integration of sources makes access to the resources transparent, as they are pre-filtered on the basis of a method that uses statistical text mining techniques to distinguish the content to the sent to the low level extractors. Data mining techniques recognize specific patterns linked to the type of content examined. Recognizing a pattern coincides with the discovery of an event that is relevant for security matters.
The SINTESYS platform, based on open source technologies, offers specific, configurable services that can be applied to the analysis and prevention of crime and terrorist attacks, providing security experts with virtual environments where they can retrieve and navigate through information, suggest information linked to the context of investigations and aid the formulation of investigative hypotheses. The system created can be adapted to all realms of public security, from prevention of terrorist attacks to the prevention of common crime and micro-crime, such as bullying. It is also possible to use methods and services realized in the forensic field.
Defining innovative data mining techniques for extracting information from multimodal sources
The platform created has unique characteristics relating to the various integrated technologies for achieving the goals set and excellence of analysis techniques. In addition to using state-of-the-art techniques, innovative services for analyzing text and multimedia content are currently being designed and created. The text analysis is carried out using different approaches (statistical, linguistic, deep semantics) depending on the investigative phase that is being conducted. Sub-symbolic text mining techniques also recognize typing errors found in fragmented texts, and find the "nearest" word.
The linguistic approach allows the public security domain to be shaped, distinguishing the various contexts of reference to which events can be traced. A new method for social contextualization of text documents has also be created, which works on the modeling of social groups. Based on categorization techniques, it is possible to see the result of the context analysis using an innovative molecular metaphor.
The deep semantics analysis provides for understanding of the complex sense of fragmented texts, even if terms are missing or words are used with a shifted meaning, thanks to the use of pragmatic disambiguation, that hypothesizes the most reasonable attachment points starting from linguistic knowledge and knowledge in the field. A recognition service and weapon classification has been created in the image processing field that is based on three global description factors: color, texture and shape.