CUbRIK: The human-enhanced multimedia search

This research project promotes the realization of "white-box" systems for processing multimedia content and query processing in order to favor innovation in the next generation of multimedia research applications.

The cube metaphor is used to highlight the fact that innovation progress travels along three main dimensions: technology platform, human platform and business platform.

The technology dimension focuses on the realization of an "open" search platform in which it is possible to integrate and orchestrate open source and proprietary software using "pipelines" for modeling and implementation of the processes. This allows professionals from the sector, researchers and final users to join forces in order to encourage progress of currently existing state-of-the-art technologies.

The second dimension concerns the "Human in the Loop" vision of CUbRIK: the strategy focuses on crowd sourcing within the analysis and processing cycle and multimedia content processing in all its phases, from extracting research characteristics to validating the results obtained.

Lastly, the vision of CUbRIK in the business platform is not only of supporting innovators such as SMEs in implementing new algorithms, components and solutions for multimedia research, but also of favoring the emergence of a community where reciprocal collaboration helps new business models to emerge for distribution of multimedia research solutions in vertical markets.

CUbRIK “puts humans in the Loop” by introducing an innovative networked human computation framework.

One of CUbRIK's main goals is to improve multimedia research by adopting a framework that foresees the explicit crowd involvement which also involves areas such as user behavior analysis, crowd sourcing, social networks and trust analysis, and gaming with a purpose.

In CUbRIK, “Put humans in the Loop” means:

  • Defining the modeling and implementation of tasks that include explicit human tasks, and emphasis on the integration of said humans with machine tasks.
  • Applying the paradigm of Games with a Purpose (GWAPs), exploiting the classic approach to social network analysis and
  • Adopting crowdsourcing techniques to address and solve typical multimedia research problems (content tagging, relevance feedback gathering and quality evaluation).

This entails progress in the state of the art of architectures and languages for specification and execution of processes. The final result is therefore an "open", integrated platform which, thanks to harmonization of the human component and the purely technological component:

  • processes and indexes (making content searchable) multimedia content by extracting low and high level features.
  • improves the quality of searches thanks to feedback provided by people both during content noting and during result validation.

In addition, the involvement of volunteers and/or enthusiasts provides a business opportunity based on crowdsourcing models for SMEs involved in the project.

CUbRIK “open the search box” for the realization of an integrated multimedia search platform

One of the project's goals is to create an open platform for the integration of domain applications (search multimedia application). In fact, it is planned that the platform will be released using the open source license model and that both API and SDK will be available.

This is possible owing to the in-depth design phase of processes where new components and already existing third-party services can be integrated. In this way, results obtained in previous multimedia search research projects can also be used again.

The SMILA framework (projects.eclipse.org/projects/rt.smila) is an important resource for the definition and execution of processes and offers developer communities the possibility of integrating their own components and also of developing new multimedia search applications specifically for CUbRIK architecture.

www.cubrikproject.eu/