WP1: Data distribution and end user perspective
Understanding the motives and desires of end user groups and rethinking the concept of interaction with digital libraries is essential to design new search tools. WP1 collects user requirements to guide the design of the search technologies and interaction paradigms, which include tools for collaborative annotation, visualisation and tracking of information trails. Feedback from validation sessions will be provided to the technical work packages in various stages as input for the design iterations. Finally, WP1 addresses the building of appropriate test collections for the use cases.
WP2: Multimodal analysis of people
People play a central role in most audiovisual material. Recognizing people based on their visual appearance and/or their voice is the central issue studied in WP2. This work package aims at developing novel tools to efficiently detect categories of people (e.g. people of a certain type, accomplishing certain actions, in a given type of interaction, …) and to identify specific individuals. These annotations will be produced using a mix of unsupervised and weakly supervised learning techniques, using audio-visual content as primary data, and external textual sources as side information. These annotations will help linking, searching and navigating the archives, either used independently or in combination with annotations such as places and events.
WP3: Multimodal analysis of categories and places
Apart from people, users are also interested in the location (e.g. the Eiffel Tower) or objects (e.g. an airplane) present in the archives. The goal of this work package is to produce annotations for specific places/buildings, for categories of buildings and objects in large scale video collections. Here we distinguish between specific objects and locations as well as category-level objects and locations. Scalability towards a large number of objects or places is an important focus point, as are similarity measures used for query-by-example.
WP4: Multimodal analysis of events
A fourth entity of interest to uses is the event. We differentiate between recognition of event categories and recognition of specific events, which often requires exploiting the spatio-temporal nature of the audiovisual material. WP4 focuses on developing approaches that characterize the temporally dynamic nature of events in videos, making use of multi-modal information (i.e. integrating visual and audio features).
WP5: Spoken and written entities
Some information in audiovisual sources is text-oriented and language dependent. The goal of this work package is to extract information from spoken or written content, and propagate this information in a unified form to the other technical work packages for cross-modal entity extraction. This includes machine translation and OCR methods. We will explore approaches for integrating iterative user feedback into the ASR/Video-OCR workflows, and investigate new ways to time-align external data with related segments of a multimedia document.
WP6: Link management and structured search
Work packages 2 – 5 generate a lot of metadata (in the form of annotations) as well as links (in the form of detected similarity between audiovisual fragments). WP6 investigates ways for the user to deal with all these metadata in a structured way and designs, prototypes and evaluates a service-oriented search system for linked audiovisual data.
WP7: Experiencing digital libraries
The AXES project does not only want to allow for archival search based on keywords, but develop various innovative ways of experiencing digital libraries, such as browsing or exploring the rich archive contents. The objectives of WP7 are to determine how to design novel navigation based on the requirements investigated in WP1, to functionally integrate the various tools and algorithms into proof-of-concept systems suitable for technical validation and to evaluate and validate these prototype systems within the context of international benchmarking activities.
WP8 makes sure that the overall and detailed architecture of the different versions of the AXES system, developed by the different partners, can work together and are integrated in a bullet-proof system. Building on the open source WebLab platform, all components should be made accessible as web services and work together fluently.
WP9: Dissemination, exploitation and training
This work package is responsible for the project web site and for making the project known in both scientific, archival and research sectors. A second crucial goal us training the targeted user groups. Various types of workshops and/or seminars for the different user groups are considered, as well as the organization of summer schools. The work package finally establishes a roadmap to exploit, use and promote the relevant final results.
WP10: Scientific coordination
WP10 scientifically animates the project and makes sure that synergies between RTD activities and tasks are developed. It aims to ensure the highest scientific quality of the research activities and to maximise the chances of delivering the highest possible impact.