Article: "Multimedia Classification" From BBC Research

From a BBC R&D Blog Post:

As John Zubrzycki mentioned yesterday, this project is running as part of BBC R&D’s Archive Research Section, is developing new ways in which to open up the BBC’s archive to the public. The aim of the project is to allow people to search and browse the archive to find content that they want to watch or listen to, but didn’t know existed.
[Clip]
In R&D, we are currently developing systems that can watch and listen to a programme in a similar way to people.  We are developing systems that can recognise, and more importantly understand, what is in a programme (e.g. people, places, objects such as cars or Daleks), what these are doing (e.g. are character’s talking or shouting to each other? Is someone running? What are the characters saying to each other?) and what is the mood, or feeling of the programme. The mood element helps people find the programmes they want in order to be entertained – to match the mood of the programme to their current mood or desired mood.
In order to do these we are focussing on three main areas. The first is what we’ve termed characteristics extraction. This is where we analyse the audio and video signals and try to identify key properties of it – such as cuts, motion, luminance, faces, any key audio frequencies or audio frequency combinations or especially strong or weak parts of the signal – using signal processing techniques such as the power spectral density or taking a Fast Fourier Transform of a section of the signal. This then gives us a set of numbers, or vectors, which represent the audio and video signals based on their key properties.
The next stage is what we’ve termed feature extraction. Using the extracted characteristics, we aim to use them to identify key features, or objects in the programme. We do this using machine intelligence techniques which map the extracted characteristics to a library of characteristics taken from known features.  These systems then make a decision as to which of the known features the extracted characteristics most closely match.

Read the Complete Blog Post
See Also: “BBC Archive and Digital Public Space: an Overview of Our Work on the Archive from Preservation to Multimedia Classifications”

Filed under: Libraries, Preservation, Resources