On the Optimal Placement of Multiple Visual Sensors

Visual sensor arrays are used in many novel multimedia applications such as video surveillance, sensing rooms, assisted living or immersive conference rooms. Often several different types of cameras are available. They differ in their ranges of view, intrinsic parameters, image sensor resolutions, optics, and costs.

Most of the above mentioned applications require the layout of video sensors to assure a minimum level of image quality or image resolution. Thus, an important issue in designing visual sensor arrays is the appropriate placement of the cameras such that they achieve one or mulitple predefined goals. As video sensor arrays are getting larger, efficient camera placement strategies need to be developed.
result configuration by greedy approach
For more information on optimal camera placement please contact Eva Hörster

Audio Brush: What You See is What You Hear

Hearing, analyzing and evaluating sounds is possible for everyone. The reference-sensor for audio, the human ear, is of amazing capabilities and high quality. In contrast editing and synthesizing audio is an indirect and non-intuitive task needing great expertise.

To overcome these limitations we are creating Audio Brush, a smart visual audio editing tool. Audio Brush allows to edit the spectrogram of a sound in the visual domain similar to editing bitmaps. At the core is a very flexible audio spectrogram based on the Gabor analysis and synthesis. It gives maximum accuracy of the representation, is fully invertible, and enables manipulating the signal at any chosen time-frequency resolution.
Audio Brush screen shot by greedy approach
For more information on Audio Brush please contact Gregor van den Boogaart


Real-Time Event Detection and Control in Live Video Streams

It is nowadays very common that public places such as pubs, restaurants, and fitness club have large TV screens to entertain their customers -- especially during national or international sports championship events. For the venue owner it would be desirable if they could control which commercials are shown to their audience. In other words they may have the desire to replace untargeted commercials by target commericals of their choice.

In this joint project with Half Minute Media Ltd. we research algorithms for robost real-time commercial detection and control (such as replacement) in live streams. We are especially developing fast and extremely reliable algorithms for
  • Mining video channels automatically in order to extract all commercials and
  • Detecting known commericials in live streams using highly compact, but discriminate clip descriptors

References:

  • Rainer Lienhart, Christoph Kuhmünch and Wolfgang Effelsberg. On the Detection and Recognition of Television Commercials, Proc. IEEE Conf. on Multimedia Computing and Systems, Ottawa, Canada, pp. 509 - 516, June 1997. also Technical Report TR-96-016, University of Mannheim, Dezember 1996.

Bayesian Face Recognition on Infrared Image Data

The availability of high-performance and low-cost desktop computing systems and digital camera equipment has given rise to a public interest towards applications that include the visual identification of human individuals. Examples for such applications are surveillance, biometrical identification or computer-human interaction.

To that effect, research in biometrical technologies follows naturally. Above other methods, images of human faces offer a non-intrusive and easy-to-use means of identification. Although the recognition of faces is a problem that is effortlessly solved by human beings during their daily routine, it poses a challenge for researchers and scientists. Boundary conditions like illumination and occlusion, as well as pose and expression of an individual lead to intrapersonal variations that often exceed those between images of different persons under similar conditions.

In association with Falcontrol Security GmbH we are researching reliable face recognition algorithms by using Bayesian methods on infrared image data.

For more information on Bayesian Face Recognition on infrared camera images please contact Jochen Lux.

Parallel Algorithms for Fast Machine Learning

Machine learning applications are emerging as the most promising approaches to many current problems in computer science. However, machine learning algorithms typically require the processing of large data sets and thus, long training times (sometimes on the order of several days or even weeks). Especially for newly developed approaches, high performance implementations are not available; most implementations are designed with a serial model of execution in mind.
At the same time, shared memory multiprocessing architectures are becoming more and more commonplace. The computational power of these machines could be used to solve machine learning problems much faster and in parallel, if we only knew how to properly exploit it.

The goal of our research is to reduce training times speed up machine learning algorithms by developing design patterns and strategies for parallelizing them on multiprocessor computers.
For more information on parallel algorithms for fast machine learning, please contact Simon Hoffmann.

Image Retrieval on Large Scale Image Databases

Nowadays there exist online image repositories containing hundreds of millions of images of all kinds of quality, size and content.

These image repositories grow day by day making techniques for navigating, indexing, and searching prudent. Currently indexing is mainly based on manually entered tags and/or individual and group usage patterns. Manually entered tags, however, are very subjective and not necessarily referring to the shown image content. This subjectivity and ambiguity of tags makes image retrieval based on manually entered tags difficult.

In this project we employ the image content as the source of information to retrieve images and study the representation of images by topic models. The developed approaches are evaluated on real world, large scale image databases.
Main
result retrieval
References:
  • Eva Hörster, Rainer Lienhart and Malcolm Slaney. Image Retrieval on Large-Scale Image Databases. ACM International Conference on Image and Video Retrieval (CIVR) 2007 pp. 17-24, Amsterdam, Netherlands, July 2007. also Technical Report Apr. 2007 [PDF]
  • Eva Hörster and Rainer Lienhart. Fusing Local Image Descriptors for Large-Scale Image Retrieval. International Workshop on Semantic Learning Applications in Multimedia (SLAM), Minneapolis, USA, June 2007. also as Technical Report [PDF]
  • Rainer Lienhart and Malcolm Slaney. PLSA on Large Scale Image Databases. IEEE International Conference on Acoustics, Speech and Signal Processing 2007 (ICASSP 2007), Hawaii, USA, April 2007. also Technical Report Dec. 2006 [PDF]