The lgf3 Project
Table of Contents
The lgf3 project develops a software development framework for image-based modeling and rendering. Both computer graphics and computer vision apply these techniques to capture and reconstruct real scene information and present novel views at interactive rates to the viewers. For the first time this framework combines the requirements of vision and graphics in a single versatile platform.
A typical image-based modeling and rendering system first captures information of the scene by recording a set of image either with a photo or a video camera. This image sequence is the inital source for all image-based modeling techniques. In order to reconstruct novel views, the images have to be placed into a geometric context, i.e. the location and orientation of the camera when each image was recorded must be known. This process is called calibration and is an important research topic in the field of computer vision. If the geometric context is available then often geometric information on the scene itself is required. Therefore, either geometric models or depth maps for each camera need to be estimated. The depth reconstruction process is another challenging task in computer vision. If the calibrated images and the depth information is available then an initial image-based model of the scene is available. In computer graphics, the model is taken and for novel (virtual) camera positions the view in the scene is reconstructed out of the available data. Elaborate rendering technique use modern graphics hardware to immediately reconstruct a novel view at interactive rates. For efficient storage and representation the initial image sequence might be transformed into a model more suitable for rendering. The models are summarized as light field models since the set of images represents a set of the incoming light of a scene.
The lgf3 framework is build on top of this general description of an image-based system. It gives a general description of this processing pipeline by introducing modules, e.g. for calibration or rendering and defines a central storage, the scene database. There all information recorded from a scene and later on derived during calibration is stored and can be accessed by all modules, e.g. for rendering.
With the help of the lgf3 framework, a large set of state-of-the-art techniques were implemented and contributed to the framework. The rich environment of lgf3 allows to implement these methods very quickly and robust. This framework automatically takes care of resource management and handles large image sequences efficiently.
The lgf3 framework supports development for research prototypes and robust applications. The project supports the development of research prototypes with its rich feature set, its well-defined structure and greatly improves development speed and robustness. Robust applications can profit from these attributes and can directly use the novel techniques invented in research work.
The development of lgf3 is currently embedded in the SFB 603. There the subprojects C2 and B6 are the main contributors. The two institutes at the University of Erlangen-Nuremberg involved in the development are the Computer Graphics Group (LGDV) and the Chair for Pattern Recognition (LME).
The lead developers of the project are:
Other active contributing developers are:
- Florian Vogt (LME)
- Jochen Schmidt (LME)
- Marco Winter (LGDV)
Furthermore numerous students helped to improve the project with their contributions embedded in thesis work.
If you are a student here at the university and interested in topics handled by the lgf3 project then do not hesitate to contact one of the lead developers. We have many ideas for the project that still need to be implemented. You can either contribute with a thesis work or in a part-time job (HiWi).
Currently the source code of the framework is not available for the public. Nevertheless, we are quite open with the access policy of the source. In a joint cooperation, access to the source is usually not an issue.