We present an image-based reconstruction approach for mobile, multitouch enabled devices (like the iPhone). A novel scene description based on a multi-agent system is used to allow a real-time reconstruction workflow even on devices with relatively low processing speeds. Using the built in camera along with data from an accelerometer, compass and GPS module allows us to easily add new camera objects to our reconstruction world with an initial estimation for position and orientation. Our multi-agent scene description proved to be flexible enough to perform modelling tasks at interactive speeds beyond image-based reconstruction using multitouch gestures only.