The T_Vision Database


Overview

The current implementation is a very simple prototype with a very limited functionality. The database basically consists of pairs of index and data files containing 128x128 pixel texture images (surface,clouds) and 16x16 point elevation data. Geometry and Billboards are not stored at other places in the current implementation, but this will change in the future.

Subdividing the Planet

Main part of the key is a string we call "Global Area Identifier" (GAI). Every patch has bounds generated by binary subdivision of the whole coordinate system:

  • longitude: (-180, 180) west <-> east
  • latitude : (90, -90) north <-> south
  • There are three ways how this rectangle can be divided into 2 or 4 sectors, resulting in 8 possible sectors:

    Image showing GAI codes

    The GAI can be seen as a kind of telephone number for reaching a particular sector of the planet. The number of digits corresponds to the level of detail; the higher the number, the finer the resolution. The reason for allowing horizontal and vertical subdivision and not only the four quadrants is that if you map the rectangle onto a sphere you can choose a subdivision strategie that gives you more "quadratic" patches, especially around the poles. Other advantages of this GAI coding compared to storing bounds:

  • You dont have to worry about floating point representation inacuracies
  • Fast and easy searching can be performed
  • Unlimited number of subdivisions can be coded (unlimited precision)
  • Codes can be used to support a very large scale data distribution on a global ATM network
  • More dimensions can be added in the future
  • Loading the Planet

    The task for the renderer database client is very simple. The renderer computes the GAIs according to the field of view and makes a simple query. In the current implementation the renderer gets a file name and an offset and performs the reading itself to let the renderer control at which timeslot this should happen. The index files are loaded at startup time and reside in the main memory. Access to remote data is done via NFS on an ATM-network, but many things this will change in the future when a "real" distributed database software will provide more powerful functionality. The performance of the current implemention allows to load 20 to 30 64k texture patches per second and is limited by I/O bandwith only.

    Generating the Planet

    The database is generated from various data sources like satelite imagery from LANDSAT and MOS for surface data and clouds from Meteosat, GOES and GMS weather satelites. The current generator can only use data in "cylinder projection", which means that the bounds of the map have to be parallel to the longitudes and latitudes. They can cover any part of the world with any resolution. Currently the source data consists of around 10 GB of image and DEM data, covering the whole world in 4 km/pixel, USA and Europe in 1km, Japan in 50m, some areas in USA and Germany in 50m, and parts of Berlin and Tokyo down to 30cm.

    The database is generated by point sampling all patches at the required resolution for every level of detail that can be generated from the source data with magnifying the source maps. This results is a tree of different depth depending on the maximum resolution of the source maps that cover the particular sector. The generator decides for every pixel individually from which source map it is taken.

    The generator can be used in a difference mode where only those patches are generated which will be different if you add a new source map. This is necessary to avoid generating the whole database after adding a new map of a small region like the center of Berlin, which does only affect a few existing patches in the database.
    A simple graphical user interface exists to browse through the database and to view and position the source data:

    Image showing user interface

    Problems with the Planet

    If there is one thing about the earth that I learned then this is the fact that the earth is really, really big. Bigger than your main memory, bigger than your filesystem, bigger than a 32Bit address space, and bigger than the precision of a 32Bit float.
    Unfortunetaly, with IRIX 5.3 a lot of work had to go into working around system limitations, for example a "virtual" virtual memory manager that lets you map dozen of gigabytes into your 4GB (in fact 2GB!) of virtual memory. Even worse, with RE2 graphics subsystem you can not simply store your coordinates in 32 Bit floats, but you have to scale and translate everything(!) while flying around, your camera, your speed, your model, your clipping planes. The database generator uses 64Bit doubles everywhere, but the renderer can not take full advantage of that at the moment.

    The Future of the Planet

    One of the first things I will do is to throw away all the code I have written for this prototype and replace it with a real object oriented full feature distributed high performance real time database. How I get it ? I will buy one, if possible, or we will build something around an existing database kernel, or use some sophisticated tools to let the computer do most of the programming. :-)

    Seriously, there is a real need for some new operating systems, distributed object oriented real time operating systems.

    Meanwhile we will work with the best stuff we can get.