The T_Vision Database
Overview
The current implementation is a very simple prototype with
a very limited functionality.
The database basically consists of pairs of index and data files
containing 128x128 pixel texture images (surface,clouds) and 16x16 point
elevation data. Geometry and Billboards are not stored at other places
in the current implementation, but this will change in the future.
Subdividing the Planet
Main part of the key is a string we call "Global Area Identifier"
(GAI). Every patch has bounds generated by binary subdivision
of the whole coordinate system:
There are three ways how this rectangle can be divided into
2 or 4 sectors, resulting in 8 possible sectors:
The GAI can be seen as a kind of telephone number for reaching
a particular sector of the planet. The number of digits corresponds
to the level of detail; the higher the number, the finer the resolution.
The reason for allowing horizontal and vertical subdivision and
not only the four quadrants is that if you map the rectangle
onto a sphere you can choose a subdivision strategie that
gives you more "quadratic" patches, especially around the poles.
Other advantages of this GAI coding compared to storing bounds:
Loading the Planet
The task for the renderer database client is very simple. The
renderer computes the GAIs according to the field of view
and makes a simple query.
In the current implementation the renderer gets a file name
and an offset and performs the reading itself to let the
renderer control at which timeslot this should happen. The
index files are loaded at startup time and reside in the main
memory.
Access to remote data is done via NFS on an ATM-network,
but many things this will
change in the future when a "real" distributed database software
will provide more powerful functionality.
The performance of the current implemention allows to load 20 to 30
64k texture patches per second and is limited by I/O bandwith only.
Generating the Planet
The database is generated from various data sources like satelite
imagery from LANDSAT and MOS for surface data and clouds from
Meteosat, GOES and GMS weather satelites.
The current generator can only use data in "cylinder projection", which
means that the bounds of the map have to be parallel to
the longitudes and latitudes. They can cover any part of the world
with any resolution.
Currently the source data consists of around 10 GB of image and DEM data,
covering the whole world in 4 km/pixel, USA and Europe in 1km,
Japan in 50m, some areas in USA and Germany in 50m, and parts of
Berlin and Tokyo down to 30cm.
The database is generated by point sampling all patches at the required
resolution for every level of detail that can be generated from the
source data with magnifying the source maps. This results is a
tree of different depth depending on the maximum resolution of
the source maps that cover the particular sector. The generator
decides for every pixel individually from which source map it is
taken.
The generator can be used in a difference mode where only those patches
are generated which will be different if you add a new source map.
This is necessary to avoid generating the whole database after
adding a new map of a small region like the center of Berlin,
which does only affect a few existing patches in the database.
A simple graphical user interface exists to browse through the database
and to view and position the source data:
Problems with the Planet
If there is one thing about the earth that I learned then this is the
fact that the earth is really, really big. Bigger than your main memory,
bigger than your filesystem, bigger than a 32Bit address space, and
bigger than the precision of a 32Bit float.
Unfortunetaly, with IRIX 5.3 a lot of work had to go into working around
system limitations, for example a "virtual" virtual memory manager that lets you
map dozen of gigabytes into your 4GB (in fact 2GB!) of virtual memory.
Even worse, with RE2 graphics subsystem you can not simply store your
coordinates in 32 Bit floats, but you have to scale and translate everything(!)
while flying around, your camera, your speed, your model, your clipping
planes.
The database generator uses 64Bit doubles everywhere, but the renderer can
not take full advantage of that at the moment.
The Future of the Planet
One of the first things I will do is to throw away all the code
I have written for this prototype and replace it with a real
object oriented full feature distributed high performance real time database.
How I get it ? I will buy one, if possible, or we will build something
around an existing database kernel, or use some sophisticated tools to let
the computer do most of the programming. :-)
Seriously, there is a real need for some new operating systems,
distributed object oriented real time operating systems.
Meanwhile we will work with the best stuff we can get.