This section describes the major components of Chromium and how they fit together.
The configuration mothership
The mothership handles all configuration requests for all components of the system. The mothership is written in python and can be found in the mothership/server/mothership.py file.
Several example configuration scripts
The mothership/configs directory contains a number of sample configuration scripts. You'll probably want to use one of these scripts as the basis for making your own.
A "render" SPU
This SPU hands all OpenGL calls to the system's OpenGL implementation. Nearly every Chromium configuration will use the render SPU (or readback SPU).
A "pack" SPU
This SPU packs a stream of OpenGL commands into a buffer which is sent over the network to a network server node. Although the calls are encoded verbatim (i.e., with no analysis), some state tracking is still necessary in order to properly unpack pixel data for texture maps.
An OpenGL replacement / faker library
This library "fakes" unmodified applications into using Chromium. The main thing that the faker library does is to load the client-side SPUs and dispatch OpenGL calls to the first SPU in the chain. Although SPUs implement the OpenGL interface, they typically only export a very small number of symbols. The SPU loader will create a collection of function pointers for the SPUs functions. The replacement library is necessary to actually resemble the system's OpenGL interface sufficiently to broker requests from applications to SPUs. Note that an application could easily load a SPU from disk itself and use it without the OpenGL library, which would be one possible way to render to a local screen and over a network simultaneously.
An application launcher
The launcher, called crappfaker, launches
applications in a special way that causes them to load the OpenGL
replacement library rather than the system's library.
The application launcher is usually refered to as an application
node. Application nodes and server/network nodes host
SPU chains.
A network server
This server, called crserver, is
responsible for dispatching incoming blocks of encoded OpenGL
commands (such as from the pack SPU) to a
chain of SPUs. In fact, as we will see, the network server is really
a stream serializer. It is the job of the server to accept
streams from multiple clients, and context switch between them as
allowed by the parallel API
extensions. In addition, a server can manage multiple display
tiles, and is responsible for re-decoding a block of work multiple
times, with appropriate modifications to the viewing matrices for
multiple tiles.
The crserver is otherwise known as a
network or server node.
In addition to the SPUs shown so far, several more are provided. Some of the SPUs are intended to be used directly in a configuration graph, and others are more useful as starting points for new SPUs, or SPUs to inherit from.
The "dist_texture" SPU
This SPU allows you to have texture images saved to disk on nodes
running crserver processes. The textures can later be loaded by
specifying only the filename, thus avoiding the network traffic cost
of sending the entire image. This is accomplished by intercepting
glTexImage2D calls and checking the
value of the type parameter. If it is
GL_TRUE, then the texture data (in the
pixels parameter) is assumed to be a
NULL-terminated filename string followed by texture data
of the given width and height (which need not be powers of two). The
image data must be interleaved RGB values in
GL_UNSIGNED_BYTE format and will be written to disk as a
raw PPM image. Once the file is written, the image will be padded if
necessary (so that the width and height are powers of two) and sent
downstream as a valid glTexImage2D
call. If the type parameter is GL_FALSE,
then the pixels parameter is assumed to be the filename
of a raw PPM image file which will be read, padded if necessary, and
sent downstream as a valid glTexImage2D
call. This extension to glTexImage2D
was developed for the
NCSA Pixel Blaster (a movie player for tiled displays), which is
a good example of its use. Unlike the original NCSA modifications to
WireGL, the distributed texture SPU does not ban absolute filenames
because Chromium's crserver processes are meant to be started by the
same user who is running the application. The original WireGL
modifications were by Paul
Rajlich, but don't bug him about this version; all the bugs are
due to David Thompson.
The "error" SPU
This SPU will issue a fatal error on any OpenGL call. The error message will contain the name of the function called. This SPU is useful for handling "impossible" function calls. If the reader is already familiar with SPU inheritance, all SPUs are implicitly a subSPU of the "error" SPU.
The "feedback" SPU
This SPU will catch all related Feedback and Selection OpenGL calls. This allows the abstraction of this layer away from other SPUs. If you need support for Feedback and Selection, then insert this SPU into the chain.
The "motion blur" SPU
This SPU uses the OpenGL accumulation buffer to simulate motion blur in a dynamic (animated) scene.
The "nop" SPU
This SPU will silently discard all OpenGL calls. This SPU is most useful for understanding the performance of applications and cluster configurations. By replacing the render SPU with the nop SPU, time spent actually executing OpenGL commands can be eliminated. Similarly, the nop SPU can be used on the client to make all OpenGL calls "free". This helps establish a baseline for performance.
The "passthrough" SPU
This SPU will pass all calls to the next SPU in a chain. Note that it can therefore never appear as the last SPU in a chain (this is a checked, fatal error). This SPU is not useful by itself, but it is a very useful SPU to inherit from in order to implement a subset of the OpenGL API. An example of such a SPU is shown in the "Writing a new SPU" section.
The "perf" SPU
This SPU will monitor OpenGL traffic and display 'interesting' results on what needs to be monitored. See the Perf SPU page for more details on its configuration.
The "print" SPU
This SPU was mentioned in the "Configuration scripts" section. It produces
a human-readable dump of all OpenGL calls, either to standard output
(default), or to a file. This is an invaluable tool for debugging,
although it does have a fairly serious impact on performance. Some
attempt is also made to turn GLenum types back into
human-readable strings (thanks to Kekoa
Proudfoot for this code).
The "readback" SPU
This SPU is derived from the render SPU. As
such it accepts a stream of OpenGL commands and renders them into its
window. When SwapBuffers is called,
however, the color buffer image is sent to the child SPU via
glDrawPixels. Optionally, the depth (Z)
buffer image is also sent to the child SPU. By sending the color and
depth images from several readback SPUs to
one render SPU, one can implement a
sort-last (Z-compositing) architecture.
The "saveframe" SPU
This SPU saves the current frame to a JPEG or PPM file. Just like the
print SPU, this SPU can be very useful for
debugging streams. It uses its downstream child to do a ReadPixels, so this SPU can never be the last one
in the chain. (It also might require a render SPU in the same node, though this hasn't been
tested.)
The "template" SPU
This SPU actually implements no functions (it is therefore exactly equivalent to the SPU, due to the implicit inheritance). However, a script is provided to rename all files and update the source code to create a new SPU of a given name. Therefore, when creating a new SPU, a copy should be made of the spu/template directory, and this script should be run. This is the approach used in the Writing a new SPU section.
The "tilesort" SPU
This SPU uses the techniques described in our SuperComputing 2000 paper to drive a tiled display efficiently from a single serial stream of OpenGL commands. One-third of the logic of WireGL can be found in this library (the other thirds being in the server, and the state tracker, listed below).
The "wet" SPU
This SPU warps and distorts the rendered image to simulate a watery surface.
Chromium also builds several shared libraries which can be used by any component.
An "application stub" library for Chromium-aware applications
This library can be used by Chromium-aware applications that wish,
for instance, to use the Parallel API. It also provides routines for
creating an OpenGL context without ever opening a window, a useful
thing for a distributed application to do (since it doesn't
necessarily make sense for dozens of blank faked-out client windows
to spring up all over the cluster, especially when all of your
cluster nodes aren't running X-Windows). This approach is preferred
over defining the Chromium-specific functions as OpenGL extensions
and using something like wglGetProcAddress to extract the function pointers,
since the application stub library is portable and applications don't
need to concern themselves with the type of system they are running
on.
A client library for the configuration mothership
Applications that need to talk to the configuration mothership will
use this library to make queries. It hides the fact that the
mothership is possibly on a remote machine through an opaque
CRConnection data structure. New SPUs or
other system components can access any piece of configuration through
this library.
A packer library for packing OpenGL commands into network buffers
This library provides a "packing" function for each OpenGL function.
When such a function is called, the function arguments and an opcode
are packed into a buffer for eventual transmission. Although this
library exports a full OpenGL interface, it is not really a SPU. In
particular, the packing functions for OpenGL calls requiring pixel
data (e.g., glTexImage2D) require
additional arguments detailing the unpacking state. For an example of
this usage, see spu/pack/pack_pixel.c.
Currently, this library is used by both the pack and tilesort SPUs.
A library for loading SPUs from disk
This library loads a SPU from a named file and creates its dispatch table. Although this library is mostly designed to be used internally by the OpenGL stub library and the server, applications can load SPUs explicitly if they wish.
A library for tracking OpenGL state
This library provides the necessary functionality to track the
complete OpenGL graphics state. This library is quite complex, both
because the OpenGL state is large and intricate, and also because it
contains the logic for doing differencing between contexts. Graphics
contexts are represented with "hierarchical dirty bits" so that they
can be compared extremely efficiently, a frequent operation when
updating a tiled display or switching between contexts on the server.
This library does most of the OpenGL error checking, and satisfies
glGet* requests locally. For a
description of the techniques used by this library, see our 2000 Eurographics/SIGGRAPH Workshop on Graphics Hardware
paper. This library is used heavily by both the tilesort SPU and the server, and the pack SPU uses it sparingly to keep track of the pixel
unpack state for textures.
A library for unpacking OpenGL command buffers
The dual of the packing library mentioned above, this library decodes network-encoded OpenGL streams and dispatches them to an arbitrary SPU interface. Currently this is only used by the server, although a special SPU could use the pack library and the unpack library together to make multiple passes over a stream.
A "utility" library
This library contains a number of useful routines and data structures that are used by many components of the system. A (possibly non-exhaustive) list of features exported by this library is:
A few small programs are built as part off the Chromium build process. They won't all be listed here, as many of them are small test or demo programs.
resetms
This program resets the mothership to its initial state. Because multiple applications or servers could run on one computer (called, say, "sluff"), the mothership remembers when it has been contacted by a server running on "sluff", so the next time a server running on "sluff" identifies itself, the mothership knows which one it is. This means that after a Chromium run, the mothership is in an unusable state, since it will no longer accept server connections from "sluff". Running resetms will destroy all of the mothership's internal state. This is generally preferable to quitting the mothership and starting it again, especially since the socket isn't always properly released, which will prevent the mothership from starting up on its default port for a minute or two.
quitms
This program contacts a running mothership and tells it to quit. Useful in circumstances when you can't easily get to the machine the mothership is running on.
sputest
This program is a simple demonstration that an application can load a SPU and use it without the help of crappfaker or the OpenGL stub library. It draws a single triangle rotating around the center of the screen.
psubmit
This is just about the simplest example of a parallel OpenGL program. Instance "1" of this program will draw a blue triangle, and instance "2" of this program will draw a green triangle. With the right config script (see mothership/configs/psubmit.conf for an example), this program can be used to verify that the server's parallel API synchronization primitives are working and that parallel submission is, in theory, possible.
Most of the other programs that are built are designed to exercise one particular portion of the system. Some of them are contributed by WireGL users (thanks in particular to Peter Kirchner for providing some VERY stripped down programs that trip a bug), and there are a number of programs that each demonstrate the functionality of a single OpenGL extension, thanks to Chris Niederauer (all found in the progs/extensions/* directories). Run them. Show off anisotropic texture filtering on your display wall, it's fun.
The Chromium distribution contains some other useful stuff that isn't a SPU, library, or program:
A pretty slick build system
Chromium's build system is mainly designed to achieve two goals. First, creating a new Makefile for a library or program should be as simple as possible. Second, it should be possible to build the code on multiple operating systems from the same network-mounted drive without any object files or dependencies conflicting with each other. Users who want to port Chromium to a new system will need to create a new make-system-configuration file in config/, most likely just using one of the existing files as a template. The build will also quit with an error on various system-dependent things in the "utility" library, such as the abstraction for DLLs, etc. The build system also includes a very fast perl-based C/C++ dependency generator due to Matthew Eldridge in scripts/fastdep.pl. This was rewritten in Python by Christopher Waters to remove the Perl requirements from Chromium.
This documentation
A copy of this documentation can be found in doc/, if that's not where you're reading it from already.
A parsed representation of the OpenGL API
The scripts in glapi_parser/ parse a
massaged version of the Windows OpenGL header, and also a
Chromium-specific header containing our supported extension functions
(e.g., glBlendColorEXT, glBarrierExecCR, etc). This parsed representation
is used by almost every component of the system to automatically
generate some or all of their code.
Useful routines for manipulating the parsed representation of the OpenGL API
The routines available in opengl_stub/stub_common.py are available for use by any
Python script that is generating Chromium code. It is used quite
heavily throughout most of the system. In particular, it provides a
very easy way to tag certain OpenGL functions as "special". For
example, many of the OpenGL packing functions can be auto-generated,
but some (like glMaterialfv) are more
complicated and need to be coded by hand. The complex functions can
simply be listed in a separate file, and the contents of that file
efficiently examined through the API exposed by these Python
routines.