A companion to the Visual Diary, my Technical Content series will go into more arcane aspects of Daggerfall Scout. Today’s tech-talk is about how XML is used in the Daggerfall Cache to store world files in a format both Humans and computers can read. Before going into more detail, a quick primer for The Daggerfall Cache is required.

After running the Cache Manager to build your personal cache, the following directory structure is built:

-MyDaggerfallCache
	-Default
		+Bitmaps
		+Blocks
		+Maps
		+Meshes
		+Materials
	Default.Provider.xml

In this model, Daggerfall Scout is both a cache provider and a cache consumer. When opening DF Scout, it first locates the Default.Provider.xml file containing a description of subpaths in which content is held. DF Scout is built to understand the layout created by the Default provider. It would be possible to build your own cache provider or consumer. Each distinct cache provider has a unique parent folder and .Provider.xml file describing the layout and builder versions. A consumer program opens its matching provider to locate content.

Each of the content groups has been homogenized into one media type. Where Daggerfall has multiple bitmap formats, the Default builder converts all of these into alpha-enabled PNG files. 3D objects and materials are Ogre .mesh and .material files. Blocks and Maps are saved as pure XML. Starting with a birds-eye view, let’s look deeper at these XML files and how one is used to build a simple location.

When opening a map, DF Scout locates map filenames in this format: RegionName_LocationName.map.xml. So the city of Daggerfall in the region of Daggerfall would be: Daggerfall_Daggerfall.map.xml. A map simply contains layout information for specific blocks. Take a look at Daggerfall_The Ashcroft Graveyard.map.xml below. This map contains only a single location block and a very small dungeon.

<?xml version="1.0" encoding="UTF-8"?>
<Map Name="The Ashcroft Graveyard" NativeType="140" TypeDesc="Ancient Graveyard" HasDungeon="1">
  <Location Identifier="20934" BlocksWide="1" BlocksHigh="1">
    <Block GridX="0" GridY="0" Filename="GRVEAS15.RMB">
    </Block>
  </Location>
  <Dungeon Identifier="20935" BlocksWide="2" BlocksHigh="2">
    <Block GridX="-1" GridY="0" Filename="B0000008.RDB" PlayerStartBlock="0">
    </Block>
    <Block GridX="0" GridY="-1" Filename="B0000006.RDB" PlayerStartBlock="0">
    </Block>
    <Block GridX="0" GridY="0" Filename="M0000000.RDB" PlayerStartBlock="1">
    </Block>
    <Block GridX="0" GridY="1" Filename="B0000003.RDB" PlayerStartBlock="0">
    </Block>
    <Block GridX="1" GridY="0" Filename="B0000001.RDB" PlayerStartBlock="0">
    </Block>
  </Dungeon>
</Map>

The <Map> element encapsulates the document. Its attributes describe the map name as seen on the travel map, a native (Daggerfall-specific) type number, and a descriptive string. Of particular note is the HasDungeon attribute, denoting if the map contains dungeon data or not. This can also be derived by the existence of a <Dungeon&#62 element, but the HasDungeon attribute helps in selectively parsing the XML file using a forward-only reader.

Both the <Location&#62 and <Dungeon&#62 tags encapsulate block layout on a 2D grid. Child <Block&#62 elements describe the filename for each block, and its position on the grid. Now that we’ve seen how maps are laid out, let’s drill down into a block to see how it’s composed. In this case, we’re using GRVEAS15.RMB as it’s the only location block used to represent The Ashcroft Graveyard.

A single block XML file is a bit large to fit comfortably in this post. You can view the entire contents of GRVEAS15.RMB if you wish. The general block structure is as follows:

<Block Identifier="GRVEAS15.RMB">
  <Records RecordCount="17">
    ...
  </Records>
  <MiscMeshes Count="34">
    ...
  </MiscMeshes>
  <MiscFlats Count="2">
    ...
  </MiscFlats>
  <GroundPlane>
    ...
  </GroudPlane>
</Block>

The <Block&#62 element encapsulates all block data. <Records&#62 describes general 3D objects populating the block, such as houses and temples. <MiscMeshes&#62 are 3D clutter such as wagons, fountains, and fences. <MiscFlats&#62 represent 2D clutter like cows, horses, shrubs, and statues. Finally, the <GroundPlane&#62 element details the 16×16 texture square grid under each block. Every block is 4096×4096 world-space units in size, and the ground plane sits under this perfectly, each texture square being 256×256 world-space units.

So what does all this stuff actually mean? Taking the map XML file above, you could render individual components to visit The Ashcroft Graveyard just as it appears in the game. Map layouts range from 1×1 blocks, all the way up to 8×8 blocks for the largest areas such as Daggerfall City. Rendering large areas is just a matter of snapping all the blocks together and rendering their individual components.

Speaking of components, let’s go into the <Records&#62 element in a little more detail. An individual record looks like this:

<Record RotateX="0" RotateY="0" RotateZ="0" TranslateX="-1536" TranslateY="0" TranslateZ="512">
  <Meshes Count="1">
    <Mesh Identifier="750" RotateX="0" RotateY="0" RotateZ="0" TranslateX="-24" TranslateY="0" TranslateZ="24">
    </Mesh>
  </Meshes>
  <InsideMeshes Count="0">
  </InsideMeshes>
</Record>

Attributes of the parent element describes how this 3D object should be rotated and translated into world space, relative to the parent block. <Meshes&#62 describes how many individual 3D objects make up this particular record. For outdoor areas there is typically only one mesh per record, although there are certainly exceptions. <InsideMeshes&#62 plots out how the interior of this record should appear. A good example is any house in Daggerfall – the act of going inside the house loads the inside meshes. This is used extensively in cities to keep locations with interiors nicely grouped. For outdoor scenery, such as GRVEAS15.RMB, the <InsideMeshes&#62 element is rarely populated.

Back to the <Block&#62 itself, <MiscMeshes&#62 and <MiscFlats&#62 are simply strewn relative to their parent. Their transforms are detailed as attributes for each child item.

Each block <GroundPlane&#62 is described by 256 <Square&#62 elments. These are a bit unusual as they don’t specify an image archive, just an image index. This is because there are several viable ground texture sets, based on the terrain (temperate, desert, swamp, and mountain) and the climate (snowing or raining). By applying different texture sets to the meshes and ground planes, Daggerfall achieves a wide variety of appearances using the same 3D data. Each <Square&#62 may also be rotated 90 degrees or flipped in both X an Y axes.

Finally, you will notice the record above specifies a mesh with an identifier of “750”. This tells the engine which mesh to load for that record before applying terrain/weather processing (if required) and transforming into position.

There is much more data held inside a block: doors, trees, action records, NPCs, and more. As these features come online in Daggerfall Scout, I will add them to the XML files exported by Cache Manager. Every time I release a new version of Daggerfall Scout, it’s possible to increment the internal cache version. The program will prompt you to rebuild the cache from your Daggerfall CD so you can explore the new feature.

This Technical Content article has been an overview of how Daggerfall Scout represents blocks and maps using XML. The goal is to stay close to how Daggerfall stores data, while representing it in a Human-readable format. The purpose of this design is to allow technically-minded players to explore the game world using nothing more than Notepad, and never need to deal with complex native file formats.

Below, you can see the results of loading The Ashcroft Graveyard in Daggerfall Scout. The engine neatly parses all the XML and loads the appropriate resources.