Items Part 1 – Bootstrapping

Loot. Kit. Swag. Treasure. Whatever you call it, items are an important part of any RPG game loop. They provide the means for your character to defeat ever more powerful foes and create incentive to keep playing in search of the next big upgrade. While Daggerfall’s items don’t quite tickle the reward centres of the brain like Diablo 3 or World of Warcraft, they’re still a vital part of the play experience. Without decent gear and enchantments, you’re unlikely to survive the grueling ordeal of Mantellan Crux.

In this series, I’ll describe the process of adding items to Daggerfall Unity. I wanted to approach items early on as they will be involved at almost every level of the game. Shops sell them, blacksmiths repair them, monsters drop them, quests reward them. Your character may have a special affinity for bladed weapons, or be forbidden the use of shields. Even the biography questions when building a character can grant you items like the near-essential Ebony Dagger. With items embedded in almost every major game system, the hardest part was working out where to begin.

I decided to start with existing items as part of importing classic Daggerfall saves then bootstrap the whole item back-end from there. That way I could be certain I was dealing with the most real-world data possible. Having built support for classic saves in 0.1, I could already identify item records parented to the main character record and visualise them with a custom Unity Editor script. They looked a bit like this at first:

 

Items1

It’s not much, but at least I could find item records belonging to the character. The “Container” record is just a generic parent record. In this context, think of it as the character’s backpack.

The next step was to break apart the item record format. Fortunately the UESP came to the rescue here with most of the bytes already solved, but far from the whole story as you’ll see once the names are revealed:

Items2

A Frosty what of Ice Storms? OK, so there’s more to this than just the save record. How to we go about filling in the blanks? The key here is the “category” 16-bit field in that UESP article. This is actually a pair of 8-bit values. The first byte is the item group, the second byte is a table lookup for the item template within that group. The template indexed by this lookup has all the missing pieces of information we need to complete our item data. Now we have two more problems to solve. Where are the templates, and how to use those category bytes to find them? Let’s start with the templates.

Item templates are actually built into FALL.EXE. The offset is a little different depending on your version, but the easy way to locate them is open a hex editor and search for “ruby”. You will find the following data:

Items3

Here are all the item templates laid out one after the other. They even follow a certain kind of logic, with gems, weapons, armor, etc. all more or less grouped together. Fortunately this isn’t exactly unknown data and the UESP came to the rescue again with a good starting point for these templates. I just had to fill in some blanks.

I didn’t want to keep this data in the .exe however, it’s much harder to modify these templates later. That’s why I exported the item templates to JSON format. Once exported the above data looks like this:

Items4
Much easier to work with. There are still a few unknowns to work out but those will be solved over time. The next problem was how to link up instantiated items like our Frosty %it of Ice Storms back to their original template. I had to reproduce the lookup table Daggerfall was using internally.

It was here Lypyl provided a helping hand thanks to his research into magical items and artifacts. The file format of MAGIC.DEF is very similar to instantiated items found in save games. Furthermore, the creators of old item editors had solved quite a few of these problems back then. Armed with all this, Lypyl could derive enough information to rebuild the group and item tables which he kindly provided to me in C# enums. All I had to do then was link the enums back to their template index in the above JSON file.

The main group enum looks like below. It corresponds to the first byte of the earlier category short.

Items5

For every element in the above enum (such as Armor, Weapons, etc.) there is an enum for every individual item in that group. For example:

Items6

For the item enum, the individual item value is an index back into the template table. The order within the enum corresponds to the second byte of the category short. With a helper class to bring all this together, it was now possible to perform lookups from instantiated items back to their template data. This is how our items viewer looks now:

Items7

Success! We can now resolve an item’s template by type to discover the full name and other useful information. The next step was to determine which items are equipped on the character. Fortunately the “equipped” table is just another record in your save game, and was already known about thanks to that first UESP article. I just had to work out how that table referenced items and I could isolate which were equipped. Items marked by an asterisk are equipped to character.

Items8

There are almost two dozen equipment slots in total that map to specific parts of the character’s body and elsewhere. I will describe this in more detail in a future article.

With all of that research out of the way, my next job was even less visual than above. I had to write support classes such as API helpers and an entity item collection class. I also required a new type of image reader to handle the job of loading and caching item images for the inventory UI, tinting them based on material, cutting out unique alpha indices like the hair mask, and so on. Anyway, boring or not these new classes form the foundation of items in Daggerfall Unity and will continue to grow as needed.

With everything finally in place, I could start building the equipment UI to sort, view, and equip items imported from classic Daggerfall save games. Besides a few UI enhancements and fixes, the following came together fairly quickly.

[gfycat data_id=”PeacefulReflectingBarebirdbat”]

Some of the enhancements in this gfy include a scrollbar and mouse wheel scrolling. No reason we can’t have a few light modern touches to make our lives easier.

More About Scaling

One of the issues that caught me by surprise was a UI scaling problem. This turned out to be more interesting than expected, so I decided to write up a visual diary post about it.

For those who didn’t experience the problem, this is how it looked (1280×960).

ScaleBug1

What’s happening here isn’t technically a scaling problem, it’s a positioning problem. See, Daggerfall has a fixed-size UI of 320×200 pixels. My custom UI system was designed from the ground up to scale Daggerfall’s fixed UI to any resolution while maintaining correct aspect ratio. Depending on the width to height ratio of your resolution, the fixed UI may need to be pillarboxed (black bars down the sides) or letterboxed (black bars top and bottom). The goal is to fit the entire UI into the display without any clipped off edges and keeping that pixel-perfect ratio.

So what’s going in the screenshot above? The answer is that I forgot to turn on vertical alignment in the UI for the parent panel. This means the UI is stuck to the top of the screen instead of letterboxing like it’s supposed to. This is how the above display should look (1280×960).

ScaleBug2

The scaling and aspect ratio are correct in both cases, just the UI wasn’t centering vertically like it should for the letter-box effect at that resolution.

Unfortunately, it really isn’t possible to avoid pillarboxing or letterboxing with a fixed UI, unless you have a resolution that is an exact multiple of 320×200. For example, the screenshot below is a perfect x4 multiple of 320×200 and fits the frame completely at the correct aspect ratio (1280×800).

ScaleBug3

Now it occurs to me that some people simply don’t want letterboxing or pillarboxing. The best solution I can offer is a new option in the INI called “FreeScaling”. When this is enabled, the GUI will scale width and height independently. Here’s an example with FreeScaling enabled (1280×960).

ScaleBug4

The result is the UI is stretched as required to fill entire viewport. This obviously means the aspect ratio is no longer correct, but the chunky pixels still don’t look that bad with a little stretching. I’m willing to bet a lot of people actually play this way in DOSBox without noticing. It’s all down to personal preference anyway. If you want perfect aspect ratio, just leave things at default and the UI will scale and position itself properly now. If you definitely want to get rid of the black bars, then enable FreeScale and enjoy.

It’s also worth noting this does not apply to the game view rendering, which always fills the entire viewport. Only the classic 320×200 UI has this quirk.

I’m just happy my retro UI system is robust enough to handle all these different resolutions, scales, and positions while still working as it should. That’s an accomplishment by itself.

First Look At Classic Save Importing

Just a quick update today, as I’m right in the thick of it getting the test build ready.

The below GFY shows the current early state of importing classic Daggerfall save games inside Daggerfall Unity. At the moment this is just spawning a character to exact position in world, but I hope to be importing all vital character details eventually. As with everything else, classic save import will become more advanced over time. My ultimate goal is for you to be able to import your Daggerfall characters and pick up the game more-or-less where you left off.

This represents a nice milestone in development so far, one that has required several weeks of planning and engineering just to accomplish this much. Now to put some meat on those bones.

[gfycat data_id=”JitteryAdventurousEnglishpointer”]

Character Creation – Part 2

I have been working through the character creation process screen-by-screen. This has required a few diversions to work on hacking some additional file formats and of course writing new controls for the GUI system.

I nearly have the whole process finished, with the current limitations of not supporting biographies, custom classes, or the questions to select your class. These will be added in a second pass over character creation once more atomic systems are in place. The biographies in particular interface with systems like inventory and reputation, neither of which are implemented yet. All of these systems will come online in their own good time.

One great side-effect of using the existing Daggerfall files as-is will be that existing translation packs by international Daggerfall fans should work flawlessly with Daggerfall Unity. This comes with the other side-effect of having the same text limitations as Daggerfall, but to remedy this I have created a text interface in Daggerfall Tools for Unity called ITextProvider. Programmers are familiar with interfaces as a kind of contract between provider and consumer. Right now, all text you see is being routed through the default ITextProvider, which uses Daggerfall-native string methods. However, it’s possible to swap this out for another ITextProvider implementation using a different source for text. This is still quite early right now, but should pave the way for more advanced translation efforts in the future.

To mix things up a little from this typically English-centric site (it’s the only language I know, I’m sorry everyone) here’s the current state of character creation after applying Le Projet French Daggerfall to my installation. Everything here is running 100% inside Unity using a standard Daggerfall install for data.

Translation1 Translation2 Translation3 Translation4 Translation5 Translation6 Translation7

 

Daggerfall VID Playback

I’m going all-out in Daggerfall Tools for Unity 1.4. You will also be able to play Daggerfall’s VID movies right inside Unity to any texture you like. I’ve even provided a VID player panel for the native UI discussed in my previous post.

This is just a prototype, currently without audio or correct timings. I’ll show this off in a YouTube clip sometime in the future once its properly integrated.

[gfycat data_id=”UnrulyElaborateBagworm”]

Native User Interface Intro

The portable UI being developed for Daggerfall Tools for Unity 1.4 is coming together. Rather than use a uGUI simulacrum of the Daggerfall interface, I decided to re-implement Daggerfall’s native UI screen by screen, making it as true to Daggerfall as possible.

StartScreenUI

 

 

This has involved a fair amount of work to ensure everything is rendered at true native resolutions and scales proportionately to any screen size. Here’s an early test clip where I’m playing with the scaling, which runs in real-time.

[gfycat data_id=”PoliticalPitifulEgg”]

Another challenge was pixel-perfect text rendering. The internal UI in Daggerfall Tools for Unity uses Daggerfall’s pixel fonts exactly, which required a custom text renderer. Below is the book reader UI flicking through The Real Barenziah. The formatting codes from Daggerfall are followed so precisely that badly formatted text even overflows in the same places it does in game.

[gfycat data_id=”ImprobableGreedyIriomotecat”]

Behind the scenes, the GUI runs on a state machine stack where windows can be pushed and popped as required. Each window is a hierarchy of components (panels, buttons, labels, etc.) that all follow the same consistent layout behaviour and handle fundamental UI tasks like firing messages when clicked. Everything is rendered and hit-tested internally at Daggerfall’s native 320×200 resolution.

Communication between windows uses a custom messaging system that runs independently of platform. The messages themselves can even work a bit like command strings in that they’re able to pass parameters directly inside the message.

And the best news is that you aren’t bound to using this GUI in your projects. It exists inside the Demo namespace, which means that nothing in the core library or API has any dependency on it. You’re free to use this GUI or ignore it completely to implement your own interface using whatever UI platform you’re most comfortable with.

There’s still a lot of work to do before it’s ready for Daggerfall Tools for Unity 1.4, but everything is taking shape very quickly.