Items Part 1 – Bootstrapping

Loot. Kit. Swag. Treasure. Whatever you call it, items are an important part of any RPG game loop. They provide the means for your character to defeat ever more powerful foes and create incentive to keep playing in search of the next big upgrade. While Daggerfall’s items don’t quite tickle the reward centres of the brain like Diablo 3 or World of Warcraft, they’re still a vital part of the play experience. Without decent gear and enchantments, you’re unlikely to survive the grueling ordeal of Mantellan Crux.

In this series, I’ll describe the process of adding items to Daggerfall Unity. I wanted to approach items early on as they will be involved at almost every level of the game. Shops sell them, blacksmiths repair them, monsters drop them, quests reward them. Your character may have a special affinity for bladed weapons, or be forbidden the use of shields. Even the biography questions when building a character can grant you items like the near-essential Ebony Dagger. With items embedded in almost every major game system, the hardest part was working out where to begin.

I decided to start with existing items as part of importing classic Daggerfall saves then bootstrap the whole item back-end from there. That way I could be certain I was dealing with the most real-world data possible. Having built support for classic saves in 0.1, I could already identify item records parented to the main character record and visualise them with a custom Unity Editor script. They looked a bit like this at first:

 

Items1

It’s not much, but at least I could find item records belonging to the character. The “Container” record is just a generic parent record. In this context, think of it as the character’s backpack.

The next step was to break apart the item record format. Fortunately the UESP came to the rescue here with most of the bytes already solved, but far from the whole story as you’ll see once the names are revealed:

Items2

A Frosty what of Ice Storms? OK, so there’s more to this than just the save record. How to we go about filling in the blanks? The key here is the “category” 16-bit field in that UESP article. This is actually a pair of 8-bit values. The first byte is the item group, the second byte is a table lookup for the item template within that group. The template indexed by this lookup has all the missing pieces of information we need to complete our item data. Now we have two more problems to solve. Where are the templates, and how to use those category bytes to find them? Let’s start with the templates.

Item templates are actually built into FALL.EXE. The offset is a little different depending on your version, but the easy way to locate them is open a hex editor and search for “ruby”. You will find the following data:

Items3

Here are all the item templates laid out one after the other. They even follow a certain kind of logic, with gems, weapons, armor, etc. all more or less grouped together. Fortunately this isn’t exactly unknown data and the UESP came to the rescue again with a good starting point for these templates. I just had to fill in some blanks.

I didn’t want to keep this data in the .exe however, it’s much harder to modify these templates later. That’s why I exported the item templates to JSON format. Once exported the above data looks like this:

Items4
Much easier to work with. There are still a few unknowns to work out but those will be solved over time. The next problem was how to link up instantiated items like our Frosty %it of Ice Storms back to their original template. I had to reproduce the lookup table Daggerfall was using internally.

It was here Lypyl provided a helping hand thanks to his research into magical items and artifacts. The file format of MAGIC.DEF is very similar to instantiated items found in save games. Furthermore, the creators of old item editors had solved quite a few of these problems back then. Armed with all this, Lypyl could derive enough information to rebuild the group and item tables which he kindly provided to me in C# enums. All I had to do then was link the enums back to their template index in the above JSON file.

The main group enum looks like below. It corresponds to the first byte of the earlier category short.

Items5

For every element in the above enum (such as Armor, Weapons, etc.) there is an enum for every individual item in that group. For example:

Items6

For the item enum, the individual item value is an index back into the template table. The order within the enum corresponds to the second byte of the category short. With a helper class to bring all this together, it was now possible to perform lookups from instantiated items back to their template data. This is how our items viewer looks now:

Items7

Success! We can now resolve an item’s template by type to discover the full name and other useful information. The next step was to determine which items are equipped on the character. Fortunately the “equipped” table is just another record in your save game, and was already known about thanks to that first UESP article. I just had to work out how that table referenced items and I could isolate which were equipped. Items marked by an asterisk are equipped to character.

Items8

There are almost two dozen equipment slots in total that map to specific parts of the character’s body and elsewhere. I will describe this in more detail in a future article.

With all of that research out of the way, my next job was even less visual than above. I had to write support classes such as API helpers and an entity item collection class. I also required a new type of image reader to handle the job of loading and caching item images for the inventory UI, tinting them based on material, cutting out unique alpha indices like the hair mask, and so on. Anyway, boring or not these new classes form the foundation of items in Daggerfall Unity and will continue to grow as needed.

With everything finally in place, I could start building the equipment UI to sort, view, and equip items imported from classic Daggerfall save games. Besides a few UI enhancements and fixes, the following came together fairly quickly.

Some of the enhancements in this gfy include a scrollbar and mouse wheel scrolling. No reason we can’t have a few light modern touches to make our lives easier.

In the next installment of this series, I’ll talk about the paper doll system and equipping items to character.

Back On Deck For 2016

Happy New Year everyone! I’m back from holidays and almost on top of my RL workload again. That means a whole new round of updates to Daggerfall Unity and DFTFU are about to begin. I’ve picked up where I left off last year with the item and inventory system, and will be posting more on this shortly.

Sometime in the next few weeks, I’ll start adding new test builds leading up to the 0.2 release. Key features of 0.2 will be:

  • Basic inventory system and loot tables. Import items from classic saves.
  • Travel map interface (by LypyL).
  • Dungeon and interior automap interface (by Nystul).
  • More bug fixes and incremental updates.
  • Some more community resources for contributors.

I’ve also made a small new year’s resolution to post more technically-minded articles in 2016, as I let this slip with all the rapid-fire updates leading to 0.1. It was quite a shift for me going from pure tool development to building a game, and I rather miss just talking about what I’m working on.

Thanks for all your patience during the holiday season. I look forward to reading your feedback with the next round of updates.

Daggerfall Unity 0.1 Release

DaggerfallUnity0.1Release

 

After several test iterations, Daggerfall Unity is feeling reasonably solid across the two main platforms Windows and Linux. A huge thanks to all testers who helped discover early bugs and quirks. There’s obviously still more to fix, and a lot of game features to implement, but we have a pretty good starting point with this build. At some point, I need to push a stick into the ground and say “it starts here.”

 

It Starts Here

Daggerfall Unity 0.1 is now available for general download. Key features of this build are:

  • Create a new character or load an existing character from a classic Daggerfall save.
  • The entire world of the Illiac Bay is ready to explore at 1:1 scale to Daggerfall itself. Go anywhere, enter any dungeon, and explore any town.
  • Quick-save and quick-load your progress.
  • Dungeons are populated with fixed and random enemies just like in Daggerfall.
  • Basic combat mechanics with the ebony dagger.
  • Game console for enabling god mode, setting run speed, teleporting around world, etc.
  • Lypyl’s enhanced sky mod with dynamic skybox, procedural clouds, and even phases of the moon!
  • Nystul’s beautiful far terrain mod and improved terrain generation.

 

Download

Updated to 0.1.1.

Download Daggerfall Unity 0.1.1 (Windows) Download Daggerfall Unity 0.1.1 (Linux)

 

Manual

A PDF manual is included with the download, but you can also download a standalone copy.

 

Feedback

If you would like to offer feedback and bug reports, please use this thread on the forums or contact me directly.

 

What’s Next

I will be a little quiet the next few weeks due to current work commitments, but rest assured the next release is already being worked on. I will soon repeat the process of dropping test builds leading up to the next release milestone. Here’s a list of features scheduled for the next release.

  • Early item support! Inventory will be imported from classic saves.
  • Ability to open inventory window and equip items.
  • Loot enemy corpses and treasure piles.
  • Full save/load UI allowing for multiple saves.
  • In-game options UI for changing game settings.
  • The ability to rest to recover health.
  • More contributor mods.
  • More bug fixes.

There’s a lot of open source activity happening at the moment from contributors, so it’s possible we might end up with more features in the second release than listed above. I will reveal these on Twitter as they become ready to show off.

Daggerfall Unity Test Build 3

DaggerfallUnityTest3Splash

Download

Download Daggerfall Unity 0.0.6 (Windows) Download Daggerfall Unity 0.0.6 (Linux)

Please also download the ReadMe!

 

Patch Notes

  • Changed how genders are imported from classic saves based on new idea that gender byte is actually a bitfield and only first bit controls gender (0 for male, 1 for female). This should allow more classic saves to open. Requires more testing, please report any further issues with gender-swapped characters. Vampire/werewolf characters are still not considering working.
  • Fixed issue where player would appear in wrong exterior cell after loading game to an interior cell.
  • Fixed issue where saved player position would be incorrect after changing terrain sampler (e.g. enabling/disabling Nystul’s improved terrain then loading a saved game). When a change to terrain sampler is detected, player will be relocated to outside current location, or to origin of map cell if no location present. This logic may trigger when loading a quick-save created prior to 0.0.6.
  • Fixed another casing issue when lazy-loading saves in Linux for save importer.
  • Some buildings do not have an interior. These will now display the old chestnut “This house has nothing of value.” rather than throw an exception.
  • Class-based enemies (e.g. Spellsword, Thief) now have a health pool and can hurt you. Look out!

 

Controls

General

  • Mouse to look.
  • W, S, A, D to move.
  • SHIFT (hold) to run.
  • SPACE to jump.
  • LEFT-CLICK mouse to open doors, enter dungeons, operate switches, etc.
  • ESC to pause game or go back to previous window.
  • F5 to open Character Sheet.

Weapons

  • Z to toggle weapon.
  • RIGHT-CLICK and drag mouse to swing weapon.

Save/Load

  • F9 to quick-save.
  • F12 to quick-load.

 

Feedback

Preferred method of feedback is the following thread on the forums.

http://forums.dfworkshop.net/viewtopic.php?f=18&t=161

 

When reporting bugs, please include the following in your report.

  • A clear description of what went wrong and process to reproduce problem (if possible).
  • Main system specs (OS, CPU, RAM, GPU).
  • Any output logs or save files as requested. See ReadMe for more information.

 

Thankyou!

And once again, a big thankyou to all contributors, testers, and supporters of this project. You are each helping to create something wonderful.

Daggerfall Unity Test Build 2

DaggerfallUnityTest2Splash

Download

Updated to 0.0.5.

Download Daggerfall Unity 0.0.5 (Windows) Download Daggerfall Unity 0.0.5 (Linux)

Please also download the ReadMe!

 

Controls

General

  • Mouse to look.
  • W, S, A, D to move.
  • SHIFT (hold) to run.
  • SPACE to jump.
  • LEFT-CLICK mouse to open doors, enter dungeons, operate switches, etc.
  • ESC to pause game or go back to previous window.
  • F5 to open Character Sheet.

Weapons

  • Z to toggle weapon.
  • RIGHT-CLICK and drag mouse to swing weapon.

Save/Load

  • F9 to quick-save.
  • F12 to quick-load.

 

Feedback

Preferred method of feedback is the following thread on the forums.

http://forums.dfworkshop.net/viewtopic.php?f=18&t=154

 

When reporting bugs, please include the following in your report.

  • A clear description of what went wrong and process to reproduce problem (if possible).
  • Main system specs (OS, CPU, RAM, GPU).
  • Any output logs or save files as requested. See ReadMe for more information.

 

Thankyou!

And once again, a big thankyou to all contributors, testers, and supporters of this project. You are each helping to create something wonderful.

Upcoming Leave Time

Hey everyone! I’m putting the wraps on the second test build of Daggerfall Unity, which should be ready sometime in the next 24 hours.

Before then, I wanted to take a minute to let you know that I’m heading to Melbourne next week for PAX Australia. We’re going for the whole week, not just the PAX weekend, which means I won’t be around to answer emails, drop by the forums, or commit any changes to git.

So if you don’t hear from me and everything seems to go quiet – don’t worry! I’m just taking well-deserved break and will be back to work starting in early November. I will still answer any forum queries I can if I get some spare time in the evenings.

I also wanted to extend a huge thank you to everyone who sent me their feedback on the first test build. You have helped me find several bugs and other problems that would have otherwise gone unnoticed. When the first true release hits, it’s going to be that much better for your time and information. I’m looking forward to repeating the process on the next test build.

All the best!

More About Scaling

One of the issues that caught me by surprise was a UI scaling problem. This turned out to be more interesting than expected, so I decided to write up a visual diary post about it.

For those who didn’t experience the problem, this is how it looked (1280×960).

ScaleBug1

What’s happening here isn’t technically a scaling problem, it’s a positioning problem. See, Daggerfall has a fixed-size UI of 320×200 pixels. My custom UI system was designed from the ground up to scale Daggerfall’s fixed UI to any resolution while maintaining correct aspect ratio. Depending on the width to height ratio of your resolution, the fixed UI may need to be pillarboxed (black bars down the sides) or letterboxed (black bars top and bottom). The goal is to fit the entire UI into the display without any clipped off edges and keeping that pixel-perfect ratio.

So what’s going in the screenshot above? The answer is that I forgot to turn on vertical alignment in the UI for the parent panel. This means the UI is stuck to the top of the screen instead of letterboxing like it’s supposed to. This is how the above display should look (1280×960).

ScaleBug2

The scaling and aspect ratio are correct in both cases, just the UI wasn’t centering vertically like it should for the letter-box effect at that resolution.

Unfortunately, it really isn’t possible to avoid pillarboxing or letterboxing with a fixed UI, unless you have a resolution that is an exact multiple of 320×200. For example, the screenshot below is a perfect x4 multiple of 320×200 and fits the frame completely at the correct aspect ratio (1280×800).

ScaleBug3

Now it occurs to me that some people simply don’t want letterboxing or pillarboxing. The best solution I can offer is a new option in the INI called “FreeScaling”. When this is enabled, the GUI will scale width and height independently. Here’s an example with FreeScaling enabled (1280×960).

ScaleBug4

The result is the UI is stretched as required to fill entire viewport. This obviously means the aspect ratio is no longer correct, but the chunky pixels still don’t look that bad with a little stretching. I’m willing to bet a lot of people actually play this way in DOSBox without noticing. It’s all down to personal preference anyway. If you want perfect aspect ratio, just leave things at default and the UI will scale and position itself properly now. If you definitely want to get rid of the black bars, then enable FreeScale and enjoy.

It’s also worth noting this does not apply to the game view rendering, which always fills the entire viewport. Only the classic 320×200 UI has this quirk.

I’m just happy my retro UI system is robust enough to handle all these different resolutions, scales, and positions while still working as it should. That’s an accomplishment by itself.

Test Release Fixes Incoming

Thanks to feedback from intrepid testers, I have a solid list of priority items to fix. I will start rolling out fixes in the next 24-48 hours, and may need to spread fixes across multiple test releases.

On the whole, the first compatibility test went very well. Daggerfall Unity ran for most people on a wide variety of systems, and most of the bugs found can be easily resolved. A very positive start to things.

Here’s the list of priority fixes coming in next couple of test builds.

  • UI scaling issue at many resolutions.
  • Alternate save path not used by SaveLoadManager.
  • Linux not loading career data for class selection dialog.
  • Needs Direct3D11 for full deferred shading on Windows. Will add INI options for downlevel rendering paths for all systems. (Note: This may reduce quality of lights and shadows).
  • Game not launching, crashing on video files. Confirmed *.VID files not present in Arena2. Need to fail cleanly and advise users on how to workaround.
  • INI options for tuning mouse weapon swinging.
  • INI options for some cheating, GodMode and ExplorerMode (from demos).
  • Pressing load with nothing to load will hang game.
  • Genders occasionally not read properly from vanilla saves.
  • Reaction setting occasionally not read properly from vanilla saves.
  • Loading some saves will drop you through the world.
  • Vampire characters not read cleanly. Need to at least read these characters back to starting race and appearance. Vampirism won’t be implemented until after quest system is done.
  • Rebindable keys. Will do this in a config file first, then later provide a UI.

Daggerfall Unity Test Build 1

StartingCave

Download

The first test build of Daggerfall Unity is now available for download. Grab your version below.

Download Daggerfall Unity 0.0.1 (Windows) Download Daggerfall Unity 0.0.1 (Linux)

Note: Please see Setting Up below for more information on running build.

Note for Linux users: This build has videos and MIDI audio disabled. This will be fixed in test build 2.

Update

  • Users are reporting UI scaling issues. I have been able to reproduce this and will fix as soon as possible.

Purpose

I plan to drop multiple test builds leading up to first true release. Each test build will open up more features and fix more bugs. By the time the first true release is available, I’m hoping most of the big problems will be resolved.

Thank you to any testers who jump in this early and let me know of problems you encounter. While testing, please keep in mind this build is specifically to test the below key items. At this time, I don’t require any bug reports unless it pertains to the following.

  • Does the build run on your system?
  • Can you start a new game, create a character, and spawn to the bottom of Privateer’s Hold?
  • Can you exit Privateer’s Hold into the outside world?
  • Can you quick-save and quick-load successfully while inside a dungeon and in wilderness?
  • Can you load an existing character from a classic Daggerfall save game?
  • Can you use F5 in-game to view your character sheet, and was your character loaded successfully? (Note: character items are not currently loaded).

Limitations

This build is missing several features planned for the first true release. Following are the main limitations of this build.

  • Only a single quick-save slot is currently available for Daggerfall Unity games. This is pending more work on custom UI windows.
  • Cannot save or load inside buildings.
  • When loading a classic save, the player will not spawn inside buildings or dungeons. If you have a save where your character is inside a building or dungeon your character will instead spawn to an approximate exterior position.
  • No fast travel.
  • Cannot rebind keys.
  • No resting to recoup health.

Setting Up

To play this build of Daggerfall Unity, please follow these steps.

  1. Have a clean install of Daggerfall available (patched to .213). DaggerfallSetup is preferred.
  2. Unzip the download package.
  3. Edit settings.ini inside Daggerfall Unity_Data folder to setup your game. See below section on INI file.
  4. Run the Daggerfall Unity application.

Note: CD-based installs of Daggerfall may not install videos locally, resulting in a crash to black screen. If this happens (and your Daggerfall path is definitely correct), check that you have 17 *.VID folders in your installs arena2 folder. If these are missing, you can copy .VID files from the CD to your local arena2 folder.

INI File

The INI file contains all game settings. Not all settings are currently live. Key settings are.

  • MyDaggerfallPath. This should be set to your top-level Daggerfall folder. This is the folder containing the “arena2” and “SAVE” folders. For example, this path might be C:\ Games\Daggerfall on Windows. On Linux, it will look something like /home/YourName/Games/Daggerfall. Remember that Linux paths are case-sensitive.
  • MyDaggerfallUnitySavePath. When left blank saves will be directed to Daggerfall Unity_Data folder.
  • InvertMouseVertical. If you want to invert your mouse.
  • MouseLookSmoothing. Toggles mouse smoothing.
  • MouseLookSensitivity. Set this value between 0.1 – 4.0 to adjust your mouse look sensitivity.

Controls

General

  • Mouse to look.
  • W, S, A, D to move.
  • SHIFT (hold) to run.
  • SPACE to jump.
  • LEFT-CLICK mouse to open doors, enter dungeons, operate switches, etc.
  • ESC to pause game or go back to previous window.
  • F5 to open Character Sheet.

Weapons

  • Z to toggle weapon.
  • RIGHT-CLICK and drag mouse to swing weapon.

Save/Load

  • F9 to quick-save.
  • F12 to quick-load.

Feedback

Preferred method of feedback is the following thread on the forums.

http://forums.dfworkshop.net/viewtopic.php?f=18&t=141

When reporting bugs, please include the following in your report.

  • A clear description of what went wrong and process to reproduce problem (if possible).
  • Main system specs (OS, CPU, RAM, GPU).
  • If requested, please email output_log.txt file from inside the Daggerfall Unity_Data folder.
  • If requested, please email QuickSave.txt file from inside the Daggerfall Unity_Data/Saves folder (or from wherever your saves are directed).
  • Note that because log and saves are both plain text files, it is also possible to post these to forum thread above.

Thankyou!

And once again, a big thankyou to all contributors, testers, and supporters of this project. You are each helping to create something wonderful.

First Look At Classic Save Importing

Just a quick update today, as I’m right in the thick of it getting the test build ready.

The below GFY shows the current early state of importing classic Daggerfall save games inside Daggerfall Unity. At the moment this is just spawning a character to exact position in world, but I hope to be importing all vital character details eventually. As with everything else, classic save import will become more advanced over time. My ultimate goal is for you to be able to import your Daggerfall characters and pick up the game more-or-less where you left off.

This represents a nice milestone in development so far, one that has required several weeks of planning and engineering just to accomplish this much. Now to put some meat on those bones.