Improved File Format for omwaddon files
Posted: 04 Apr 2020, 00:53
I don't know what the current plans for the future of the omwaddon file format are, but I had an idea for a new file format to make mod creation/interaction simpler.
Now, I'm not as familiar with how esps/omwaddons work as I'd like to be, but current esp/omwaddon files cause a number of problems when interacting with each other, which, to my understanding, are largely due to the fact that records overwrite existing records of the same name, so to modify a record, even if only modifying a single subrecord, you need to include a copy of the entire record, plus your modifications.
Why not create a new format that instead can modify individual subrecords? With such a format tools such as the Bashed Patcher, Mator Smash, TES3Merge, etc. wouldn't be necessary, as each mod would only make the changes they need to make, and mods would cleanly apply on top of each other (mods that modify the same subrecord would still conflict, but then it just comes down to load order).
I.e rather than just defining records, you have an AddRecord operation, a ModifyRecord operation, and a RemoveRecord operation. AddRecord would need to have all fields, but ModifyRecord could include as many or as few fields as the author wants to modify, and all other fields will remain unchanged vs any other plugins that defined/modified that record earlier in the load order.
It's worth noting that, while it would be great to have support for this in-engine, it could also be done outside the engine, with a tool that will take your plugins and merge them together into a vanilla-style plugin. Further, we could also have a tool to automatically convert old-style esps/omwaddons to this new format, in essence re-creating TES3Merge in two steps.
A big plus for this would be that we could use a purely text-based format, meaning that mods would be much easier to read, modify and store in version control (This isn't something inherent to the proposed idea, but we may as well do it if introducing a new plugin format). It would also be possible to include comments inside the plugin.
The first example I tried was the MRM Puzzle Canal fix, as its small, but it seems like CELL object references are already handled nicely, so it's not really worth reproducing here, given that it doesn't do anything that the current esp format doesn't already do.
The second example I tried better demonstrates the idea, and is from At Home Alchemy, which has been graciously provided under a free license by Syclonix.
Below is a mock-up of what such a file could look like, omitting most records for brevity's sake. I've used yaml as the markup language, as I find it handles this sort of thing in a clean and readable way, however it would be relatively easy to support different input formats such as xml or json.
Fortunately the output of "tes3cmd dump" is actually in a relatively yaml-like format already.
Now, I'm not as familiar with how esps/omwaddons work as I'd like to be, but current esp/omwaddon files cause a number of problems when interacting with each other, which, to my understanding, are largely due to the fact that records overwrite existing records of the same name, so to modify a record, even if only modifying a single subrecord, you need to include a copy of the entire record, plus your modifications.
Why not create a new format that instead can modify individual subrecords? With such a format tools such as the Bashed Patcher, Mator Smash, TES3Merge, etc. wouldn't be necessary, as each mod would only make the changes they need to make, and mods would cleanly apply on top of each other (mods that modify the same subrecord would still conflict, but then it just comes down to load order).
I.e rather than just defining records, you have an AddRecord operation, a ModifyRecord operation, and a RemoveRecord operation. AddRecord would need to have all fields, but ModifyRecord could include as many or as few fields as the author wants to modify, and all other fields will remain unchanged vs any other plugins that defined/modified that record earlier in the load order.
It's worth noting that, while it would be great to have support for this in-engine, it could also be done outside the engine, with a tool that will take your plugins and merge them together into a vanilla-style plugin. Further, we could also have a tool to automatically convert old-style esps/omwaddons to this new format, in essence re-creating TES3Merge in two steps.
A big plus for this would be that we could use a purely text-based format, meaning that mods would be much easier to read, modify and store in version control (This isn't something inherent to the proposed idea, but we may as well do it if introducing a new plugin format). It would also be possible to include comments inside the plugin.
The first example I tried was the MRM Puzzle Canal fix, as its small, but it seems like CELL object references are already handled nicely, so it's not really worth reproducing here, given that it doesn't do anything that the current esp format doesn't already do.
The second example I tried better demonstrates the idea, and is from At Home Alchemy, which has been graciously provided under a free license by Syclonix.
Below is a mock-up of what such a file could look like, omitting most records for brevity's sake. I've used yaml as the markup language, as I find it handles this sort of thing in a clean and readable way, however it would be relatively easy to support different input formats such as xml or json.
Fortunately the output of "tes3cmd dump" is actually in a relatively yaml-like format already.
Code: Select all
Version: 1.3
Is_Master: false
Author: "Syclonix"
Description: "Version 1.0
Allows you to finally use alchemy apparatuses without having to
place them in your inventory. Just \"fire up\" the apparatuses you
want to use to bring up the alchemy menu (mortar and pestle must be
activated last).
Requires Tribunal"
Masters:
- Morrowind.esm
Records:
- Record:
type: GLOB
NAME: syc_AHA_Calcinator
FNAM: short
FLTV: 0.00
# At Home Alchemy only adds SCRI subrecord, but
# also reproduces the entirety of the original record
# All we really need here is the id and the new SCRI
# subrecord. all other fields could be determined using the
# original record.
- ModifyRecord:
NAME: apparatus_a_calcinator_01
SCRI: Script:syc_AHA_a_calcinator_01
# Original: For reference.
# Record: APPA "apparatus_a_calcinator_01" Flags:0x0000 ()
# NAME: ID:apparatus_a_calcinator_01
# MODL: Model:m\Apparatus_A_Calcinator_01.NIF
# FNAM: Name:Apprentice's Calcinator
# AADT: Type:(Calcinator) Quality:0.50 Weight:25.00 Value:10
# ITEX: Icon:m\Tx_calcinator_01.tga