Hello there, OpenMW forums. At the behest of psi29a, and with the approval of the OpenMW leadership, a new subforum has been created for tes3mp. I wish to extend my gratitude to everyone involved in that decision. OpenMW is one of the greatest open source projects and communities I have ever come across, I have thoroughly enjoyed working with its code, and I am humbled to see our fork become part of it in a more official capacity.
For my first post on this subforum, I would like to go through tes3mp's history thus far, its current set of features, the discrepancies between it and OpenMW, and some of the issues we will be facing in its future.
tes3mp is a fork of OpenMW started in December of 2015 and made public in July of 2016 that intends to add a seamless multiplayer experience to your project, the kind that fans of Morrowind have dreamt of playing with their friends and families since 2002.
I am one of its two developers along with Koncord, and – as a hypothetical pull request from us would reveal – we have thus far added 42,915 lines of code to our fork that don't exist in OpenMW.
It was Koncord alone who worked on tes3mp during its first seven months and came up with most of its architecture, drawing inspiration from projects like San Andreas Multiplayer and the Vault-Tec Multiplayer Mod for Fallout 3, having himself been a developer on the latter.
With so little development time, however, his initial release of the code on GitHub in July 2016 was simply a proof of concept for player synchronization. Players were able to see each other's movements, animations, equipment changes and melee attacks quite well in interior cells, but there were certain glaring problems, such as the inability of players to ever see each other again after moving across exterior cells, the lack of state saving, and a plethora of graphical glitches that ranged from Khajiit players intially spawning as Dark Elves with misshapen tails to players completely losing all animation after putting on armor. It was also near-impossible to build tes3mp's server app on Windows, with it using a C++14 feature that was not supported in Visual Studio 2015 and a dependency that could not be built in MinGW-w64 (a Lua replacement named terra).
That's where I came in. Although tes3mp could not live up to the hype already created around it, I noticed its networking and packet systems were very sound, its Lua scripting system was quite promising, its desire to keep most multiplayer logic separate from OpenMW's code allowed it – in theory – to effortlessly take in almost all changes made to OpenMW, and the fact that it was based on a well-made C++ recreation of an Elder Scrolls game's engine meant it would avoid some of the unsolvable difficulties that had plagued previous fanmade multiplayer attempts.
For those reasons, I've been fixing it and adding features to it with Koncord ever since, as well as providing most of its planning and conceptual direction.
To quote what I've written in our Steam group's FAQ:
In addition to the gameplay features mentioned above, we also have a server browser and a master server, allowing players to easily find and connect to each other.Player movements and cell changes are very well synchronized. You can go anywhere on Vvardenfell and meet up with other players correctly. You can even load up Tamriel Rebuilt, give yourself a massive Acrobatics skill using the console, jump across the entire width of Morrowind a few times and meet up with a player doing the same.
Player equipment, stats, combat and spell casting are also well-synchronized. Player stats, inventories and spellbooks are saved to and loaded from the server.
Picking up or dropping items in the world makes them appear and disappear correctly for other players, including in containers, as does spawning and deleting objects via the console, and their state will get saved to the server and then loaded for new arrivals. Opening, closing, unlocking and locking doors is synchronized and their changing states are saved to the server. Using certain buttons and levers – like the ones at Ghostgate, Dwemer ruins or Sotha Sil's Clockwork City – is also synchronized, although their state is not yet saved to the server.
However, other important aspects of the game are not yet synchronized, saved or loaded, including NPCs, time, weather, quest states, dialogue topics, bounties, faction membership, and the vast majority of ingame scripts.
We also have fully working ingame chat, as well as chat commands that run serverside Lua scripts.
Additions to OpenMW
The most immediately noticeable additions are two new apps: openmw-mp (which contains the code for tes3mp-server) and browser (the code for tes3mp-browser).
We also have two new dependencies: RakNet (our networking middleware) and Lua (our serverside scripting language). Lua can optionally be replaced with terra in Linux builds. RakNet is required by our client, server and browser, while Lua is only required by the server.
tes3mp adds a folder named openmw-mp to OpenMW's components. Inside it are classes for packets, packet controllers and the base objects whose data is sent via packets, and they are all used by both the client and the server.
tes3mp also adds a new folder named mwmp to the openmw app. In an attempt to prevent the multiplayer code from spilling into the rest of openmw, it contains most of the functionality for reading and sending packets on the client, handling our own GUI elements, continuously checking the local player's state, updating the NPC representations of other players, finding and making changes to objects in the world.
Any installation of the tes3mp server also requires a folder, with a customizable name, inside whose subfolders the Lua scripts, Lua libraries and server data can be stored.
Modifications to OpenMW
That being said, there have been some unavoidable modifications in the rest of the openmw app. A few notable examples follow:
The player spawns in Pelagiad for now instead of the Imperial Prison Ship or OpenMW's other default spawn point of exterior cell 0, 0.
The menu screens for saving and loading the game are gone, and both quicksaving and quickloading are disabled.
The booleans tracking whether the game should be paused have been changed so they are always false.
Most methods dealing with changes made to the world – such as when items are picked up or dropped – tend to have extra lines of tes3mp code that send a packet about what has taken place. Additionally, places where changes to the player happen that should not be checked every frame in a separate loop – such as inventory or spellbook changes as opposed to position or animation changes – also have accompanying code.
A boolean named canChangeCell has been added to MWWorld::LiveCellRefBase that determines whether an object is allowed to move to another cell in World::moveObject() – so as to prevent the client from doing momentum-based cell transitions that have not been approved by the server.
A boolean named sendPackets has been added to InterpreterContext that determines whether script value packets should be sent for the script attached to it. Script whitelists and blacklists are thus made possible.
Code making NPCs use random attacks, auto-equip items and report crimes has had additional conditions added to it to prevent if from affecting player-controlled NPCs.
Code related to combat and spellcasting has been modified to ensure that hit success and damage is not recalculated on every client.
In OpenMW, placing or spawning an object in the world gives it a reference number of 0. To allow players to send each other packets about newly placed or spawned objects, we have had to assign them reference numbers on the fly.
A relatively small number of methods have been added to existing classes whenever they have simplified aspects of tes3mp. Examples include LocalMapBase::updatePlayerMarkers(), MapWindow::updatePlayerMarkers(), MWMechanics::NpcStats::setLevelProgress(), MWMechanics::NpcStats::getSkillIncrease(), MWMechanics::NpcStats::setSkillIncrease(), World::saveDoorState(), CellRef::setRefNumIndex(), CellStore::getContainers() and CellStore::searchExact().
Our main goal right now is the implementation of NPC synchronization, traditionally the greatest challenge for fanmade multiplayer. Having considered possible approaches to it for a while, we are one burst of activity away from our first take on it, which will involve clients splitting the processing of NPCs among each other. We may move to serverside AI in the future, though doing so would make for a massive divergence from OpenMW code, as well as potentially requiring faster, more powerful servers, which is why we should only do it after careful deliberation.
Our secondary goal is making our server data get stored in an SQL database instead of the temporary text files we are using right now. Doing so will allow us to add timestamps to every change made in a cell since the start of the current server session, which in turn will allow us to send cell data to players on a more need-to-know basis.
The synchronization of quests, dialogue topics, faction membership, reputation and bounties should be much easier to do, but will only become relevant once NPCs are working.
Time and weather require divergence from OpenMW's code, which is why we have avoided doing them so far, though they should also be easy in practice.
One unique challenge
The greatest challenge for tes3mp will probably be the finding of a comprehensive solution for script synchronization. For certain scripts in the game, we can simply send a packet to other players whenever a short value in the script changes or a float gets set to a value without decimals, thus ensuring perfect sync for everyone else in that area with no packet spam. This works great for buttons and levers, like the ones at Ghostgate or in Sotha Sil, and is already implemented for them.
Unfortunately, other scripts in the game – such as banners flapping in the wind or objects floating on water – set values every other frame, and would lead to infinite packet spam if we applied the same system to them that works for others.
Going through all the scripts in the game and manually putting together a synchronization whitelist or a blacklist would probably work for Morrowind and its expansions, but would not provide a solution for all the plugins in existence or for any future projects based on OpenMW.
Some kind of script value spam detector may end up being the best solution, though false positives will be a constant danger.
Thank you for reading all of that. I'll be happy to answer any of your questions.