A request for support rebasing TES3MP against OpenMW

Everything having to do with OpenMW's TES3MP branch.
User avatar
GrimKriegor
Posts: 4
Joined: 17 Apr 2016, 00:14
Location: Portugal
Contact:

A request for support rebasing TES3MP against OpenMW

Post by GrimKriegor »

Gentlemen of the Vehklands, good night.

First of this is my first post in your forums, so allow me to take the opportunity to thank you all for making OpenMW a reality, I have been following this project closely for around 4 years with great interest, compiling it weekly, and even daily during those times the engine was mostly unplayable and every change would make me a step closer to being able to bully that stupid bosmer Fargoth.

Recently, as you are well aware, Koncord released the code for TES3MP, the multiplayer fork of OpenMW and many players have shown great, great interest in the project. I am one of those players and since there weren't many people working on it, I jumped at the opportunity to help out in any way I could. Unfortunately my skills are very basic and still don't go beyond documentation, scripting, bug reporting and packaging.

At the moment there are very few people working on the project and Koncord has been requesting assistance porting back the latest OpenMW. I believe it is very important to treat TES3MP as a fork to be able to merge the latest changes from upstream smoothly and I also believe it is a good idea to do it sooner than later, before things diverge too much and make the "merge" more difficult.

I have been following psi29a's advice in the other TES3MP post on how to accomplish this by adding the changes to the OpenMW 0.37.0 tag/commit, make a branch out of them, and then rebase them against master, but without knowledge of the inner workings of neither OpenMW or TES3MP I am afraid I don't have the competence to resolve conflicts.

That is why I am asking you guys for support. If you think this is important and have some free time, please give Koncord a hand in this.

Thank you for your time, and keep up the awesome work!
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: A request for support rebasing TES3MP against OpenMW

Post by scrawl »

believe it is a good idea to do it sooner than later, before things diverge too much and make the "merge" more difficult.
Right, I would merge more often than every 5 months. It can get exponentially more difficult the longer you wait.

What I would have done is a step by step approach. Start with a clean OpenMW master branch, then apply the TES3MP patches one by one, resolving conflicts as you go and compiling/testing every couple of commits.

Unfortunately, there seems to be no revision history for the multiplayer commits whatsoever, which makes things more difficult. Almost all changes are in the massive commit titled "Syncing inventory, animations, position, 8 key attributes". So you'll have to merge everything at the same time, resolve all conflicts, and pray that it works. Good luck.

Edit: maybe you should just go step-by-step from the OpenMW perspective. First rebase onto a version of OpenMW from February. Then March. And so on.
aesylwinn
Posts: 243
Joined: 14 Dec 2015, 20:30

Re: A request for support rebasing TES3MP against OpenMW

Post by aesylwinn »

There actually aren't that many merge conflicts (the only serious one is the Character::kill() function). I made an attempt at merging the other day and was moderately successful. However, some behavior that TES3MP relies on may have changed. From my brief amount of testing, it looks like the ai broke partially. Also, I wasn't able to properly test multiple clients. Previous to the merge and a commit or two from Koncord, it was difficult to get two clients to both show up on the other when on the same machine. I wasn't able to get that working at all this time, but I only made a couple of attempts.

If it is desired, then I can make a pull request with my merge, and Koncord or someone else can work on fixing whatever may have broken. However, I'm not sure that is the optimal way for TES3MP to continue. The version that Koncord started with is playable and is really only missing some bug fixes (some of which are related to providing a vanilla experience). If TES3MP continues down the path it has taken, then the code is going to keep on getting more and more different. Maintaining that is going to become progressively more difficult. Why bother? I think it would be best for TES3MP to work on implementing features and fixing bugs as they get noticed.

The major problem is that the multiplayer-client implementation isn't isolated from the single-player client implementation. This inherently makes it incompatible with OpenMW. Ideally, the multiplayer client would be moved to its own project folder.

Edit: The last bit was a suggested change. My coherency dropped a bit towards the end there.
User avatar
GrimKriegor
Posts: 4
Joined: 17 Apr 2016, 00:14
Location: Portugal
Contact:

Re: A request for support rebasing TES3MP against OpenMW

Post by GrimKriegor »

Gentlemen, thank you both for your feedback and expertise. I have forwarded the information to Koncord and the team.

Maybe I have overestimated the issue of diverting from OpenMW over time, and while I do understand that since the TES3MP specific code changes the single player client and makes it progressively more incompatible, I also feared of missing out the new features and support implemented on single player.

People seem to be loving the ability to play multiplayer and so am I, today we had a server filled with 12 people at one point and everyone was having a blast goofing around and testing the recently fixed character synchronization on exteriors. Thats one of the reasons why I was so concerned about the future of the project and diverting from OpenMW's development.

Anyway, you guys are obviously more than invited to come goof around on one of the future test sessions.

aesylwinn, thank you for testing, if its not too inconvenient please send a pull request to this repository, it will surely help Koncord decide on what to do in the future.
User avatar
psi29a
Posts: 5355
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: A request for support rebasing TES3MP against OpenMW

Post by psi29a »

Looks like Aesylwinn took the time to massage in the work here:
https://github.com/TES3MP/openmw-tes3mp/pull/1

Major thinks there Aesylwinn for taking the time to do that!


Did I misread about there being two clients? From a code management perspective, having two clients in one project will make development very difficult.

At some point in the future there must be an investment in time to make sure that there is but one client which can be used as single or multi-player. Better to do this now than later.
aesylwinn
Posts: 243
Joined: 14 Dec 2015, 20:30

Re: A request for support rebasing TES3MP against OpenMW

Post by aesylwinn »

You didn't misread. I think multiple clients are the way to go.

I'm not entirely sure whether the multiplayer is heading towards a coop or an mmo style of game-play. The amount of divergence is dependent on that to some degree. However, I do know that massive changes will be required to make a single player game work in a multiplayer environment, regardless.

This will include game mechanics. For example, how will sleeping/waiting be handled? Will you have to sleep before you can level up? I would expect this too be handled very differently in a multiplayer context.

More changes will have to be made to the quest system. Will there be a party system? Are you going to scale the difficulty based on the number of players?

To me, the two clients are incompatible. Coding a single client to do two jobs seems like an unnecessarily complication. Furthermore, I suspect it would slow development, and we may end up with something we don't like. Take a look at single player games that have added coop as an afterthought. It's generally a crap implementation because they don't want to break the single player experience.

That's not to say that the two projects cannot share code. I just think that any changes made for multiplayer should be placed in its own directory. Logic/physics are likely to diverge over time, but the rendering code should stay convergent. The multiplayer project can use the files from the single player client directory as needed, and perhaps even take advantage of polymorphism. Theoretically, that would make it easier to merge as well.
Chris
Posts: 1625
Joined: 04 Sep 2011, 08:33

Re: A request for support rebasing TES3MP against OpenMW

Post by Chris »

Ideally the engine should take on a client-server model. Even for single-player (and single-process) games, this can be beneficial since you're separating input from logic from display. Multiplayer obviously benefits because you can essentially just stick a network transport in between those components*, but there's other benefits too when it comes to things like parallelization (if input, logic, and display are already designed to run separately, they can be handled by multiple cores), and doing things like demo recordings by simply capturing the input and replaying it (although this will need more deterministic logic, which will always produce the same output for a given input).

* It can be a bit more involved than that, since you'll want client-side prediction to reduce the apparent latency. But it's still a start.

Multiplayer-specific things, such as handling quest statuses between multiple players, scripts, player references (who counts as 'player'?), world time during inventory/dialog/etc, etc, still need to be handled somehow. But that stuff can be separate from the actual networking.
aesylwinn
Posts: 243
Joined: 14 Dec 2015, 20:30

Re: A request for support rebasing TES3MP against OpenMW

Post by aesylwinn »

I think that's a great solution that addresses both of the concerns Psi29a and I had, assuming he's fine with two servers.:lol: Specifically, reworking everything to fit in a client-server model would allow the graphics code to stay independent of the logic, which is really what is needed. However, this solution would take a lot of work.

I'm interested in pursuing this. Is anyone else? This is looking like a large project, so it would be nice to collaborate.

Just to prevent potential misunderstandings, OpenMW isn't a "single-process" game. From what I understand, OSG is already rendering on a separate thread to some degree. Scrawl doubtlessly knows. :) I doubt there will be much if any performance gain.
davidcernat
Posts: 256
Joined: 19 Jul 2016, 01:02

Re: A request for support rebasing TES3MP against OpenMW

Post by davidcernat »

aesylwinn wrote:You didn't misread. I think multiple clients are the way to go.

I'm not entirely sure whether the multiplayer is heading towards a coop or an mmo style of game-play. The amount of divergence is dependent on that to some degree. However, I do know that massive changes will be required to make a single player game work in a multiplayer environment, regardless.

This will include game mechanics. For example, how will sleeping/waiting be handled? Will you have to sleep before you can level up? I would expect this too be handled very differently in a multiplayer context.

More changes will have to be made to the quest system. Will there be a party system? Are you going to scale the difficulty based on the number of players?

To me, the two clients are incompatible. Coding a single client to do two jobs seems like an unnecessarily complication. Furthermore, I suspect it would slow development, and we may end up with something we don't like. Take a look at single player games that have added coop as an afterthought. It's generally a crap implementation because they don't want to break the single player experience.

That's not to say that the two projects cannot share code. I just think that any changes made for multiplayer should be placed in its own directory. Logic/physics are likely to diverge over time, but the rendering code should stay convergent. The multiplayer project can use the files from the single player client directory as needed, and perhaps even take advantage of polymorphism. Theoretically, that would make it easier to merge as well.
First off, thank you for the merge. With your help, we can now update to the latest version of OpenMW almost immediately.

Concerning the creation of such a separate client... Like Grim Kriegor, I'm a new addition to the team of what was until now Koncord's one-man project, but here's the way I see things.

As far as its C++ code goes, TES3MP's client should be a completely straight port of OpenMW's singleplayer one. That means that any changes to the gameplay should be absolutely minimal and thus allow for a singleplayer experience in TES3MP that is mostly identical to the one in OpenMW.

So how do we make that work for multiplayer? How do we make it playable and fun? How do we deal with sleeping, waiting, quests, parties, increased difficulty based on player count?

Server-side code allowing for flexible settings and scripts, exclusively. We don't tell server owners that they need to have an MMO experience or a coop one. We don't impose anything on them, but we give them the tools to make their own choices.

A few hypothetical examples of server configuration settings and their effects follow...

1) Sleeping

We can have a "gametime_sync" option in the server configuration file.

When it is false, every player can wait or sleep for as long as they want without affecting other players. For some players it will be day, for others it will be night, but they can still play together. If a player's sleep is interrupted by a monster, so be it. Simply have a monster spawn next to them which is visible to the other players as well.

When it is true, the game time is always the same for all players, and players must all simultaneously sleep (or – alternatively in our case – wait) like on Minecraft servers. As a result, it either becomes morning (like in Minecraft), or a default amount of time passes, or the lowest amount of time that the players can all agree on passes. Perhaps we can have a server option for that as well, or simply implement the simplest solution, but it would be almost entirely serverside code.

2) Quests

Taken care of by a "quests_handling" option.

When its value is "shared", all the players have the same quest states at all times. In other words, we are all doing the exact same Mages Guild quest, and if we become the Arch-Mage, each one of us is interchangeably treated by the NPCs as the Arch-Mage. As far as the game's NPCs are concerned, all players have the exact same identity. To prevent desyncs (potentially caused by different players talking to the same NPC at the same time), allow only one player to talk to NPCs at a time, while displaying the content of their conversation to the others.

When its value is "local", the players have entirely unsynchronized quests... and can easily and immediately block each other from advancing in them by invalidating their objectives or killing their quest givers. This would make for short-lived chaotic servers full of broken quests, but would be interesting in its own way.

When its value is "singleplayer", there is a party leader who is the only person who can ever communicate with NPCs. All the other players are relegated to the role of mute henchmen, and are treated by the game's NPCs simply as other NPCs.

3) Parties

I don't think we should allow for more than one party at a time as far as quests go, as it would be terribly complicated to make anything like that work, but we can have different parties on PvP servers.

4) Balance changes

Handled by Lua scripts (already fully supported and embraced by TES3MP) and/or a list of mandatory client mods for each server (with a checksum checker ensuring players have the same ones).


TES3MP can live as its own project separate from OpenMW, but I prefer that it easily follow OpenMW's progress by having as few changes as possible on the client side – mostly related to handling events, sending and receiving network packets with world information, and multiplayer-specific GUI elements – while having all major gameplay divergences on the server side, where they should be optional and customizable.

That way all improvements to OpenMW effortlessly become improvements to TES3MP as well, while simultaneously allowing OpenMW to absorb TES3MP officially in the future if desired.
aesylwinn
Posts: 243
Joined: 14 Dec 2015, 20:30

Re: A request for support rebasing TES3MP against OpenMW

Post by aesylwinn »

Glad I could help. :)

Your hypothetical examples are quite interesting. I think that's a good way to approach the problem for now. Occasionally conflicts will pop up, but they should be easier to resolve if you aren't drastically changing the structure of the program. Ultimately, I think Chris's suggestion to move to a client-server model is what's needed to make it possible to join the two projects. Hopefully, it wouldn't be too difficult to port changes made by tes3mp. I plan to work on that after finishing up my current projects, unless someone has an objection.

2) I like the option to allow everyone to see what is being said. Recently I've been playing Grim Dawn, and it has always bugged me that you can't see what the npcs are saying when talking to a different player, even when in the same party. It would be nice if the developers of that game added a similar option. :) On another note, theoretically you could synchronize the quest status on the server, but the logic would be complicated and a version of the game would probably have to be running on the server.

3) You could potentially treat each party as a "player," but yes, it would complicate matters with your approach.
Post Reply