OpenMW's Roadmap and Future

Anything related to PR, release planning and any other non-technical idea how to move the project forward should be discussed here.
User avatar
psi29a
Posts: 5357
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

OpenMW's Roadmap and Future

Post by psi29a »

General statement about feature development:

Perfect is the enemy of good. - Voltaire

We should nail down a feature's core as soon as possible without sacrificing OpenMW's stability. OpenMW must not crash. We can throw an error, let a user know what is going on, but we must not allow OpenMW to CTD with no reason. In this way we can gain features going forward and leave further polishing, extension and optimisation for later. It doesn't have to be perfect, it just has to work reasonably well. We can then move foreword at a steady rate without sacrificing stability and leaving the door open for other developers to build upon the work. Effort should be made for maintainability first and following, wear possible, the general architecture guidelines.

We should also be more willing to be accepting of drive-by commits and features. Some people just have an itch they want to scratch and never come back. This should not be a motivation to deny/reject something. Instead we should engage the developer and again, work them to get a good core and encourage further additional MRs they can work on. Not everyone has years of experience and OpenMW shouldn't take an elitist approach to code contribution. OpenMW is built upon the fact that people come and go, so we shouldn't fear that there won't be enough manpower to sustain the project. No one is an expert in all parts of OpenMW and the project has existed in such states, such as areas of code that hasn't been touched in years. We should encourage people to become experts in the areas they are working on and help train up people who wish to help.

When it comes to testing a feature, sometimes we just have to make the call to bring in a feature into master and then let those who use nightlies use our issue tracker for what it is meant for. We should also add an additional label "nightly" to also indicate that the issue is within our main development branch and not a bug from a stable release. It also indicates that it doesn't need a changelog entry. This also implies that our nightly users are our testers. By encouraging this behaviour, we'll track down issues faster and polish new features off faster than had we them just sitting around and rot. It also means that we can roll out releases faster.

Code Quality:
A topic that pops up from time to time. As subjective as this can some times be, we should try to make it easy by providing a clang-format file for editors that allow for consistency for developers. We can encourage its use by newbies but we should reject contributions purely because of style. Most people come to the project wanting to be apart of it and are willing to change things if guided nicely. We can revisit our code-style guidelines in the wiki and update them to. That being said, we shouldn't be afraid to take a PR/MR over and do additional follow-up work to polish it off if the original creator is incapable of it. We should also keep them attributed by keeping their initial commits.

Transparency:
I feel like we need to get away from discussions behind a closed door, discussions about OpenMW should be done in the public eye. OpenMW is not a closed cabel. Technical discussions should happen in the public forum or #development.

Mediation and interpersonal relationships withing OpenMW:
It's also come to my attention that there are those people who don't get along in the project. I believe that everyone means well here, this is a passion project after all and people can have some rather strong opinions. That even the "we will just have to agree to disagree" doesn't just stop the conversation, it just pauses things until the next disagreement comes up. I would like to believe that we are all pulling on the same rope here but perhaps there are other motivations at hand here. I'm interested in hearing from you how we could resolve this? It is something I will think further about but I encourage everyone working on OpenMW to either talk to me in private or just talk about the issues that are bothering them in the forum here for the team to discuss.

Eventual 1.0 release and OpenMW-CS version:
Currently both OpenMW and Editor are 'tied' together. Original plan was that we delay 1.0 in order to have the editor catch up; that isn't going to be the case. So for now, this is alright, when we get to 1.0 then we need a separate way to track versions. Suggestions:
* bump OpenMW major number to 1, leave minor number to 50 (for example): version 1.50 as a result with OpenMW-CS as 0.50
* have different versions for major 'apps': thus OpenMW becomes 1.0 and OpenMW-CS either stays at 0.50 or gets reset to 0.1 (considering the state of the editor).


Road to 1.0 (the blockers):
* 1.0 issues: https://gitlab.com/OpenMW/openmw/-/issu ... name[]=1.0
* MCP: https://wiki.openmw.org/index.php?title=MCP

So while MGE:XE, MCP and MWSE are 3rd party libraries that extend Morrowind.exe; OpenMW can at least provide parity with them to the extent that we can say that they are unnecessary as OpenMW already has or extends the features they provide. I think we should evaluate what is to be considered a blocker for 1.0 and which is for post 1.0

For consideration as 1.0 or even 1.1:
* MGEXE: https://wiki.openmw.org/index.php?title=MGE_XE

* MWSE-mwscript: https://wiki.openmw.org/index.php?title=MWSE
^-- This is open for debate as to what we want to do. I get the feeling that modders have abandoned this anyway. So it's just a wasted effort and we should avoid 'extending' mwscript and let it just die.

Not for consideration for 1.0 but very well can come later:
* MWSE-lua compatiblity is not a requirement for 1.0

The problem here is that this is a moving target and I don't believe OpenMW should be the one to 'track' this. It is entirely possible here that if MWSE-lua implements new features that they should also commit supporting code in OpenMW for that feature. Should they need a specific feature that isn't provided in the API, they can create a ticket to discuss with OpenMW to see what the best way forward is then they can submit a MR for the feature. This should further integrate MWSE-lua devs into OpenMW development there by creating and strengthening the bond between the two projects.

Bounties:
There have been bounties set on features by people outside the project, OpenMW has no problem with this and if anyone feels insentivised to claim a bounty they are free to do so here: https://www.bountysource.com/trackers/1 ... nmw-openmw
We will mark issues on our tracker as those with bounties available and we encourage people who would also like to support the project monetarily, they are still free to support the core developers (FAQ entry) and of course post bounties on issues that are important to them. Keep in mind that code acceptance into the repo is a requirement for the bounty and that we will treat the MR or PR with the same vigour and quality as any other. Bounties should also only but put on issues on our tracker, OpenMW reserves the right to deny inclusion of code that is of bad quality or with no related issue in our tracker.

Roadmap:
The goal here is to 'guide' development, but again no one is forcing anyone to work on things they don't want to. If someone has an itch, like working purely on making on Morrowind support, they are free to do so. Our job, as the core OpenMW developers is to guide them.

Short Term:
0.47 targets:
* Working through existing 1.0 related issues
* Polishing up Object Paging and Active Grid
* Landing double precision bullet support
* Landing movement solver
* Landing threaded physics
* Further work bettering NIF support

0.48:
* Collada support either native or via osgAnimation (perhaps worked on in 0.47 but won't be ready until 0.48)
* Basic Lua subsystem (perhaps worked on in 0.47, but won't' be ready until 0.48)
* Working through existing 1.0 related issues

Medium Term:
* Rally around remaining 1.0 issues; anything that blocks. It is to be assumed that more and more issues will come up and we'll have to triage them as we go as blockers or not.
* Improved lighting system (made more noticeable by active grid)
* Version 1 Definition of our Lua API, we should keep standard high here in terms of documentation

Code: Select all

    * Single player game running seamlessly in Client/Server model
    * Version 1 Transpiler to take mwscript and generate lua-based equivalent
* Support for Oblivion NIF
* Support for Oblivion ESM
* Active Paging on by default when Object Paging is turned on

Long Term:
* Support for Skyrim and FO3 NIF
* Support for Skyrim and FO3 ESM
* Support for Havok data found in NIF
* Support for Facegen
* Support for Speedtree
User avatar
psi29a
Posts: 5357
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: OpenMW's Roadmap and Future

Post by psi29a »

I'll update the text above as we discuss it further here. I figured it was worth posting now instead of letting it sit any further into the 0.47 dev cycle.

This only works if everyone is onboard, so if there are things that you don't agree with, voice them here. That way we have something to guide OpenMW to 1.0 and beyond.

I'm not apposed to adding more things, taking stuff away or modifying the order of how things should be done if it makes sense to do so. There are additional topics I haven't yet had time to think about but we should address them anyway.

Are there things people are working on behind the scenes that they would like to see landed in 0.47 or 0.48?
User avatar
akortunov
Posts: 899
Joined: 13 Mar 2017, 13:49
Location: Samara, Russian Federation

Re: OpenMW's Roadmap and Future

Post by akortunov »

psi29a wrote: 12 Jul 2020, 22:15 * MWSE-mwscript: https://wiki.openmw.org/index.php?title=MWSE
^-- This is open for debate as to what we want to do. I get the feeling that modders have abandoned this anyway. So it's just a wasted effort and we should avoid 'extending' mwscript and let it just die.
There was a long related discussion before when we decided which scripting language to use after 1.0. The conclusion was that further development of ancient MWScript which is not used outside of Morrowind is just a waste of time, and it should be used only as a fallback for aseets, created for original, not modified Morrowind's engine. We decided that we should focus on a proper scripting implementation based on popular existing language (Lua, for example), and all new content should use it as a main scripting language.
All popular MWSE mods either have Lua clones now or will have them in the foreseen future, so yes, just let an old MWSE die. If anyone wants to add its support, he can create his own fork, but we should not merge it upstream and spend a time to recreate all MWSE 1.x bugs to make outdated MWSE mods more-or-less working.
psi29a wrote: 12 Jul 2020, 22:15 We should also add an additional label "nightly" to also indicate that the issue is within our main development branch and not a bug from a stable release. It also indicates that it doesn't need a changelog entry
We already use a "regression" tag for it.
psi29a wrote: 12 Jul 2020, 22:15 We should also be more willing to be accepting of drive-by commits and features. Some people just have an itch they want to scratch and never come back. This should not be a motivation to deny/reject something.
A quite strange approach. If developer is not really interested in OpenMW and he just wants to do something fun (to implement a dwemer computer with MS DOS, for example), there is no point to merge his work - he will not contribute further anyway and will not bring OpenMW closer to 1.0. From the other hand, if developer's motivation is improve OpenMW instead of scratching itches, closing his work which does not fit project goals, roadmap or quality standants will not harm it. For example, I did not stop to contribute when Zini declined to merge my reverse pickpocketing and extractable arrows. I'd suggest you to investigate why team members pay long attention to the project instead of leaving it.

Otherwise you can come to the feature creep, when you try to replace features quality by their quantity. It can harm project on a long distance pretty bad.

Also if the plain is to have a fully working Lua subsystem within the year, I'd suggest to do not hurry to merge a lot of optional game formulas tweaks or bulk new mechanics which add tons of new features (e.g. head bobbing), since it will just require additional efforts to de-hardcode them and probably it will be faster to write them from scratch.
psi29a wrote: 12 Jul 2020, 22:15 Medium Term (rest of 2020, first half of 2021):
* Version 1 Definition of our Lua API, we should keep standard high here in terms of documentation
* Single player game running seamlessly in Client/Server model
* Version 1 Transpiler to take mwscript and generate lua-based equivalent
May be a bit too optimistic since we still do not have a design for the new scripting system (I ask about this design for two years), but you already expect an implementation soon.
For example, it is unclear how you are going to handle scene graph, custom input bindings, IO and other client-specific things on the server.
And it is unclear what to do if we fail to implement it to defined deadline.
User avatar
Capostrophic
Posts: 794
Joined: 22 Feb 2016, 20:32

Re: OpenMW's Roadmap and Future

Post by Capostrophic »

I can't agree. The idea that was abandoned was the idea of making our own MWSE, extending and improving mwscript to fit the future goals independently from MWSE. We won't lose anything by trying to support all past mods in the viable scope: e.g. trying to support MWSE-Lua bindings directly clearly at least somewhat conflicts with adding in TES3MP's scripting system, which instead should get a compatibility layer for MWSE but that's completely optional. Meanwhile the mwscript scripting system that could get compatibility with mwscript component of MWSE is already there.

We're not rejecting the idea of supporting shittily written mods from 2002-2005 just because they're old and shittily written. Morrowind supported them, and openmw must always be compatible with Morrowind mods using Morrowind content. Forever. It's not a stretch to assume mods made with MWSE-mwscript in mind should have some degree of support too -- Morrowind, with some hacks, did support them! That's not feature bloat nor does it harm openmw's moddability. That's still maintainable, is in the scope of openmw, doesn't conflict with any openmw-exclusive modding improvements, doesn't make mwscript un-deprecated. It's obviously not required to support it but there's no reason to reject a well-made implementation of some of MWSE-mwscript functionality.

I'll go further and say that if in time there'll be an Oblivion or a Papyrus subset of the scripting system then OBSE, NVSE, SKSE etc. obscript or Papyrus-side instructions and improvements should be a thing too.
User avatar
akortunov
Posts: 899
Joined: 13 Mar 2017, 13:49
Location: Samara, Russian Federation

Re: OpenMW's Roadmap and Future

Post by akortunov »

Capostrophic wrote: 13 Jul 2020, 08:25 We're not rejecting the idea of supporting shittily written mods from 2002-2005 just because they're old and shittily written.
Only because they are written for the unmodified engine ("Morrowind supports them"). We do not support, for example, MWE, Talky Morrowind or unoffical translations which modify Morrowind executable, and an old MWSE is not really different here. 3d-party patches != Morrowind.
Capostrophic wrote: 13 Jul 2020, 08:25 I can't agree. The idea that was abandoned was the idea of making our own MWSE
There is no real difference between making our own MWSE and allowing someone else to make his own MWSE and then spending a lot of time to add and maintain our own implementation for functions from that MWSE. MWSE-Lua is a completely different thing, I talk about an old and shitty MWScript-based MWSE, which was abandoned even by its own developers in favour of more advanced MWSE-Lua.
Capostrophic wrote: 13 Jul 2020, 08:25 We won't lose anything by trying to support all past mods in the viable scope
We will lose an irretrievable resource - time. How many manhours we spend to maintain MWScript now, and how many additional manhours we will need to spend to implement a bug-to-bug-compatible MWSE layer to make most of outdated MWSE mods to work properly? These manhours can be spent, for example, to enhance Lua subsystem in OpenMW or to make an MWSE-Lua -> OpenMW-Lua transtator.
User avatar
akortunov
Posts: 899
Joined: 13 Mar 2017, 13:49
Location: Samara, Russian Federation

Re: OpenMW's Roadmap and Future

Post by akortunov »

psi29a wrote: 12 Jul 2020, 22:15 Instead we should engage the developer and again, work them to get a good core and encourage further additional MRs they can work on. Not everyone has years of experience and OpenMW shouldn't take an elitist approach to code contribution.
I suppose that you want to encourage people to lean OpenMW internals, enhance their coding skills and write a better code, so OpenMW can become better on the long distances, but you may get an opposite result with such approach.

Did you decide to use such approach after bzzt drama, right? But this drama is his fault, not ours:
1. He does not accept any critics
2. He does not like to explain what his code does
3. He sometimes uses very strange or hackish decisions or optimizations, which work well only on his setup
As a result, he likes to write huge obfuscated merge requests and hide hacks and bug in the working code, and it takes a lot of efforts and time to review them properly. For example, I spent several months to get rid of bugs in his "distant terrain optimizations" MR which really affected the whole rendering system.

Your new "merge as it is and hope that not too many things are broken" approach does not motivate him to behave better - you will just get more bugs and bzzt-style behaviour and code. Also keep in mind that testing does not prove that code does not have errors - they may exist in the engine for years before tracking them down.
User avatar
psi29a
Posts: 5357
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: OpenMW's Roadmap and Future

Post by psi29a »

akortunov wrote: 13 Jul 2020, 08:44 3d-party patches != Morrowind.
This indicates that you're a Morrowind purist, which is fine but OpenMW is also not Morrowind. OpenMW is an engine that can play Morrowind, so you could frame it that it is a 3rd-party patch as well. OpenMW is not Morrowind. So while we are hands off when it comes to content, we do have a say as to how things run, including what things to include and what not. We've implemented some MGE:XE and some MCP where it makes sense to do so. We also make a point about the original engine's design, bugs and oversights that have an impact on gameplay.
akortunov wrote: 13 Jul 2020, 08:44 We will lose an irretrievable resource - time. How many manhours we spend to maintain MWScript now, and how many additional manhours we will need to spend to implement a bug-to-bug-compatible MWSE layer to make most of outdated MWSE mods to work properly? These manhours can be spent, for example, to enhance Lua subsystem in OpenMW or to make an MWSE-Lua -> OpenMW-Lua transtator.
This is just pure nonsense and you're treating developers as if they are paid for resources working at a company, they are not. People are going to work on things they want to work on in their free time. If someone wants to take the time to work on something they want, they are free to do so. Just as you are free to work on whatever it is you want to work on. There is a very real chance though that it will rejected though as they do not align to the project goals.

That happened with Scott in 2015 added Python and Lua support to OpenMW. This was a drive-by request that was rejected and that's fine. It gave a proof-of-concept and triggered discussion that eventually led us some years later to ditch mwscript+ and move on to Lua.

No one is asking you, akortunov, to work on anything. No one is asking you to clean up after someone or do work you don't want to do. If you don't like, then don't do it. You'll just end up frustrated. That is just how Open Source works and I've been at it for a long time on many different projects. Think of FOSS developers as cats... (cat herding). Saying to a cat they should work on this, this and that and they will either scratch your eyes out or show you their ass. Contributions though, whether asked for or not are looked at individually... does it close an issue and work alright?

We're all on this project because we believe in OpenMW and have its best interest at heart.
User avatar
akortunov
Posts: 899
Joined: 13 Mar 2017, 13:49
Location: Samara, Russian Federation

Re: OpenMW's Roadmap and Future

Post by akortunov »

psi29a wrote: 13 Jul 2020, 10:52 This is just pure nonsense and you're treating developers as if they are paid for resources working at a company, they are not.
No, I just remeber that the year still has 365/366 days and day still have 24 hours indifferently from if the project is a proprietary or opensource one. Your assumption that we have an unlimited amount of time is OK if you expect 1.0 in 2090, but it starts to fail once you start to set deadlines. When you set a deadline, it means than you calculate how much manhours tasks from your deadline require and how many manpower your team has.
psi29a wrote: 13 Jul 2020, 10:52 People are going to work on things they want to work on in their free time. If someone wants to take the time to work on something they want, they are free to do so.
In this case your roadmap is useless at all - feature X is done when it is done. It can happen in 2021, it can happen in 2030, it can never happen.
psi29a wrote: 13 Jul 2020, 10:52 Just as you are free to work on whatever it is you want to work on.
Yes, but there is a work which I'd like to do and work which needs to be done. Personally I spent a huge amount of time to solve issues which did not affect my setup, because these bugs needed to be fixed, and there were no other persons which were going to fix them.

So from what I can tell, you treat opensource projects as paid ones, but with unlimited resources. From the one hand, you try to use termins from paid projects (you try to set goals and estimated time to achieve that goals), from the other hand you say that project development is fully unpredictable and there is no point to set priorities, goals and deadlines, because no one is going to respect them. It is weird.

If you really want 1.0 in the next year (IIRC, you wanted it in the 2020), set strict priorities to achieve this goal, otherwise you just will move deadlines again and again without any significant progress. If you try to protect everything, you protect nothing.
User avatar
psi29a
Posts: 5357
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: OpenMW's Roadmap and Future

Post by psi29a »

akortunov wrote: 13 Jul 2020, 10:33 Did you decide to use such approach after bzzt drama, right? But this drama is his fault, not ours:

..

Your new "merge as it is and hope that not too many things are broken" approach does not motivate him to behave better - you will just get more bugs and bzzt-style behaviour and code. Also keep in mind that testing does not prove that code does not have errors - they may exist in the engine for years before tracking them down.
While bzzt was difficult to work with he has toned things down and I've been managing things with him just fine and there is support from other developers as well. His contributions didn't just come out of the blue, just as AON3 was tutored by Scrawl, so was bzzt. So I consider those contributions in a different light than from some rando from the Internet who comes along with a huge contribution. Take for example Scott, he also made a huge contribution that got rejected and is a friend of mine. The two are not to be compared.

To be honest, the only drama that seems to come up is of your own making, akrotunov. You're hypercritical and come across brash and arrogant which doesn't help anything at all. You bait people into discussions, ignore critique and constructive criticism and then ignore others when other point out mistakes. It is as if only your opinion matters and that everyone else is wrong. Not to mention that you bad mouth developers behind their back on other servers.

This is toxic behaviour.

Poor Capo is trying to hold the peace and he shouldn't be made to be a go-between people with issues with each other. If there are issues, this is a good enough place to discuss them.

We're all human, we all make mistakes. We're all in this to make OpenMW better, but I won't tolerate toxic behaviour in the project, especially from someone who should know better and has worked so long and hard on OpenMW as you have.
User avatar
psi29a
Posts: 5357
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: OpenMW's Roadmap and Future

Post by psi29a »

That being said, you do and have helped others. Sometimes you do very well and I'm proud of you. So it is with very mixed feelings that I post this.
Post Reply