Using version numbers on the bugtracker

Anything related to PR, release planning and any other non-technical idea how to move the project forward should be discussed here.
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Using version numbers on the bugtracker

Post by scrawl »

So, I try to ignore this topic as much as I can but it keeps waving in my face in one way or another so I guess we have to talk about it. I think we had one or two previous threads on this but they didn't go anywhere so I didn't bother digging them up.

The issue is that we have no guidelines for what the 'version number' for an issue on the bug tracker means, and different people seem to have different opinions on this, everyone does what they wants and the bugtracker becomes a mess. A common opinion I see is:
...I can see why people forgot about it; it was never assigned a version number and had no visibility).
IMO, this reasoning is not great. We don't 'forget' issues because they don't have a version number. If we do forget, it's because there are 560 open issues currently, with or without version number. As a developer (speaking for myself anyway) I don't even look at the current version's roadmap, because its so cluttered and useless.

In my opinion, version numbers should be used for release planning, not for 'this issue is important'. A better way to give visibility is to 'Confirm' issues, set a Category and Priority, and/or add more information that is relevant.

Assuming that we actually want to use version numbers as a 'future roadmap' for the next version (which I'm not sure if we should, as hardly anything is planned), we should only set a version number for an issue when we are reasonably certain what needs to be done and how to do it, so that anyone can do so. Otherwise the issue is just going to carry on from one release to the next indefinitely and what good is 'visibility' then?

Also, discuss: would it be a good idea to disable version number changes for non-developer accounts?
JohnMaster
Posts: 38
Joined: 09 Aug 2017, 05:32

Re: Using version numbers on the bugtracker

Post by JohnMaster »

As a non-developer, I have read the previous posts (some links below) and scrawl has made his case quite clear a lot of times and Zinni backed this back in 2014 by supporting not changing a system that is working. For less clutter and for the sake of scrawl, read his previous posts.

A target version should only be set if the issue has been assigned. The exception for issues without an assignee should be approved by a dev, e.g. scrawl and Zinni. These are usually issues for a 1.0 release.

Disable target version for non-developer accounts.

Suggestion for the bug tracker (27 May 2014)
Bugs with no target version (13 May 2015)
Concerning «the forgotten bugs» (27 Nov 2016)
When should bug tracker issues be tagged with a version? (01 Aug 2017)
VSync can cause graphical glitches (11 Aug 2017)
User avatar
DestinedToDie
Posts: 1181
Joined: 29 Jun 2015, 09:08

Re: Using version numbers on the bugtracker

Post by DestinedToDie »

Guilty as charged, just didn't know earlier that it was better to leave my submits without version number.
User avatar
Thunderforge
Posts: 503
Joined: 06 Jun 2017, 05:57

Re: Using version numbers on the bugtracker

Post by Thunderforge »

I'm a pretty new developer to this project and the version numbering schemes are definitely something I've found confusing. I even created a forum topic about a month ago asking about it, but only got one response (not sure if they were a developer or not).

If this has been discussed before, then I think we need to put this information on the wiki just like we do for the issue statuses. I don't think it's fair to expect new developers to have to dig through the forums to find year and a half old forum posts.
scrawl wrote: 10 Sep 2017, 16:01 As a developer (speaking for myself anyway) I don't even look at the current version's roadmap, because its so cluttered and useless.

In my opinion, version numbers should be used for release planning, not for 'this issue is important'. A better way to give visibility is to 'Confirm' issues, set a Category and Priority, and/or add more information that is relevant.
See, I have the opposite problem. We currently have 556 issues that are open. Of those, 463 (83%) are Normal priority, 10 (1.8%) are High, and none are Urgent or Critical. If I don't want to, or am not able to, work on those 10 High priority bugs, how do I know which Normal priority ones I should work on? The way I do it is by looking at the Roadmap and seeing which ones are prioritized for the 0.43 release (and I currently have 61 to choose from, plus another 35 in 1.0).
scrawl wrote: 10 Sep 2017, 16:01 Assuming that we actually want to use version numbers as a 'future roadmap' for the next version (which I'm not sure if we should, as hardly anything is planned)
That may be the problem: hardly anything is planned. To give an example, I keep hearing about how things need to be done before 1.0. But I really don't have a good idea of when we will actually be ready for 1.0. To do that, we would have to plan out what bug fixes and features need to be implemented. I figure the best place to communicate that would be the Roadmap.

Similarly, how will we know that we are ready for a 0.43 release (aside from our video editor being available)? Are there certain bugs that must be fixed before then? Are there certain features we are hoping to have completed? Right now, I have no idea.

For me personally, what I want to know is if a particular issue
  • Should be completed as soon as possible (as in, the next release)
  • Should be completed any time before 1.0
  • Cannot be completed until after 1.0
scrawl wrote: 10 Sep 2017, 16:01 we should only set a version number for an issue when we are reasonably certain what needs to be done and how to do it, so that anyone can do so. Otherwise the issue is just going to carry on from one release to the next indefinitely and what good is 'visibility' then?
Perhaps the issue then is that we have too many issues where we don't know how to fix it, or how to fix it has not been communicated. Right now, only 271 (49%) of issues have a version, and it sounds like we are wanting to remove a version from more. Does that mean that over half of the issues are ones that nobody knows how to fix?
scrawl wrote: 10 Sep 2017, 16:01 Also, discuss: would it be a good idea to disable version number changes for non-developer accounts?
I'm not necessarily opposed to non-developers judge priority, but I don't know that someone who is posting their first issue report knows enough to say which version it should be targeted for.

Given that it's a process already to get a developer account (one that I think is too much of a process, but that's a different issue), I would be in favor of doing this if only because it's given to people who have been around for a while. I would not be opposed to non-developers getting developer accounts to prioritize things as well on a case-by-case basis (basically, if they were in a "business analyst" role for the project).
JohnMaster wrote: 10 Sep 2017, 17:12 A target version should only be set if the issue has been assigned. The exception for issues without an assignee should be approved by a dev, e.g. scrawl and Zinni. These are usually issues for a 1.0 release.
I am against only assigning versions to In Progress or Completed issues, again because there is no planning. How do I communicate that something is a post-1.0 feature with Low, Normal, High, Urgent, Immediate as my options for priority? Assigning it to "openmw-future" is a great way to do that. And similarly, I want to distinguish those issues from the ones that need to be done before 1.0, or that must be done before the next release.
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: Using version numbers on the bugtracker

Post by scrawl »

If this has been discussed before, then I think we need to put this information on the wiki just like we do for the issue statuses. I don't think it's fair to expect new developers to have to dig through the forums to find year and a half old forum posts.
Yes to the wiki, but probably not on the Bug reporting guidelines page (since that's targeted as users and we seem to be reaching an agreement towards users not being able to set versions anymore)
how do I know which Normal priority ones I should work on?
You generally don't, because this is an extremely hard question, not to mention depending on your area of expertise. The 'easy' stuff will naturally get picked up first, leaving mostly tasks that require a lot of knowledge of the codebase, or ones where no one knows what the hell needs to be done anyway. For someone who's looking for things to do, it might be easiest to follow recent additions/discussions to find something new to do before others find it.

Maybe we need to define 'work'. Is that just coding or also research, triaging, discussion, etc.? Should your proposed roadmap be based on just available coding work or also the other types of work? Because if we include both, then the roadmap is just... pretty much every single issue, right? But if we only include coding, the roadmap is just that tiny fraction of issues that are 'ready to code' and they tend to get picked up quickly anyway.
To give an example, I keep hearing about how things need to be done before 1.0. But I really don't have a good idea of when we will actually be ready for 1.0.
That's what the 'openmw-1.0' milestone is for. It just has that tiny little problem that it always keeps growing. I suspect that's in part due to not much consensus on what 1.0 should represent, and another part of 'every little user being able to change our roadmap'
Similarly, how will we know that we are ready for a 0.43 release (aside from our video editor being available)? Are there certain bugs that must be fixed before then? Are there certain features we are hoping to have completed? Right now, I have no idea.
We release after a certain period of time (2-4 months), more or less regardless of what has been added.
That may be the problem: hardly anything is planned.
Maybe - but it's also to be expected, with everyone being here on their free time, contributors coming and going all the time etc. Planning is futile when we can't bring the plan to action.
For me personally, what I want to know is if a particular issue

Should be completed as soon as possible (as in, the next release)
Should be completed any time before 1.0
Cannot be completed until after 1.0
Problem solved:
3) is openmw-future
2) is openmw-1.0
1) is anything else.
For more fine grained status, we have 'Priority' on top of that.
I would not be opposed to non-developers getting developer accounts to prioritize things as well on a case-by-case basis (basically, if they were in a "business analyst" role for the project).
Me neither. We already have one such person. (Thanks to you know you who are!)
Perhaps the issue then is that we have too many issues
brb, reporting a new issue
where we don't know how to fix it, or how to fix it has not been communicated.
Often, figuring out how to fix the issue is already 90% of the work... so once that's done, the solution tends to be applied quickly (often by the same person) and the issue is closed. That means the remaining issues that haven't been closed yet will appear to be something that no one knows how to do... until someone comes along and just does it.
I am against only assigning versions to In Progress or Completed issues, again because there is no planning. How do I communicate that something is a post-1.0 feature with Low, Normal, High, Urgent, Immediate as my options for priority? Assigning it to "openmw-future" is a great way to do that. And similarly, I want to distinguish those issues from the ones that need to be done before 1.0, or that must be done before the next release.
I think it was implied that openmw-1.0 and openmw-future keep functioning as they are, and the 'only in-progress/completed' rule applies to the next release only.
Chris
Posts: 1625
Joined: 04 Sep 2011, 08:33

Re: Using version numbers on the bugtracker

Post by Chris »

scrawl wrote: 10 Sep 2017, 21:44
For me personally, what I want to know is if a particular issue

Should be completed as soon as possible (as in, the next release)
Should be completed any time before 1.0
Cannot be completed until after 1.0
Problem solved:
3) is openmw-future
2) is openmw-1.0
1) is anything else.
For more fine grained status, we have 'Priority' on top of that.
I always figured openmw-future would be for post-1.0 stuff, and openmw-1.0 is for things that need to be done for 1.0. openmw-future makes it sound like it's a far-off (future) consideration, rather than ASAP.
User avatar
Zini
Posts: 5538
Joined: 06 Aug 2011, 15:16

Re: Using version numbers on the bugtracker

Post by Zini »

In my opinion, version numbers should be used for release planning, not for 'this issue is important'. A better way to give visibility is to 'Confirm' issues, set a Category and Priority, and/or add more information that is relevant.
Makes sense. But we would have to define more clearly the meaning of the individual priority options. And maybe the severity field, too?
Also, discuss: would it be a good idea to disable version number changes for non-developer accounts?
I support this idea.
That's what the 'openmw-1.0' milestone is for. It just has that tiny little problem that it always keeps growing. I suspect that's in part due to not much consensus on what 1.0 should represent, and another part of 'every little user being able to change our roadmap'
I think we have a pretty clear idea of what 1.0 is. Its just that the remaining tasks are really hard and most developers don't have the time or the skill to tackle them and so people get unruly and sneak other things into the roadmap to fill their development time.
Should be completed as soon as possible (as in, the next release)
Should be completed any time before 1.0
Cannot be completed until after 1.0
3) is openmw-future
2) is openmw-1.0
1) is anything else.
I do not fully agree with this. First openmw-future is for things that are not required for 1.0. Doesn't always mean that we can't do them (see above, unruly developers).
Completed as soon as possible is also a bit too vague, unless we specifically mean blockers.

I think a good part of the problem is the hard-1.0-tasks-that-no-one-wants-to-tackle issue. I was hoping we could sit this out. 1.0 will come around eventually and then the problem will go away on its own.
But if people really dislike it so much, maybe we should consider a openmw-1.0-optional milestone that would collect all non-essential issues from openmw-1.0. Note that this would not include any bug reports. We should at least attempt to release 1.0 bug free (if 1.0 is ready otherwise and there are a few insignificant bugs left that won't be fixed any time soon, we can always decide to go ahead with the release).

Or (as an alternative) we could go over the existing list of issues both in the next release and openmw-1.0 and see if we can move some of them to openmw-future.

About the current milestone filling up with more and more tasks that are carried on indefinitely:
Again, I was hoping to address that after 1.0, because I don't like the idea of doing a reorganisation now. But what we could do (on the next release) is to move all inactive tasks from current to openmw-1.0 (or openmw-future/openmw-1.0-optional in some cases). That would at least get rid of the bug tracker update spam on each release.
MithrilLeaf
Posts: 33
Joined: 25 May 2014, 19:53

Re: Using version numbers on the bugtracker

Post by MithrilLeaf »

I'm a longtime lurker so my opinion isn't worth a ton, but for what it does worth I would probably suggest the quick and dirty fix.

A quick run through of the issues to select whichever ones are actually needed for 1.0, then dump the rest for after this son of a gun is finished. Then it can be a big reorganization effort to better make use of the very likely large influx of labor. Plus finally reaching 1.0 would be killer for morale.
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: Using version numbers on the bugtracker

Post by scrawl »

Chris wrote: 11 Sep 2017, 02:26I always figured openmw-future would be for post-1.0 stuff, and openmw-1.0 is for things that need to be done for 1.0. openmw-future makes it sound like it's a far-off (future) consideration, rather than ASAP.
Sorry for the confusion, the list has to be read backwards compared to the one above it, that's why I numbered it 3-2-1 (the 'everything else' part made more sense that way).
Makes sense. But we would have to define more clearly the meaning of the individual priority options. And maybe the severity field, too?
Would that definition be in any way specific to our project? If it's not, do we really need it?
I support this idea.
I'll ping lgro about the permissions change.
First openmw-future is for things that are not required for 1.0. Doesn't always mean that we can't do them (see above, unruly developers).
Is it? From what I've seen things not required for 1.0 have 'no version' (unless we can't or don't want to do them for 1.0, in which case they're openmw-future). That's how the tracker has been used, in my experience. If we decide this usage was wrong, we need to prepare for a lot of reorganisation work.

If we went by your proposed logic, then everything triaged will naturally fit into either openmw-1.0 or openmw-future (which could then just be renamed 'openmw-rest' or 'openmw-undecided'). This holds less information than using openmw-1.0, openmw-future and 'none' separately ('future' only for things that we can't or don't want to do now, and 'none' for undecided).

I'm getting the impression people think using 'no version' delibarately is bad because it would mean 'untriaged'. But that's not what it is. Issues with a version are triaged, but not all issues without a version are untriaged. There are many reasons why one would not be able to decide on a version after triaging an issue. IMO we should use the 'Confirmed' field as meaning 'Triaged'. (Note there isn't a 'Confirmed' status for Features, but that's not really a problem - most features are either openmw-1.0 or openmw-future anyway - and for the rest, I'd argue that there isn't a single way to 'triage' a feature, often it's a continous process with many changes/discussions along the way, even after implementation starts)

If we wanted to use versions for triaging, we have to add another milestone 'openmw-undecided' or 'openmw-triaged' and that is just too confusing and noisy.

I think what we're doing now is perfectly fine, with the only problem being the 'next version' milestone being used as a dumping ground that no ones knows what it means.
But if people really dislike it so much, maybe we should consider a openmw-1.0-optional milestone that would collect all non-essential issues from openmw-1.0.
So yet another version/milestone, that seems confusing. I'd just give them 'no version' to communicate that the issue is not (longer) required for 1.0. Anyway, we don't have to do that now and I agree the growing list is not really an issue (it should shrink eventually).
About the current milestone filling up with more and more tasks that are carried on indefinitely:
Again, I was hoping to address that after 1.0, because I don't like the idea of doing a reorganisation now.
If I offer to do it myself, do you like the idea now? ;)
But what we could do (on the next release) is to move all inactive tasks from current to openmw-1.0 (or openmw-future/openmw-1.0-optional in some cases). That would at least get rid of the bug tracker update spam on each release.
A mass movement without actually looking at the issues would be harmful - then we're dealing with clutter on openmw-1.0, or whereever we move them, that doesn't actually belong there.

I had a quick look. The number is not as bad as I thought (only ~50 now, when it used to be 100-150). Ignoring the in progress/resolved ones, I only found one issue that has anything to do with the next release: 'Task #3776 Linux generic release package is missing OSG plugins' (relevant because it's a trivial fix and should be done, missing files needing to be added to the actual release build / packaging script). The rest are either:
1) untriaged, needing feedback/discussion, can't reproduce etc.
2) Bugs that likely originate from an external dependency (so even if they're fixed, we can't claim them as being fixed in a particular OpenMW release)
3) Issues that should strictly go to 'openmw-1.0', because Morrowind doesn't have this issue/omission (but the issue hasn't been assigned to openmw-1.0 maybe because we deemed it 'unimportant')

1) and 2) can go to no version. 3) probably goes to openmw-1.0 (that doesn't mean we'll actually end up doing them before 1.0, as you said we can do a purge by priority at some point). Sound good?
User avatar
Zini
Posts: 5538
Joined: 06 Aug 2011, 15:16

Re: Using version numbers on the bugtracker

Post by Zini »

Would that definition be in any way specific to our project? If it's not, do we really need it?
I consider most of the labels a bit too vague to be useful (at least if they are used as a primary means of organisation). Critical and blocker are clear. When to use either urgent or immediately is already less clear. The remaining labels I usable decide upon by gut feeling (when I use them at all) and that is not a good method IMO.


As for the rest of your comments, my concern with this whole reorganisation thing is that after 1.0 we probably need a different scheme, because obviously there won't a large ultra-important 1.0 milestone anymore.

Regarding the no version state: My opinion is that this should be used for issues that are not actionable, not because of external issues (need prerequisites done, needs 1.0 first etc.), but because of issues with the issue itself: insufficient information to categorise it, triage not happened yet or still in progress and so on.
Every other issue should have a version number assigned to it, because that means it will appear somewhere on the roadmap and given the right circumstances a developer can pick it up.

btw. I am not trying to push for the openmw-1.0-optional milestone. I truly consider its addition optional. But I don't think having more milestones is confusing, as long as it is clearly defined what they are for. The later part actually seems to be a bit of a problem, since apparently different people have somewhat different ideas what the existing milestones are for.
If I offer to do it myself, do you like the idea now?
Fine with me. Still would prefer if we sit it out until 1.0 and then take a fresh look at the situation, but if you absolutely want to do it, go ahead.
Post Reply