Release process changes

Anything related to PR, release planning and any other non-technical idea how to move the project forward should be discussed here.
User avatar
Ace (SWE)
Posts: 887
Joined: 15 Aug 2011, 14:56

Re: Release process changes

Post by Ace (SWE) »

AnyOldName3 wrote:It might not be of any use to this discussion, but another open-source project whose primary user group is gamers and handles things pretty well is Dolphin Emulator. While their releases are few and far between, and users are recommended to use development builds, there's never really an issue of broken packages.

Basically, instead of nightlies, 'release' packages are build for every pull request (merged or proposed) as part of the CI process. I think this is actually done on dedicated machines (or dedicated VMs in the cloud) rather than Travis or Appveyor, but I'm guessing they could theoretically be used. The gist of what I'm proposing is:
  • Appveyor & Travis do their usual thing.
  • We add a final few commands to the CI scripts that convert the build that just passed to an installable form (e.g. .msi, .deb, .dmg etc.).
  • Appveyor & Travis then upload these to other machines/VMs.
  • The other machines install the packages from a clean-slate status (e.g. they were just reimaged), and do some basic testing to check nothing's died.
  • These builds are transferred from the secondary testing machines to somewhere else, and listed under a dev build heading at openmw.org/downloads/ and are uploaded to an OpenMW-dev PPA (we might as well let people use them now we have them). They're already verified as 'working'.
The RC process would be mostly irrelevant, as all it would entail would be renaming an existing file and linking to it somewhere official. We get a consistent build environment, and as more things would be automated, there's less chance of something 'dumb' happening, like mispackaging a build. For systems that can't easily have packages generated in the CI environment, we rely on something akin to raevol's 'Option 1', but we get a bunch of coverage of users without much conscious effort.

Obviously this would require a substantial amount of work to set up in the first place (which may be seen as excessive this close to 1.0), but lots of work that happens now by hand would come out for free. There also might be a problem of me wildly simplifying something to the point of trivialising it, and missing that something I've said is actually impossible.
Would be really nice to be able to delegate the packaging to the AppVeyor instances, would free up some time and let me actually rebuild my local dev environment without fears. Though considering the fact that - until very recently - they couldn't even build all parts of OpenMW in pure unoptimized Debug without timing out...

We should really improve on our tests too, they're quite simplistic at the moment and have never compiled properly on Windows since their inception. I blame GTest for that by the way, it doesn't play well with MSVC.
darkbasic
Posts: 153
Joined: 18 Apr 2016, 15:45
Contact:

Re: Release process changes

Post by darkbasic »

raevol wrote:How does it address the resulting visitors to our site wondering why they were told elsewhere that we are released, when we are not actually released?
Let me think, the RC1 after 0.41? There is not even a single open source project having such a problem with proper naming convenctions, or maybe you think all of your users are idiot?
I was lurking here and I saw how you managed to release 0.40: final packages were published into the wild calling for testing, anyone would be deceived by that. You said you already used RCs, I say you didn't in the manner I suggested: it's just naming convenctions, if you use them in a different way or for other purposes we're talking about different beasts. You don't like the name RC because you think it will mislead your users? Call them TESTING1, SNAPSHOT1 or whatever you prefer. It isn't the name which matters, but the purpose.
raevol wrote:And how does it address broken RC builds also being posted everywhere in aforementioned fashion?
You see a problem, I see a blessing. RC builds being posted means more testing, and NOBODY is going to be deceived by a build which contains "RC" in its name. Nobody will think it's the final version.
raevol wrote:These are the exact problems we are trying to address, and the proposed solutions I have seen have not addressed these problems in any way
I do not agree.

Regarding videos, you can simply print a giant "TESTING VERSION, STILL NOT RELEASED" watermark in the video itself. Such a way nobody is going to think it has been already released if it leaks (and it will probably still do) on reddit.
You still feel hurt because of the leak, despite the giant watermark? I can understand, then publish videos in private sections before release, I agree with that.
Last edited by darkbasic on 08 Sep 2016, 11:25, edited 1 time in total.
darkbasic
Posts: 153
Joined: 18 Apr 2016, 15:45
Contact:

Re: Release process changes

Post by darkbasic »

AnyOldName3 wrote:It might not be of any use to this discussion, but another open-source project whose primary user group is gamers and handles things pretty well is Dolphin Emulator. While their releases are few and far between, and users are recommended to use development builds, there's never really an issue of broken packages.

Basically, instead of nightlies, 'release' packages are build for every pull request (merged or proposed) as part of the CI process. I think this is actually done on dedicated machines (or dedicated VMs in the cloud) rather than Travis or Appveyor, but I'm guessing they could theoretically be used. The gist of what I'm proposing is:
  • Appveyor & Travis do their usual thing.
  • We add a final few commands to the CI scripts that convert the build that just passed to an installable form (e.g. .msi, .deb, .dmg etc.).
  • Appveyor & Travis then upload these to other machines/VMs.
  • The other machines install the packages from a clean-slate status (e.g. they were just reimaged), and do some basic testing to check nothing's died.
  • These builds are transferred from the secondary testing machines to somewhere else, and listed under a dev build heading at openmw.org/downloads/ and are uploaded to an OpenMW-dev PPA (we might as well let people use them now we have them). They're already verified as 'working'.
The RC process would be mostly irrelevant, as all it would entail would be renaming an existing file and linking to it somewhere official. We get a consistent build environment, and as more things would be automated, there's less chance of something 'dumb' happening, like mispackaging a build. For systems that can't easily have packages generated in the CI environment, we rely on something akin to raevol's 'Option 1', but we get a bunch of coverage of users without much conscious effort.

Obviously this would require a substantial amount of work to set up in the first place (which may be seen as excessive this close to 1.0), but lots of work that happens now by hand would come out for free. There also might be a problem of me wildly simplifying something to the point of trivialising it, and missing that something I've said is actually impossible.
That would be the best approach of course, but it's harder and more expensive.
darkbasic
Posts: 153
Joined: 18 Apr 2016, 15:45
Contact:

Re: Release process changes

Post by darkbasic »

psi29a wrote:Just got into the office...

anyway, my rational behind having a private PPA is that the end result (non-RC, but final version) can be put there, hence the name staging. Since I have better things to do with my time than wait for an OpenMW release (really, house renovations, 4 children and work), I rather have it waiting for me, ready to go. When raevol or whoever says "GO", I would just copy the package from private staging PPA to public PPA.

The private PPA is for _me_, so I don't have to spent too much time on it. Just a click and I'm done. I can do it from my mobile.

If I do what darkbasic suggests, then I have to spend the time to modify changelogs, renaming them, build package, upload for for every release of ubuntu we are currently supporting thus delaying a release even more. I would much rather do it before hand, having it waiting and ready.

Having a Private PPA is no different than doing the above, having the packages sitting on my laptop ready/waiting to be uploaded... except that if something has gone wrong with the upload or the package, I won't know about it until the upload has happened and possibly delay the release. With the Private PPA, I can fix whatever problems come up before the official release.

I still don't see how this effects anyone other than me by making my life easier.
If the annoyance is having to rename the package version and adding a one line "No changes" changelog then we should go with the private PPA route. My fault, I didn't understand that was the problem.
Last edited by darkbasic on 08 Sep 2016, 11:27, edited 1 time in total.
darkbasic
Posts: 153
Joined: 18 Apr 2016, 15:45
Contact:

Re: Release process changes

Post by darkbasic »

Anyway guys don't misunderstand me: you should go with whatever route you prefer. I firmly believe that whoever does the work should take the decisions. I simply wanted to let you understand that you should not fight your users, you should teach them and exploit their potential. They are no harm if you handle them right.
User avatar
psi29a
Posts: 5355
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: Release process changes

Post by psi29a »

darkbasic wrote:If the problem is having to rename the package version and adding a one line changelog with "No changes" inside it then we should go with the private PPA route. My fault, I didn't understand that was the annoyance.
On a good day, it is that simple. :lol:
I've been surprised by launchpad before, right before a release, their build machines going down, acting wonky or worse... I fat fingered and now we have an openmw-0.40.0~trusty4444 release and users are apt updating and upgrading before I can purge the build. While not a travesty, it is embarrassing on my part. ;)

I would rather have the final builds built and ready to deploy ahead of time. I don't want to be fighting stupid shit and holding up a release because I did something stupid or launchpad did something stupid.
darkbasic
Posts: 153
Joined: 18 Apr 2016, 15:45
Contact:

Re: Release process changes

Post by darkbasic »

psi29a wrote:On a good day, it is that simple. :lol:
I've been surprised by launchpad before, right before a release, their build machines going down, acting wonky or worse.
I can understand, that's why I have my own build servers. Anyway if one of the packages misses the release date by a few hours or if you have to delay the release by a single day while some packages gets released in the meantime this is no problem in my opinion. Instead a full month release window with packages floating around has proven to be an issue :lol:
User avatar
Ace (SWE)
Posts: 887
Joined: 15 Aug 2011, 14:56

Re: Release process changes

Post by Ace (SWE) »

On an unrelated note; Still waiting for the Chocolatey package to go through validation, seems like that's going to take a while.

You Linux people have it good with doing releases, what with standardized packaging and dependencies that don't all have to be bundled into your application. Static linking is a pain on Windows as well, due to how the standard library / C++ runtime is set up.
User avatar
psi29a
Posts: 5355
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: Release process changes

Post by psi29a »

Ace (SWE) wrote:You Linux people have it good with doing releases, what with standardized packaging and dependencies that don't all have to be bundled into your application. Static linking is a pain on Windows as well, due to how the standard library / C++ runtime is set up.
And still we can find stuff to bitch about. ;) We're not happy until we don't have to do anything at all. :lol:
K1ll
Posts: 184
Joined: 06 Aug 2011, 21:54

Re: Release process changes

Post by K1ll »

psi29a wrote:Sure, we'll do a google hangout g/talk thing sometime. You know, in case I and/or k1ll get hit by a bus. Speaking of which, how k1ll makes his releases is also black-magic. From what I gather, he creates static builds of all libraries that he can so that it is generic enough. It also has to be built with an older version of glibc since he says it should run on older Fedora/RH/CentOS releases, probably in a VM. Would be interesting if he could also clarify how he goes about things so we can duplicate work should the unthinkable happen.

Either way, what we discuss and if he also talks about his build process, we can document for the wiki and/or openmw's documentation.
Nope no static builds involved atm. I've been thinking about it but doing static builds of everything is a lot of work and so far my current build and packaging process seems to work reasonably well.

Currently I'm using chroots/containers running debian wheezy + builds of ffmpeg, SDL2, bullet and osg (installed in /usr/local not even packaged). I have added some (quite horrible) cmake scripting that checks which libraries are used by cmake for the build and then copies them into the directory used by cpack for creating the targz archive. I'm not packaging all libraries used for the build though only the ones that seem necessary.

There is a branch named targzpackage in my GitHub repo which contains completely outdated versions of the cmake scripts. One of the reasons I stopped pushing to GitHub is that the branch currently is (and has been for a long time now) in a state where it has accumulated so much cruft that only converting it to a patch containing all that is relevant now and starting over by applying that on master can clean it up again. And I haven't found the motivation to do that yet since our cmake scripts have been quite stable (or at least merge-able) for the past few releases so pushing out a new release was mostly just merging in the tags and starting the build.

Going forward I've planned to take a good look at flatpack/snaps/appimage and make packages in those formats which should work better for newer distros while still providing the targz packages for distros/people that don't have/don't want these new formats.
Post Reply