Continuous integration

[psi29a]

This is just a kickstarter for conversation here about unit testing and the conterminous integration. I wanted to see if it is desirable to have this for OpenMW.

I have (and others here as well) have experience with CI tools like Jenkins (previously Jeeves). For those wondering what CI is like, here is a rough overview as it applies to OpenMW:

*Developer pushes something to git repository
*A post-receive hook pushes that commit on to a stack of commits to check for that tree.
*Build bot pops next testable sha1 from the stack
*Build bot checks out the code and runs a test script (compile, link, run, unit tests)
---
*If the test script succeeds, logs that the SHA1 passed tests. Test results are pushed to a webserver for later viewing.
*If the test script fails, logs the failure and collects artefacts.
---
*Test results get pushed.
*Developers get notified.
*After a failure, build bot uses git bisect to find a new commit between the last known good and the current. This becomes the next testable commit.

If you would like to read more:
http://en.wikipedia.org/wiki/Continuous_integration

The tests could be anything with an expected output, like ordering of esx files or known bad esm/esx/plugins should be rejected and not accepted, Morrowind.ini gets processed correctly or if bad data in Morrowind.ini that the importer doesn't blow up and exit badly. These are the type of things we can test for.

I would love to hook directly into functions or at least be able to play with object data via a command-line option like --debug so that we can automate these type of testing.

Things like 'invisible armour' would be non-trivial to test for this way.

What do you guys think?

[Zini]

I think we had the discussion before and IIRC lgro has already set up Jenkins for us.

What we absolutely have to cover is the integrity of content files during load-save-cycles. It would be a disaster if the editor at some point trashes files. Originally the idea was to do it via the ESMTool. But actually, using the editor directly might be a better option.

There may be some components that we can test in isolation and we should also consider testing the integrity of saved game files during load-save-cycles. Beyond that I do not see a lot of value in CI and automated testing for OpenMW. A game engine is not the most suitable type of application for it and our code base isn't designed for it either.

[lgromanowski]

Hi,
there is configuration file for Travis CI: https://github.com/zinnschlag/openmw/pull/757

Travis CI is a continuous integration server for open-source projects, integrated with GitHub - to use it you have to login with your GitHub account, next in https://travis-ci.org/profile you need to enable proper repository (by pressing on/off button) and that's all - it will start building of openmw everytime something will be merged into master or next branch. If you would like to receive notification e-mails about build status than you have to add your e-mail into .travis.yml file, in recipients section.

Whole process of building OpenMW takes about 9-10 minutes, after that unit tests are started.