Unit Tests?

Everything about development and the OpenMW source code.
Post Reply
User avatar
xirsoi
Posts: 21
Joined: 21 Oct 2014, 21:14

Unit Tests?

Post by xirsoi »

Hello! Long time lurker, rarely a poster.

In meatspace I'm a Software Engineer in Test, which means I write automated testing as my day job.

I was wondering if my experience could be utilized here? Having good unit tests is always of help in a project, especially as it nears "completion", and they're critical to the success of major refactors. Obviously I'm referring to post-1.0 de-hardcoding there.

Now, the trouble is that a lot of refactoring will probably be necessary just to make the project unit testable. Working Effectively with Legacy Code by Michael C. Feathers is a fantastic book on such things, by the way. The stickiness of that is partially mitigated by the fact that we'll have unit tests to confirm that the refactoring worked!

I'm not ready to jump in on such a project right now, but I'd like to gauge interest anyway.
User avatar
psi29a
Posts: 5355
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: Unit Tests?

Post by psi29a »

Yes please, there is a framework already running with a very limited amount of tests... but if you would like to add more, that would be great! :)
nwah
Posts: 45
Joined: 21 Nov 2013, 07:40

Re: Unit Tests?

Post by nwah »

Excellent! Whatever you can do to improve things would be much appreciated.
Last edited by nwah on 08 Feb 2018, 19:23, edited 1 time in total.
User avatar
Thunderforge
Posts: 503
Joined: 06 Jun 2017, 05:57

Re: Unit Tests?

Post by Thunderforge »

psi29a wrote: 07 Feb 2018, 11:12 there is a framework already running with a very limited amount of tests
Which framework is that? How do I run them? I had no idea this existed.
User avatar
psi29a
Posts: 5355
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: Unit Tests?

Post by psi29a »

We use google test framework:
https://github.com/OpenMW/openmw/tree/m ... test_suite

Have a look here near the end:
https://travis-ci.org/OpenMW/openmw/jobs/337120676

Code: Select all

0.01s$ if [ "$COVERITY_SCAN_BRANCH" != 1 ] && [ "${TRAVIS_OS_NAME}" = "linux" ]; then ./openmw_test_suite; fi
[==========] Running 12 tests from 5 test cases.
[----------] Global test environment set-up.
[----------] 2 tests from ContentFileTest
[ RUN      ] ContentFileTest.dialogue_merging_test
No content files found, skipping test
[       OK ] ContentFileTest.dialogue_merging_test (1 ms)
[ RUN      ] ContentFileTest.content_diagnostics_test
No content files found, skipping test
[       OK ] ContentFileTest.content_diagnostics_test (1 ms)
[----------] 2 tests from ContentFileTest (2 ms total)
[----------] 2 tests from StoreTest
[ RUN      ] StoreTest.delete_test
[       OK ] StoreTest.delete_test (0 ms)
[ RUN      ] StoreTest.overwrite_test
[       OK ] StoreTest.overwrite_test (0 ms)
[----------] 2 tests from StoreTest (0 ms total)
[----------] 3 tests from KeywordSearchTest
[ RUN      ] KeywordSearchTest.keyword_test_conflict_resolution
[       OK ] KeywordSearchTest.keyword_test_conflict_resolution (1 ms)
[ RUN      ] KeywordSearchTest.keyword_test_conflict_resolution2
[       OK ] KeywordSearchTest.keyword_test_conflict_resolution2 (0 ms)
[ RUN      ] KeywordSearchTest.keyword_test_conflict_resolution3
[       OK ] KeywordSearchTest.keyword_test_conflict_resolution3 (0 ms)
[----------] 3 tests from KeywordSearchTest (1 ms total)
[----------] 3 tests from EsmFixedString
[ RUN      ] EsmFixedString.operator__eq_ne
[       OK ] EsmFixedString.operator__eq_ne (0 ms)
[ RUN      ] EsmFixedString.empty_strings
[       OK ] EsmFixedString.empty_strings (0 ms)
[ RUN      ] EsmFixedString.struct_size
[       OK ] EsmFixedString.struct_size (0 ms)
[----------] 3 tests from EsmFixedString (0 ms total)
[----------] 2 tests from PartialBinarySearchTest
[ RUN      ] PartialBinarySearchTest.partial_binary_search_test
[       OK ] PartialBinarySearchTest.partial_binary_search_test (0 ms)
[ RUN      ] PartialBinarySearchTest.ci_test
[       OK ] PartialBinarySearchTest.ci_test (0 ms)
[----------] 2 tests from PartialBinarySearchTest (0 ms total)
[----------] Global test environment tear-down
[==========] 12 tests from 5 test cases ran. (3 ms total)
[  PASSED  ] 12 tests.
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: Unit Tests?

Post by scrawl »

Funny enough, none of those tests ever broke, which makes me think they might have been a waste of time. But maybe someday they will ...
MorrowMark
Posts: 2
Joined: 08 Feb 2018, 23:56

Re: Unit Tests?

Post by MorrowMark »

I don't think tests never failing means they were a waste of time. Good tests and wide coverage does a lot to increase code confidence, both for the person writing the test and everyone who follows after.

I'd argue this is more important for open source projects, where you have less people familiar with the whole thing and new people rotating in all the time. I'd feel much better jumping into a new, unfamiliar project if it was already well-tested.
User avatar
wareya
Posts: 338
Joined: 09 May 2015, 13:07

Re: Unit Tests?

Post by wareya »

That's only true with TDD, and TDD is basically incompatible with how game engines need to be designed. Typically, to me, when I see a small number of tests and they never get triggered, that tells me that the project doesn't do enough regression testing - but this is a game engine, so going ingame and making sure that it works close to intended is basically a rigorous regression test itself.
User avatar
xirsoi
Posts: 21
Joined: 21 Oct 2014, 21:14

Re: Unit Tests?

Post by xirsoi »

Well written and maintained unit tests are always valuable, not just in TDD.

My C++ is admittedly rusty (mostly do C# these days, with a little bit of js and java) and I'm not familiar with googletest, so it'll be awhile before I'm ready to start contributing tests.
Post Reply