Addin database

Join the team. This area is for people who want to participate in OpenMW's development in one way or another.
User avatar
AnyOldName3
Posts: 1454
Joined: 26 Nov 2015, 03:25

Re: Addin database

Post by AnyOldName3 » 19 Jan 2019, 16:56

The main reason you're seeing resistance is that you still don't seem to have quite grasped the ratio of things which such a tool would flag as a conflict versus things that it wouldn't. I don't think it would be too much of an exaggeration to say that it would find problems with any setup with more than a handful of mods, even if in reality, everything worked exactly as intended. As you said, databases aren't magic, and I'm concerned that the tool you're envisioning would actually need magic before it would start to become useful, but you might spend a lot of time on it before that becomes obvious.
AnyOldName3, Master of Shadows

Jwcolby54
Posts: 21
Joined: 01 Jan 2019, 08:48
Location: NC
Contact:

Re: Addin database

Post by Jwcolby54 » 20 Jan 2019, 03:51

AnyOldName3,

Thanks for the response.

So if I am not grasping the magnitude of the collisions, why don't you describe the collisions and why you think every single line of a mod is going to be a collision? It is not possible to understand when all that is provided is a very general "everything collides with everything" response.

It seems obvious that everything doesn't collide with everything since many mods work together. Why do some collide with each other and others don't? What collides? I have seen over and over that "everything collides with something". REALLY?

I have read the descriptions of the various items in the game. I have loaded the OpenCS and looked through the data.

So please describe in detail how items in a mod are going to collide with items in every other mod?

I am asking for your help and all I am getting is "it isn't possible".

Jwcolby54
Posts: 21
Joined: 01 Jan 2019, 08:48
Location: NC
Contact:

Re: Addin database

Post by Jwcolby54 » 20 Jan 2019, 04:59

>>>I don't think it would be too much of an exaggeration to say that it would find problems with any setup with more than a handful of mods,

I am trying to understand this. I have played Morrowind for hundreds perhaps even thousands of hours. I have installed a dozen or more mods, usually simultaneously. I have witnessed collisions, but they were always when an object was created on top of another object.

Suppose I created a house mod in balmora. Another in Gnisis. How are those going to collide? I could build two hundred houses and place them in separate towns and none of them will collide. I could place one in balmora next to the temple on the right side and another next to the temple on the left side and they won't collide. I could build an entire tomb and as long as it is not overlapping another tomb it is not going to collide. I can place a new NPC for a quest I build in Balmora. I am not sure how that is going to collide with anything unless I place it dead on top of the starting position of another NPC. Even if the paths collide, I can ANALYZE those paths in a database and spot path collisions.

You are correct of course, I am not grasping the magnitude of collisions that you are grasping.

Now... if you are discussing "better body" mods and the user wants to install two different such mods, yes I can see how that is going to collide. In which case I simply discover that they do and warn the user (or modder). I pretty much don't care that two "better body" mods from different people collide. I don't care that there are 4000 collisions between those two mods. Of course there are. Don't do that. A database could almost certainly analyze such collisions even then.

And finally... let's go back to your statement...

>>>I don't think it would be too much of an exaggeration to say that it would find problems with any setup with more than a handful of mods...

Isn't that a GOOD thing? As opposed to blindly installing stuff with no clue at all that such collisions are occuring? How about we just ignore said collisions and let the user try and figure out what's going on?

User avatar
AnyOldName3
Posts: 1454
Joined: 26 Nov 2015, 03:25

Re: Addin database

Post by AnyOldName3 » 21 Jan 2019, 16:46

I could place one in balmora next to the temple on the right side and another next to the temple on the left side and they won't collide.
That's not how the game data works. Both of these would add new objects to the same cell, and unless you also load up the collision data from the Nif files and analyse that (which isn't something a database can do), all you're going to be able to see is that both of these mods add objects to the Balmora_Temple_01 (or whatever it's called) cell, not how big they are or what space they occupy. It's also likely that they'll both remove or relocate some objects that are in that cell in the vanilla game (for example to move some crates out of the way). These changes might conflict in a visible or invisible way - if they both remove the same barrel, there's no problem, if one moves it out of the way and the other removes it entirely, you're either going to end up with the barrel out of the way or gone (both fine) or out of the way of one building but in the way of the other. Alternatively, if one mod moves it to cover up a hole in a mesh and the other puts it somewhere else or gets rid of it, you're going to be able to see the hole. All of these would look the same to a database-based approach.

With things in different places, you're going to have less of this sort of thing, but you're still going to find conflicts with big mods, such as the DLC, even if it's very likely that the mod author was fully aware of the changes they made and took them into account.

The crux of the issue is that there's no way to know if any given edit (or lack thereof, for example not moving or removing a building because you've added a new building with an adjoining wall) is required for a critical feature of a mod or if it's just incidental and can be overridden. Lots of things are completely incidental and can be overridden by another mod, and lots of things are important, and anything that correctly flags all of one type will incorrectly flag a lot of the other type.

A hypothetical example of a mod that wouldn't at first appear to conflict with anything, but would probably conflict with nearly everything in practice would be a mod that added two NPCs. These two NPCs would, when the player reached them, set off on a race where one would have to visit every settlement and the other would visit every ruin, shrine and tomb, with the order being determined such that they'd both reach the finish line at very similar times, but the settlement NPC would win, and then the other one would ask for a rematch with a slightly different order, and then he'd win. This would be disrupted by nearly any addition, modification or removal of objects or NPCs anywhere as the route would be subtly (or significantly) disrupted and the timings would be off. In practice, such a mod would almost certainly break even with the load order the same (as Morrowind isn't deterministic), but it only needs to be a thought experiment to show that in theory any change, no matter how distant or minor, could be a conflict.
AnyOldName3, Master of Shadows

silentthief
Posts: 368
Joined: 18 Apr 2013, 01:20
Location: Currently traversing the Ascadian Isles

Re: Addin database

Post by silentthief » 21 Jan 2019, 18:45

I did a quick sweep of the conversation, and perhaps I am wrong but it seems that the problem you two are having with understanding the other's perspective is the way this is being discussed. Collision versus conflict. Collision meaning two objects trying to occupy the same 3d space is not necessarily going to cause a mod conflict. Such is the nature of the morrowind world. You can overlay a static object such as a building inside of another. Teleporting doors in walls, plants growing out of another, items such as a sword sticking out of a stone in an Arthurian Excaliber kind of way. These objects do not cause mod conflicts though. Collision in that aspect rarely affects anything other than Player/NPC/creatures movement

However, if you have two mods work changes on the same object then you can have unintended results which result in errors such as scripts not working, or object duplication. This kind of error would likely be due to ambiguous reference or undoing an objects properties set and expected by another mod. These conflicts can be hard to determine though, with any mod able to change the same object

ST
(ps - edited for clarity)

(pps - not to say that checking for objects occupying the same 3d space isn't a bad idea - you try to come out of an interior cell to where a door marker "teleports" you to the exterior and find you are inside of another object it would definitely be an issue. However, these are usually not hard to troubleshoot, and if you were able to test for this you are going find that there are MANY objects occupying the same space, or a portion thereof; that another object is occupying)
"Hurry, hurry! Last boat to Solstheim! Until the next one. Hah-ha-hah."

User avatar
cc9cii
Posts: 471
Joined: 28 Mar 2013, 04:01

Re: Addin database

Post by cc9cii » 21 Jan 2019, 23:43

I don't have a good grasp of how much effort is involved, but why not build it and see if people find it useful? The algorithm to detect if a record is being modified is already available (e.g. see OpenCS).

Jwcolby54
Posts: 21
Joined: 01 Jan 2019, 08:48
Location: NC
Contact:

Re: Addin database

Post by Jwcolby54 » 22 Jan 2019, 15:11

AnyOldName,

Thanks for the more detailed reply.

>>>>>Both of these would add new objects to the same cell, and unless you also load up the collision data from the Nif files and analyse that (which isn't something a database can do), all you're going to be able to see is that both of these mods add objects to the Balmora_Temple_01 (or whatever it's called) cell, not how big they are or what space they occupy.

My first question is how is the surface of the world understood if no X/Y/Z data is available? How can I walk around in the world if the game engine does not know that I am on the ground, banging into a rock or tree? In at least a gross level the game engine must know that my foot is hitting an object, and therefore the XYZ of my foot and said object.

Does OpenMW understand meshes? Does the team understand the format? I have to assume so since they started from scratch and recreated the world from the existing world meshes. Or are the meshes just thrown at a render engine, with no understanding of what the meshes do nor how the render engine processes them?

If they are understood, then OpenMW, and I assume OpenCS knows how to load and examine them. Meshes are sets of connected triangles in 3d space. As such they have location data in 3d space. If they did not then it would be impossible to do mods and fit them into the world.

I have not looked at the mesh(es) but I actually built (programmed) a sphere made out of triangles back in 1983 in the process of writing Megatek graphics terminal drivers. Decades ago with no modern tools or existing code to work with.

So my question to you then is why you think that a database cannot hold mesh data? It is triangles, x/y/z data. Typically with fill colors and patterns on top of that. Assuming that the file format is known and code exists which reads them.

I am not attempting to say that form fitting a mesh into and around another mesh is a simple thing to do, but given the triangle data for a given object, I certainly could discover a square (or sphere) to stay outside of to prevent touching another mesh. The MaxX, MinX, MaxY, MinY, MaxZ, MinZ of any triangle in the mesh should describe a cube that entirely contains the object. So take the temple in balmora as an example, it contains thousands of triangles. But surely it is possible to discover all of the triangles which make up the entire thing?

Which it seems to me is sufficient to grossly guide "physical 3d collision detection". If a mod artists wants to avoid a physical collision in 3d space then they have to know the physical boundaries of their own mod as well as the physical boundaries of the world where they wish to place an object.

Would it be worth the effort? I have no view into the file structure of meshes yet, but in general, files have definitions, and as such can be read and written.

Jwcolby54
Posts: 21
Joined: 01 Jan 2019, 08:48
Location: NC
Contact:

Re: Addin database

Post by Jwcolby54 » 22 Jan 2019, 16:06

silentthief wrote:
21 Jan 2019, 18:45
...You can overlay a static object such as a building inside of another. Teleporting doors in walls, plants growing out of another, items such as a sword sticking out of a stone in an Arthurian Excaliber kind of way. These objects do not cause mod conflicts though. Collision in that aspect rarely affects anything other than Player/NPC/creatures movement
I was playing morrowind the other day. The assassins were attacking me. I positioned myself in a corner between a bed and a wall in order to better defend myself and when the assassins spawned they spawned inside of the wall. We have probably all seen such things, an arm or leg sticking through a door waving around. I could actually whack at the thing and it could not whack back at me.

So yea, the point is that this is a difficult problem to define, never mind prevent. But there's an old saying, "If I think I can't, then I am right." I try hard not to do "it can't be done, but rather to do a cost / benefit analysis".

Can I do what is being discussed? I don't yet know. The short answer is yes, I can do it. The longer answer is "but it might in fact just be too much work."

However, if you have two mods work changes on the same object then you can have unintended results which result in errors such as scripts not working, or object duplication. This kind of error would likely be due to ambiguous reference or undoing an objects properties set and expected by another mod. These conflicts can be hard to determine though, with any mod able to change the same object
Maybe, maybe not. If I can get a unique identifier of an object, and the same object with the same unique identifier is modified in two different mods, then it is trivial to discover (using a database) that two mods are trying to mod the same item. A more in depth analysis of what the two mods are attempting to do and do they conflict is a discussion for another day but at least I could warn the Mod artist that "this object has been modified in Mod(s) XYZ.

Again, there are thousands of mods already in existence. New ones being produced all the time. I cannot, and it is not my business to prevent physical conflicts, but rather to simply FIND such things. They can't be fixed if they are not known. And they can't be avoided if they are not known. I truly don't care if a given mod object physically collides with another object one time or a thousand. ONCE is all I need to know to say "we have a potential problem". I cannot even say that it is in fact a problem, only that it collides with another object and the mod author might want to know that. I could (possibly) list all the collisions - maybe. If a mod author wants that info, if it would be useful to them.

(pps - not to say that checking for objects occupying the same 3d space isn't a bad idea - you try to come out of an interior cell to where a door marker "teleports" you to the exterior and find you are inside of another object it would definitely be an issue. However, these are usually not hard to troubleshoot, and if you were able to test for this you are going find that there are MANY objects occupying the same space, or a portion thereof; that another object is occupying)
True, kinda. Not many objects actually occupy the exact same physical space. The example of the assassins shows just how useful it would be to be able to discover when that occurs.

This discussion has been very useful in demonstrating how many different kinds of conflicts can arise. Like any other engineering effort, one generally starts by listing the problem(s) to be solved, with the most bang for the buck and works down the list of what can be done at a reasonable cost.

I keep hearing "it can't be done". What part can't be done? And what prevents "it" being done? And how useful is it that "it" be done?

I was looking at the data in the morrowind file for the "paths" for NPCs. The points that they move to are defined. Can I read those out and then look at whether a structure is "in the way of" any such path? Yes, if I can get the dimensions of the structure or tree or whatever is being added. What happens if an object is placed on top of one of those endpoints? Can the game itself be modified to no longer allow the NPC to go there?

jwc

Post Reply