Portmod - Mod Manager

Post about your mods, learn about OpenMW mod compatibility, check for problematic mods, discuss issues, and give us feedback about your experience with modded OpenMW.
edvind
Posts: 1
Joined: 14 Mar 2020, 15:18

Re: Portmod - Mod Manager

Post by edvind »

I just have to say: This is a great command line tool! Thank you so much for developing it.

I wish I was better at programming so that I could help. Is there anything else I can do? Maybe looking up mod licenses or something?
User avatar
bmw
Posts: 81
Joined: 04 Jan 2019, 19:42
Contact:

Re: Portmod - Mod Manager

Post by bmw »

You're welcome!

The main thing that requires little or even no programming experience would be adding mods to the repository.

Creating pybuild files to add mods to the repository does require some programming knowledge, but for most mods it should be minimal as it's mainly just copying information into fields in a python file that can be copied from a template (either copy and modify an existing pybuild file, or you can also use the skel.pybuild in the root of the repo which has some descriptions of the fields). I would suggest avoiding large, complicated, mods, particularly at first. It's much simpler when mods have just one source file and few necessary configuration steps or optional features. There's also a brief guide on the wiki, which may be helpful: https://gitlab.com/portmod/portmod/-/wi ... uild-Guide

Alternatively, or for mods which you might feel are too complicated to add yourself, you could just create an issue for them on the openmw-mods repository that lists things such as the license information that you've found, any dependencies the mod has, any optional features, and maybe even the structure of the source file. This is basically the information that goes into the build file, and would be a useful starting point for someone else when creating the pybuild file. I think I'll set up an issue template on GitLab that can be used as a starting point.

The other thing of course is bugtesting. Just using portmod and reporting any problems or bugs you encounter is very helpful.
User avatar
bmw
Posts: 81
Joined: 04 Jan 2019, 19:42
Contact:

Re: Portmod - Mod Manager

Post by bmw »

It's been a while since I've posted an update here.

I've just released portmod 2.0 beta6 (right after beta5; deployment was rough). The main difference since beta4 is the merging of omwcmd (a rust executable that was doing some things I didn't want to write in python) into the portmod codebase as a rust extension (using setuptools-rust and pyo3). Primarily, this means that the build process has a significant new step (compiling the rust extension), though to facilitate distribution I've also set up pre-compiled wheels to be built for python 3.6-3.8 on Linux, macOS and Windows, which are available through pypi (usually used automatically when installing with pip). If you have any issues with installation, please let me know.
There were also a number of bugs fixed, notably some long-standing bugs affecting macOS and Windows.

Portmod is actually quite close to stable, though there remains a significant amount of restructuring work, as I'd like to make some major changes to its internals prior to the 2.0 release candidate, some of which include structural interface changes (which will break things from the perspective of the package repository), though little will change in terms of functionality.

Something else I think is worth mentioning is that I've mostly completed the initial work to internationalize Portmod (it hasn't been merged just yet), and I'm looking at setting up a Pontoon instance to make translation more accessible.
I think Pontoon could be a great resource in the long run in the context of the work I've also been doing creating a git-friendly text version of plugin files (see viewtopic.php?f=3&t=6816), as we could eventually have plugins which are stored alongside all their localizations in a single repository, and have a pontoon instance provide an interface for translating Portmod, OpenMW (I don't think I've seen any plans for OpenMW internationalization yet) and various mods (likely only those with permissive licenses), all in the same place.
darkbasic
Posts: 153
Joined: 18 Apr 2016, 15:45
Contact:

Re: Portmod - Mod Manager

Post by darkbasic »

Can't wait for the day when installing the total overhaul will be as simple as emerging a set :o
ezze
Posts: 513
Joined: 21 Nov 2013, 13:20

Re: Portmod - Mod Manager

Post by ezze »

I am probably missing something obvious... I reset my installation deleting both the directories under .config and .local

I select the profile "[4] default/openmw/1.0/morrowind-tb-bm (stable) *", set up the Morrowind directory in the configuration file, executed "omwmerge -uDN @world" and finally "omwmerge i-heart-vanilla".

"omwmerge i-heart-vanilla" asked me to download all kind of files, and I did it a part of "Wood_Fix-47469-2-1b-1588600554.7z" under "https://www.nexusmods.com/morrowind/mod ... ?tab=files" for it is not there.

I still asked to continue and process fails with

Code: Select all

>> Preparing source in /tmp/portmod/items-armor/adamantium-armor-1.0/work/adamantiumarmor ...
Applying /tmp/portmod/items-armor/adamantium-armor-1.0/work/adamantiumarmor-umopp-3.0.2/adamantiumarmor_compat_plugin.patch...
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/portmod/main.py", line 758, in main
    configure_mods(
  File "/usr/lib/python3.8/site-packages/portmod/main.py", line 496, in configure_mods
    elif install_mod(mod):
  File "/usr/lib/python3.8/site-packages/portmod/mod.py", line 366, in install_mod
    src_prepare(mod, BUILD_DIR, WORKDIR)
  File "/usr/lib/python3.8/site-packages/portmod/mod.py", line 55, in src_prepare
    mod.src_prepare()
  File "/home/paolo/.local/share/portmod/openmw/items-armor/adamantium-armor/adamantium-armor-1.0.pybuild", line 34, in src_prepare
    apply_patch(os.path.join(path, "adamantiumarmor_compat_plugin.patch"))
  File "/usr/lib/python3.8/site-packages/pybuild_/pybuild.py", line 44, in apply_patch
    for line in Git().apply([patch, "--numstat"]).split("\n"):
  File "/usr/lib/python3.8/site-packages/git/cmd.py", line 542, in <lambda>
    return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
  File "/usr/lib/python3.8/site-packages/git/cmd.py", line 1005, in _call_process
    return self.execute(call, **exec_kwargs)
  File "/usr/lib/python3.8/site-packages/git/cmd.py", line 822, in execute
    raise GitCommandError(command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git apply /tmp/portmod/items-armor/adamantium-armor-1.0/work/adamantiumarmor-umopp-3.0.2/adamantiumarmor_compat_plugin.patch --numstat
  stderr: 'error: can't open patch '/tmp/portmod/items-armor/adamantium-armor-1.0/work/adamantiumarmor-umopp-3.0.2/adamantiumarmor_compat_plugin.patch': No such file or directory'
ERROR: Cmd('git') failed due to: exit code(128)
  cmdline: git apply /tmp/portmod/items-armor/adamantium-armor-1.0/work/adamantiumarmor-umopp-3.0.2/adamantiumarmor_compat_plugin.patch --numstat
  stderr: 'error: can't open patch '/tmp/portmod/items-armor/adamantium-armor-1.0/work/adamantiumarmor-umopp-3.0.2/adamantiumarmor_compat_plugin.patch': No such file or directory'
What can I try?
ezze
Posts: 513
Joined: 21 Nov 2013, 13:20

Re: Portmod - Mod Manager

Post by ezze »

I tried to make my own repo to install mods that are not under portmod already, or that are broken. And fail to understand what I am supposed to do.

Here is the links to the guide I am trying to decypher.
pybuild-guild
Repos.cfg
pybuild-guide

According to Repos.cfg I need to prepare a directory with create the file Repos.cfg with two sections, the first about the default repository and another about my local one. Here is:

Code: Select all

[openmw]
location = /home/ezze/.local/share/portmod/openmw
auto_sync = True
sync_type = git
sync_uri = https://gitlab.com/portmod/openmw-mods.git
priority = -1000

[user]
location = /home/ezze/games/Morrowind_portmod
auto_sync = False
priority = 0
masters = "openmw"
In the directory /home/ezze/games/Morrowind_portmod I created the file metadata/layout.conf (empty) and the file profiles/repo_name with the content the single word user (plus a newline).

Then I copied the file assets-textures/intelligent-textures/intelligent-textures-2.1b.pybuild from the standard location and put in my directory, so I have /home/ezze/games/Morrowind_portmod/assets-textures/intelligent-textures/intelligent-textures-2.1b.pybuild.

Fixed the pybuild file so it does not refer to any woodfix file. Like this:

Code: Select all

# Copyright 2019-2020 Portmod Authors
# Distributed under the terms of the GNU General Public License v3
from pybuild import InstallDir, Pybuild1

class Mod(Pybuild1):
    NAME = "Intelligent Textures"
    DESC = "Replaces almost all vanilla textures with high resolution AI upscales"
    HOMEPAGE = "https://www.nexusmods.com/morrowind/mods/47469"
    KEYWORDS = "openmw"
    LICENSE = "all-rights-reserved"
    TEXTURE_SIZES = "2048"
    SRC_URI = """
        Intelligent_Textures-47469-2-1-1588592170.7z
    """
    IUSE = "atlas"
    RDEPEND = "atlas? ( assets-misc/project-atlas )"
    DATA_OVERRIDES = "assets-misc/project-atlas[-atlasgen]"
    TIER = 1
    INSTALL_DIRS = [
        InstallDir("00 Core", S="Intelligent_Textures-47469-2-1-1588592170"),
        InstallDir(
            "01 Atlas Textures",
            S="Intelligent_Textures-47469-2-1-1588592170",
            REQUIRED_USE="atlas",
        )
    ]

Then I tried to validate and got inundated by nonsense:

Code: Select all

$ pybuild assets-textures/intelligent-textures/intelligent-textures-2.1b.pybuild validate

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/portmod/omwpybuild.py", line 70, in main
    pybuild_validate(args.pybuild_file)
  File "/usr/lib/python3.8/site-packages/portmod/main.py", line 803, in pybuild_validate
    mod.validate()
  File "/usr/lib/python3.8/site-packages/pybuild_/pybuild.py", line 438, in validate
    raise Exception(
Exception: Pybuild /home/paolo/games/Morrowind_portmod/assets-textures/intelligent-textures/intelligent-textures-2.1b.pybuild contains the following errors:
Dependency assets-misc/project-atlas could not be found!
Data Override assets-misc/project-atlas[-atlasgen] could not be found!
Use flag "atlas" must be either a global use flag or declared in metadata.yaml
LICENSE all-rights-reserved does not exit! Please make sure that it named correctly, or if it is a new License that it is added to the licenses directory of the repository
ERROR:root:Pybuild /home/paolo/games/Morrowind_portmod/assets-textures/intelligent-textures/intelligent-textures-2.1b.pybuild contains the following errors:
Dependency assets-misc/project-atlas could not be found!
Data Override assets-misc/project-atlas[-atlasgen] could not be found!
Use flag "atlas" must be either a global use flag or declared in metadata.yaml
LICENSE all-rights-reserved does not exit! Please make sure that it named correctly, or if it is a new License that it is added to the licenses directory of the repository
I missing something, how one is supposed to make his own repository?
User avatar
bmw
Posts: 81
Joined: 04 Jan 2019, 19:42
Contact:

Re: Portmod - Mod Manager

Post by bmw »

ezze wrote: 29 Sep 2020, 16:03What can I try?
The bug seems to be caused by a regression in version 5.39 of the file tool, which is causing patool to misidentify the archive. I don't know anything further about the problem in file, but I have submitted a patch upstream to patool so that it can gracefully fall back to other detection methods if file fails: https://github.com/wummel/patool/pull/87.

For the moment, using either my fork of patool (until it's merged. See openmw-mods#125), or reverting to an older version of file would work around the issue.
ezze wrote: 29 Sep 2020, 16:03 "omwmerge i-heart-vanilla" asked me to download all kind of files, and I did it a part of "Wood_Fix-47469-2-1b-1588600554.7z" under "https://www.nexusmods.com/morrowind/mod ... ?tab=files" for it is not there.
Evidently the patch was removed. I don't really understand why someone would keep only some of the files from old versions of a mod, but the package file does need to be updated for the new version anyway.
ezze wrote: 30 Sep 2020, 14:39 I missing something, how one is supposed to make his own repository?
You appear to be doing everything correctly, except that you should put 'masters = "openmw"' in layout.conf, not in repos.cfg.
ezze
Posts: 513
Joined: 21 Nov 2013, 13:20

Re: Portmod - Mod Manager

Post by ezze »

After writing the messages I tried a bit more and I got some results. I understood that not all validation errors are not critical and the pybuild may work anyway, so I started installing a bit of mods.

I have to say that to start with portmod is a bit infuriating, I thought "what the hell, it was quicker to re-install everything manually" in the begining, the error management is still very rough. But what portmod gives if you manage to make it work is really grand. So thank you.

I plan to put the settings and personal repo directories in one place and keep everything under revision control, so can I reinstall Morrowind as I like easily.


Few questions:

For texture pack the dependency order is, of course, critical. For example the only good retexture I know for dead silt striders have to be after comprehensive texture packs like the Intelligent one. portmod "tiers" are a bit too rough for this, the solution is just writing the .csv file manually or I can set up the thing in the pybuild?

Are you interested in wiki edit to make the guides clearer? (based on my experience of course)
User avatar
bmw
Posts: 81
Joined: 04 Jan 2019, 19:42
Contact:

Re: Portmod - Mod Manager

Post by bmw »

ezze wrote: 01 Oct 2020, 02:27 After writing the messages I tried a bit more and I got some results. I understood that not all validation errors are not critical and the pybuild may work anyway, so I started installing a bit of mods.

I have to say that to start with portmod is a bit infuriating, I thought "what the hell, it was quicker to re-install everything manually" in the begining, the error management is still very rough. But what portmod gives if you manage to make it work is really grand. So thank you.
Indeed, the learning curve is still a bit steep, but it will hopefully improve in future. It's worth noting that the pybuild validate command, in addition to reporting errors which will cause problems, also raises some QA-related errors, which may be those you've noticed are not critical.
I'm glad it's working out though.
ezze wrote: 01 Oct 2020, 02:27 For texture pack the dependency order is, of course, critical. For example the only good retexture I know for dead silt striders have to be after comprehensive texture packs like the Intelligent one. portmod "tiers" are a bit to rough for this, the solution is just writing the .csv file manually or I can set up the thing in the pybuild?
There is also the DATA_OVERRIDES variable, which can force a package's data files to override the data files of another package in the VFS, however it's not supposed to be used for personal preferences, more for things such as mesh incompatibilities, mainly since it's a hard ordering and any cycles in the graph of such links will break the sorting algorithm. The tiers on the other hand are soft orderings, and just break ties. While they can't describe everything, you could certainly use them to distinguish between large texture packs and small specific mods, the latter of which can be assumed that the user wants loaded after the large texture pack if they decided to install them.
Note that we can always change the conventions if there's a type of mod which the current conventions don't account for. I suspect what you want in this instance is a tier between 1 and 2, so we might need to shift the numbers, or switch to a named tier system to allow tiers to be inserted without such numbering issues.
ezze wrote: 01 Oct 2020, 02:27 Are you interested in wiki edit to make the guides clearer? (based on my experience of course)
Certainly! If you want to edit it directly, I'll need to give you the right permissions, as the built-in GitLab wiki we're using can only be edited by users with at least Developer permissions of the portmod repository. I'd much rather it be a publicly editable wiki using something like mediawiki or dokuwiki, but I've never set that up.
ezze
Posts: 513
Joined: 21 Nov 2013, 13:20

Re: Portmod - Mod Manager

Post by ezze »

bmw wrote: 01 Oct 2020, 14:41 Note that we can always change the conventions if there's a type of mod which the current conventions don't account for. I suspect what you want in this instance is a tier between 1 and 2, so we might need to shift the numbers, or switch to a named tier system to allow tiers to be inserted without such numbering issues.
For textures pack I think the rough "bigger before smaller" convention would cover most cases. After all, if you put a small texture pack it is more likely to cover a detail you want to see.

ezze wrote: 01 Oct 2020, 02:27 Are you interested in wiki edit to make the guides clearer? (based on my experience of course)
bmw wrote: 01 Oct 2020, 14:41 Certainly! If you want to edit it directly, I'll need to give you the right permissions, as the built-in GitLab wiki we're using can only be edited by users with at least Developer permissions of the portmod repository. I'd much rather it be a publicly editable wiki using something like mediawiki or dokuwiki, but I've never set that up.
I'll see if I manage with pull request and such, if not we'll agree with permissions. I asked mainly because I know some developers are quite jealous of their documentation (only "I" know the software well enough to write about it!), but since it is not case I'll try to write something. I am not sure when though :S, real life can be a pain.
Post Reply