Versioning with _version_git breaks if softioc is installed

If a module installs softioc, and also uses VersionGit for versioning, the version number appears to be set to that of softioc when making a test release. This may also affect released modules.

Steps to reproduce (using Diamond tools):

dls-start-new-module.py -a python3 -n dls_example_module
cd dls_example_module/
pipenv install -d
pipenv install softioc
dls-py3 install-into-prefix
ls prefix/lib/python3.7/site-packages/

This will result in a module which shows the version to be 3.1, which is the latest release of softioc:

dls_example_module  dls_example_module-3.1.dist-info

What I have found so far:

It looks like _version_git in the your new module doesn't get called when dls-py3 install-into-prefix is called.
On a clean repository, pipenv run python -m setup bdist_wheel will get the correct version (last tag + commit hash etc). If you run dls-py3 install-into-prefix first, the wheel will get stuck with the softioc version
When running _version_git after install, it appears that GIT_REFS and GIT_SHA1 are set to the values from softioc
This exists on (at least) versions 3.0, 3.0b2 and 3.1

It seems to work for me with /dls_sw/prod/tools/RHEL7-x86_64/defaults/bin/pipenv (output below). The only suspicious code is this:

# Place the directory containing _version_git on the path
for path, _, filenames in os.walk(os.path.dirname(os.path.abspath(__file__))):
    if "_version_git.py" in filenames:
        sys.path.append(path)
        break:

which appears in setup.py of every VersionGit enabled module. If __file__ in setup.py points to a directory that also contains a softioc directory, then it might find that instead. Can you think of any way that could happen?

Here's the output of it working for me:

tmc43@pc0105:~/python $ dls-start-new-module.py -a python3 -n dls_example_module

Making clean directory structure for dls_example_module
Initialising repo in /dls/science/users/tmc43/common/python/dls_example_module
Repository created.
Staging files...
Committing files to repo... commit msg: "Initial commit"

A new Python 3 module has been created in the dls_example_module directory.
Edit setup.cfg to make sure it has the correct metadata.
Python code should typically be added to the dls_example_module package.

tmc43@pc0105:~/python $ cd dls_example_module/
tmc43@pc0105:~/python/dls_example_module $ pipenv install -d
Pipfile.lock not found, creating...
Locking [dev-packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success! 
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success! 
Updated Pipfile.lock (c0f625)!
Installing dependencies from Pipfile.lock (c0f625)...
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 1/1 — 00:00:03
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
tmc43@pc0105:~/python/dls_example_module 1m18s $ pipenv install softioc
Installing softioc...
Adding softioc to Pipfile's [packages]...
✔ Installation Succeeded 
Pipfile.lock (c0f625) out of date, updating to (2a61dc)...
Locking [dev-packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success! 
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success! 
Updated Pipfile.lock (2a61dc)!
Installing dependencies from Pipfile.lock (2a61dc)...
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 — 00:00:00
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
tmc43@pc0105:~/python/dls_example_module 1m17s $ dls-py3 install-into-prefix
Installing into /dls/science/users/tmc43/common/python/dls_example_module/prefix. This can be slow.
Creating lightweight-venv from Pipfile.lock ...
Completed after 5.5s
Installing into prefix using lightweight-venv ...
Completed after 4.7s
Installation complete after 0m 10s.
tmc43@pc0105:~/python/dls_example_module 11s $ ls prefix/lib/python3.7/site-packages/
dls_example_module  dls_example_module-0.0+untagged.g59eeebf.dirty.dist-info
tmc43@pc0105:~/python/dls_example_module $ which pipenv
/dls_sw/prod/tools/RHEL7-x86_64/defaults/bin/pipenv

It appears that this issue only affects some people. Andy Wilson, and one of the grads (so should be a fairly clean setup) got what I had. Will Rogers and another grad got the same as you

I seem to get the wrong version info too. I ran your example a half dozen times and had the same result each time. So whatever it is appears to be deterministic.

@MJGaughran

@MJGaughran @AlexanderWells-diamond please can you try applying 3618d94 to your dls_example_module and see if it fixes the issue? If so I'll create a new version of VersionGit that addresses this, and apply it to pythonSoftIOC

Apply the given code change to setup.py in my newly generated example module fixes the issue for me.

I assume this fix will also end up in the template module as well?

yes

Yes, those changes fix the issue, and I have already applied them to my other module. I'll let you close this ticket.

I've come up with a slightly safer approach in DiamondLightSource/versiongit#9 which I will release as VersionGit 2.0 as it drops python2 support

I'll apply this change for pythonSoftIOC as we're keeping python2 support for now

Navigation Menu

Repository navigation

Versioning with _version_git breaks if softioc is installed #31

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Versioning with _version_git breaks if softioc is installed #31

Description

Activity

thomascobb commented on Aug 20, 2021

MJGaughran commented on Aug 20, 2021

AlexanderWells-diamond commented on Aug 20, 2021

thomascobb commented on Aug 23, 2021

AlexanderWells-diamond commented on Aug 23, 2021

thomascobb commented on Aug 23, 2021

MJGaughran commented on Aug 25, 2021

thomascobb commented on Aug 25, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions