Compile failures due to tycho consumes artifact from local maven repo instead from target

[phermsdorf]

Hi,

we have a kind of split up build:

1. we have a tycho product build that also produces a update site
2. we have a test build (which runs on the same machine=same maven repository) which consumes the artifacts of the product ```
   build with a target file like this:

<target name="test">
    <locations>
        <location includeAllPlatforms="false" includeConfigurePhase="false" includeMode="planner" includeSource="false" type="InstallableUnit">
            <repository location="${system_property:target-repository}"/>
            <unit id="com.company.product" version="0.0.0"/>
        </location>
        <location type="Target" uri="file:${project_loc:/com.company.target}/com.company.target.target"/>
    </locations>
</target>

We currently use tycho 4.0.5.

Now we have a bundle that was build with a version qualifier like Bundle-Version: 6.0.2449.qualifier in main. Then a branch was created from that, and the bundle was changed and there were breaking API changes in that change. Because of the created MR that bundle was build on the same build server, probably only with the qualifier changed (jgit).
Before that MR was merged back a release branch was created (based on the former version of that bundle).

After the MR was merged back to main, a new release build was required of the previous release (which was expected to use the original version of that bundle).
That release build now fails with compilation error, since it seems to consume the newer version of the bundle which has these breaking api changes, instead the one from the product build of step 1 (see above).

We do not use package versioning on MANIFEST.MF for import and/or export of the packages which maybe would have helped here.
The test build uses -Dtycho.localArtifacts=ignore, but i don't know if that is a good thing or not ... i've also found this issue #3713 but i don't know if it is related or not.

Is this the expected build behavior?
Can anything be tweaked to make this setup work reliable?

Thanks for any hints on this.

Bye Peter

[laeubi]

The test build uses -Dtycho.localArtifacts=ignore

Actually this is good and I would recommend to use that setting unless you specifically need to consume local versions of your dependencies.

You can also set this in the target platform configuration now

<plugin>
  <groupId>org.eclipse.tycho</groupId>
  <artifactId>target-platform-configuration</artifactId>
  <configuration>
    <dependency-resolution>
      <localArtifacts>ignore</localArtifacts>
    </dependency-resolution>
  </configuration>
 </plugin>

We currently use tycho 4.0.5.

There is already a Tycho 4.0.10 release so you probably want to upgrade even though it should not change much here.

Because of the created MR that bundle was build on the same build server, probably only with the qualifier changed (jgit).

You should always ever use a dedicated repository per branch/build job sharing the repo can give you unexpected results.

[phermsdorf]

After further investigations, some of my information turned out to be incorrect:

• the builds do not share .m2 repositories (actually they run inside a docker container which comes with a pre-populated .m2 folder)
• in that .m2 folder are no artifacts of previous build versions of that plugin (so it's pretty clean)

In the build log i see downloads of the bundle from the wrong repository location (which explains why the new version is used in the build).
But i don't understand why it's fetched from there. The only reference i find to this "wrong"/newer location is in our referenced target file from my first comment com.company.target.target. In there is a reference like this to the newer/wrong repo:

<location includeAllPlatforms="false" includeConfigurePhase="false" includeMode="slicer" includeSource="true" type="InstallableUnit">
      <unit id="org.eclipse.swt" version="0.0.0"/>
      <repository location="http://repo-server-int.int/newerRelease/lastSuccessfulBuild/artifact/com.company.product/target/repository/"/>
</location>

But from there it should only consume a newer version of org.eclipse.swt, but it seems like other bundles including the offending one are fetched and used from there.
I find no other reference in the build or build log to that repo.
Any ideas what is going wrong here?

Thank you.

Bye Peter

[laeubi]

Any ideas what is going wrong here?

It might be that even though the use of slicer the full repository is visible to the resolver, as you are using planner mode in your parent. Do you think you can provide an integration-test to demonstrate the issue?

[phermsdorf]

What would be a good starting point?

[phermsdorf]

i tried to get an integration test going, but without much success:
https://github.com/phermsdorf/tycho/tree/testResolveMultipleLocations

it's org.eclipse.tycho.test.resolver.ResolverTests.testMultipleLocationsCorrectIsChoosen()

but in the build log none of the 2 version of the two possible target locations is printed out. just a much more recent version ...
i'm stuck for now. maybe i'll give it another try next week.

bye peter

[phermsdorf]

I now moved the newer version of org.eclipse.swt and the corresponding platform fragments to another repo where no other build artifacts exist and now the build works and also "Downloading from" output of the build log references the correct URL and artifact version ... so there seems to be some kind of resolving issue.
After getting my work done, i'll try to investigate a bit more ...

Bonus question: in the build product, we now have the more recent swt bundle, but "old" platform fragments (which wasn't the case before moving the repo). In our feature we reference the swt bundle and fragments only with version "0.0.0".
Any idea what could cause this problem?

Thanks. Bye Peter

[phermsdorf]

FYI The problem with the "not using" of the correct swt platform fragments turned out to be a problem with the repository. The not working repository was created using package drone. They create a repo.zip on the fly, which basically seems to be ok (eg. contains alls bundles and fragments, can be used with in the .target file and the generic editor in PDE without problems) but the tycho build somehow did not like that repo. I tried with official eclipse repo url and from that point on it started to work.
In the end i used the cbi aggregator to create a small repo with the swt bundle and fragment from my internal location and that works now.