Enable Dependency Checksum Verification

[halibobo1205]

Background

I noticed that Besu(Java-based Ethereum client) has enabled dependency verification by this PR: Enable dependency checksum verification. However, TRON doesn't seem to have this feature enabled yet, working with external dependencies and plugins published on third-party repositories puts the build at risk. I suggest that TRON enable dependency verification to mitigate the security risks and avoid integrating compromised dependencies in the project.

Rationale

Dependency verification uses a mechanism introduced in Gradle 6.2, and it has been promoted to a stable feature since Gradle 7.0. This feature can be used for:

• detecting compromised dependencies
• detecting compromised plugins
• detecting tampered dependencies in the local dependency caches

Implementation

• feat(build): upgrade Gradle from 5.6.4 to 7.6.4 #5869
• test(CI): add test-retry plugin for the unit test #5893
• feat(build): enable dependency checksum sha256 verification #5903

[tomatoishealthy]

Sounds like a good idea.

Can TRON currently be smoothly migrated to gradle7.0? Are there compatibility issues and other risks?

[halibobo1205]

BTW, TRON currently uses Gradle version 5.6.4, the first step is to upgrade Gradle to 7.x. Compatibility issues are inevitable and need to be carefully tested.

[endiaoekoe]

really a good idea.
this feature is really helpful if import in the project, but there are also some challenges the teams will face when implementing dependency validation functionality in the projects, as below:

• Frequent Dependency Updates: As the project evolves, dependencies may be updated frequently, causing the verification-metadata.xml file to need frequent updates. This may increase the team's workload in maintaining and updating dependency validation information.

• Ongoing Monitoring and Maintenance: The project team needs to continuously monitor the results of dependency validation and address validation failures or issues in a timely manner. This requires the team to be vigilant and invest some time and effort in maintaining the dependency validation functionality.

[halibobo1205]

@endiaoekoe 🚦 I think the extra overhead is acceptable, after all, the security is more valuable.

[forfreeday]

Are the files generated by Dependency Checksum Verification consistent between platforms?

[tomatoishealthy]

I have some questions

1. Whether checksum uses sha256 as Besu does？
2. Whether you are going to check all the dependencies, I can see that the amount of metadata is very large when compared to Besu, and how to make sure that the initial metadata is correct.
3. Will you provide a switch to enable or disable the checksum verification?

Hope to get a detailed explain

[halibobo1205]

Are the files generated by Dependency Checksum Verification consistent between platforms?

@forfreeday The jar should have nothing to do with the platform, I think it should be the same, can you give me an example?

[halibobo1205]

@tomatoishealthy

1. Whether checksum use sha256 as Besu does？

Yes, I think sha256 is secure enough for now.

2. Are you going to check all the dependencies?

Yes, as Dependency verification you can see,

Gradle will verify any artifact downloaded using its dependency management engine, which includes, but is not limited to:

1. artifact files (e.g jar files, zips, …​) used during a build

2. metadata artifacts (POM files, Ivy descriptors, Gradle Module Metadata)

3. plugins (both project and settings plugins)

4. artifacts resolved using the advanced dependency resolution APIs

Gradle will not verify changing dependencies (in particular SNAPSHOT dependencies) nor locally produced artifacts (typically jars produced during the build itself) as by nature their checksums and signatures would always change.

3. how to make sure that the initial metadata is correct

The gradle/verification-metadata.xml file was autogenerated relying on the TOFU scheme. If you don't trust it enough, you can compute it locally, under the assumption that there are no problems with the artifacts.

4. Will you provide a switch to enable or disable the checksum verification?

disabling-verification : Gradle provides 3 different verification modes.

[forfreeday]

As far as I know, using this under linux generates dependency names with .module suffix:

      <component group='org.dom4j' name='dom4j' version='2.1.3'>
         <artifact name='dom4j-2.1.3.jar'>
            <sha256 value='549f3007c6290f6a901e57d1d331b4ed0e6bf7384f78bf10316ffceeca834de6' origin=' Generated by Gradle'/> <artifact name=' dom4j-2.1.3.jar'> <artifact name='dom4j-2.1.3.jar'>
         </artifact>
         <artifact name='dom4j-2.1.3.module'>
            <sha256 value='dec7c5fd8d520dafb57b15e2f3b6be2f180561342a3f451beeee905543bc1703' origin=' Generated by Gradle'/> </artifact>
         </artifact>
      </component>

Artifacts generated on macOS have a name suffix of .pom:

      <component group='org.dom4j' name='dom4j' version='2.1.3'>
         <artifact name='dom4j-2.1.3.jar'>
            <sha256 value='549f3007c6290f6a901e57d1d331b4ed0e6bf7384f78bf10316ffceeca834de6' origin=' Generated by Gradle'/> <artifact name=' dom4j-2.1.3.jar'> <artifact name='dom4j-2.1.3.jar'>
         </artifact>
         <artifact name='dom4j-2.1.3.pom'>
            <sha256 value='cdc9c49f59d230d4271771bb764aa7097d6acbcdc2500149e4d2c951df5cac30' origin=' Generated by Gradle'/> </artifact>
         </artifact>
      </component>

How should these two suffixes be compatible？

[317787106]

@halibobo1205 Wallet-cli maybe also need this feature, because it uses private key, secutiry is very important. Do you paln to implement it on wallet-cli ?

[halibobo1205]

@forfreeday Good catch, perhaps both are needed.

[halibobo1205]

Wallet-cli may also need this feature because it uses private key, security is very important. Do you plan to implement it on wallet-cli?

@317787106 I think it's necessary for wallet-cli.

[endiaoekoe]

BTW, TRON currently uses Gradle version 5.6.4, the first step is to upgrade Gradle to 7.x. Compatibility issues are inevitable and need to be carefully tested.

should it be better to make another single issue that focuses on migrating from grade 5.6.4 to 7.x ? per investigation on the formal docs gradle publish, the migration may involve many dirty works

[halibobo1205]

Should it be better to make another single issue that focuses on migrating from grade 5.6.4 to 7.x ?

@endiaoekoe Great, I'll create 2 tasks.

[halibobo1205]

It is prepared and ready for review #5903.