Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dependency analysis in gitlab #2144

Open
wants to merge 1 commit into
base: integration
Choose a base branch
from

Conversation

cogross
Copy link
Collaborator

@cogross cogross commented Oct 23, 2023

Dependency analysis runs in gitlab
updated structure to remove most "unused declared" and "used undeclared" dependencies.

A few still remain that are used that the tool falsely identifies as unused. Notes on those below. I tried using a few different tactics but I wasn't quite able to untangle the dependencies.

locally, run
mvn -DignoreNonCompile dependency:analyze
to see results.

NOTES
datawave-ws-atom
needs abdera, axiom-impl, axiom-api, hadoop-client-runtime

datawave-ingest-core
needs accumulo-server-base, log4j-core, hadoop-client-runtime, log4japi, log4j-slf4j-impl

datawave-ingest-csv
needs datawave-ingest-configuration, accumulo-core is not test only

datawave-ingest-json
needs datawave-ingest-configuration, hadoop-mapreducew-client-core/-common

datawave-query-core
needs json-simple, jackson-module-jaxb-annotations, lucene-analyzers-common, commons-logging, datawave-ingest-configuration, hadoop-mapreduce-client-core

datawave-ingest-wikipedia
needs datawave-ingest-configuration, type-utils not just test only

assemble-datawave

datawave-ops-tools-index-validation
needs zookeeper

OLD NOTES
datawave-core:

  • removing log4j-slf4j-impl causes org.junit.ComparisonFailure: expected:<[Hash]UID> but was:<[Snowflake]UID> error - UID uses a logger.

datawave-ws-atom:

  • abdera-parser, hadoop-client-runtime, axiom-api, axiom-impl are all required

datawave-ingest-core:

  • accumulo-server-base, log4j-core, hadoop-client-runtime, log4j-api, log4j-slf4j-impl are all required

datawave-ingest-csv:

  • hadoop-mapreduce-client-common, datawave-ingest-configuration are used
  • accumulo-core not just scoped for tests

datawave-ingest-json:

  • hadoop-mapreduce-client-core, hadoop-mapreduce-client-common, datawave-ingest-configurationare all required

datawave-query-core:
-hadoop-mapreduce-client-core, json-simple are required

datawave-ws-model:

  • slf4j-api is required
  • authorization-api is not just scoped for tests

datawave-ingest-wikipedia:
-datawave-ingest-configuration is required

  • type-utils is not just scoped for tests

datawave-metrics-core:

  • datawave-query-core is required

datawave-ops-tools-config-compare:

  • hadoop-client-runtime is required

datawave-ops-tools-index-validation:

  • zookeeper, guava are required

@cogross cogross force-pushed the dependency-analysis-in-gitlab-squash branch 5 times, most recently from 8d6929c to 9fec5b7 Compare October 25, 2023 16:34
@cogross cogross force-pushed the dependency-analysis-in-gitlab-squash branch 3 times, most recently from 1f7ffe1 to c215908 Compare November 7, 2023 12:19
@cogross cogross force-pushed the dependency-analysis-in-gitlab-squash branch 4 times, most recently from 761736e to 367009b Compare November 16, 2023 16:39
@cogross cogross force-pushed the dependency-analysis-in-gitlab-squash branch 5 times, most recently from c3ba32d to 83bbbc6 Compare December 1, 2023 11:40
@cogross cogross force-pushed the dependency-analysis-in-gitlab-squash branch 3 times, most recently from 1a2278a to 3c84d00 Compare December 11, 2023 20:18
@cogross cogross force-pushed the dependency-analysis-in-gitlab-squash branch 3 times, most recently from f70b06f to 4545a0f Compare December 15, 2023 17:57
@cogross cogross force-pushed the dependency-analysis-in-gitlab-squash branch from 4545a0f to 56a8faa Compare January 16, 2024 18:41
@cogross cogross force-pushed the dependency-analysis-in-gitlab-squash branch 4 times, most recently from 200ea83 to 8f83b84 Compare January 30, 2024 15:17
@cogross cogross force-pushed the dependency-analysis-in-gitlab-squash branch 2 times, most recently from 82a9abe to a59bc19 Compare February 8, 2024 15:38
@cogross cogross force-pushed the dependency-analysis-in-gitlab-squash branch 4 times, most recently from 9cbc96d to 0321a60 Compare March 21, 2024 13:41
@cogross cogross force-pushed the dependency-analysis-in-gitlab-squash branch 3 times, most recently from eedd9fb to 7ce39ad Compare April 5, 2024 10:35
@cogross cogross force-pushed the dependency-analysis-in-gitlab-squash branch 3 times, most recently from 1c0a807 to a68b7ac Compare April 15, 2024 19:03
@cogross cogross force-pushed the dependency-analysis-in-gitlab-squash branch 4 times, most recently from c3f4042 to 8d7d769 Compare April 23, 2024 10:52
@cogross cogross force-pushed the dependency-analysis-in-gitlab-squash branch 3 times, most recently from 14ade5e to aac32b4 Compare May 6, 2024 11:36
@cogross cogross force-pushed the dependency-analysis-in-gitlab-squash branch 5 times, most recently from c7e92b5 to caa48a8 Compare May 13, 2024 15:32
@cogross cogross force-pushed the dependency-analysis-in-gitlab-squash branch 2 times, most recently from 4fd4a25 to b57034f Compare May 17, 2024 11:52
@cogross cogross force-pushed the dependency-analysis-in-gitlab-squash branch 2 times, most recently from 074caee to e8d4079 Compare May 28, 2024 17:30
Comment on lines -72 to -73
<groupId>io.astefanutti.metrics.cdi</groupId>
<artifactId>metrics-cdi</artifactId>
Copy link
Collaborator

@keith-ratcliffe keith-ratcliffe Jun 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't verified yet, but I'm pretty sure this change is the reason that the web service deployment is failing in the test runner. That lib is needed for CDI injection of the dropwizard @Metric-annotated fields.

From server.log...

org.jboss.weld.exceptions.DeploymentException: WELD-001408: Unsatisfied dependencies for type Counter with qualifiers @Default
  at injection point [BackedAnnotatedField] @Inject @Metric private datawave.webservice.operations.remote.RemoteAccumuloService.failureCounter
  at datawave.webservice.operations.remote.RemoteAccumuloService.failureCounter(RemoteAccumuloService.java:0)

Sadly, Wildfly doesn't give anything useful like a ClassNotFoundException in this case :-(

@cogross cogross force-pushed the dependency-analysis-in-gitlab-squash branch from 1553678 to a33b4c9 Compare June 10, 2024 15:39
@cogross cogross force-pushed the dependency-analysis-in-gitlab-squash branch from a33b4c9 to ea0b412 Compare July 2, 2024 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants