Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce build duration by lowering Mockito usage #2707

Open
jkronegg opened this issue Mar 16, 2023 · 5 comments · May be fixed by #2767
Open

Reduce build duration by lowering Mockito usage #2707

jkronegg opened this issue Mar 16, 2023 · 5 comments · May be fixed by #2767
Assignees
Labels
🏦 debt Tech debt 🙏 help wanted Help wanted - not prioritized by core team

Comments

@jkronegg
Copy link
Contributor

jkronegg commented Mar 16, 2023

🤔 What's the problem you've observed?

Mockito has a impact on the unit test execution time (seen using IntelliJ Profiler flame graph):

project test duration [s] mockito contribution
cucumber-junit 4.4 31% = 1.3 s
cucumber-core 7.5 10% = 0.7 s
cucumber-java 2.8 24% = 0.6 s
cucumber-java8 1.8 27% = 0.5 s
cucumber-spring 4.5 13% = 0.5 s
cucumber-testng 1.5 20% = 0.3 s
cucumber-guice 1.8 15% = 0.3 s
datatable 0.5 (unused dependency) 0% = 0 s

Total build duration: 3min17 = 197 seconds
Total test duration : 23.5 seconds
Total Mockito contribution : 4.2 seconds (18% of tests, 2% of build duration)

By reducing the usage of Mockito, the test duration can be lowered with two advantages:

  • this gives the developer a shorter feedback loop.
  • the energy consumption is lower (green IT), which is important because the cucumber-jvm project is built often (due to PR and renovate bot). 2% is not a lot, but on the long term, it makes a difference.

✨ Do you have a proposal for making it better?

I didn't check in details, but for example in JavaBackendTest, the @Mock ObjectFactory and Glue could be replaced by fake implementations, which could reduce the class test duration by about 80%.

@jkronegg jkronegg added the 🏦 debt Tech debt label Mar 16, 2023
@mpkorstanje
Copy link
Contributor

mpkorstanje commented Mar 19, 2023

Yes. This makes sense. I would agree to Mockito replacements if:

  1. Stubs implementations are declared in the test class.
  2. Stub implementations are simple.

This will likely result in some duplication but that is an acceptable tradeoff.

@jkronegg jkronegg added good first issue Good for newcomers 🙏 help wanted Help wanted - not prioritized by core team labels Mar 22, 2023
@jkronegg jkronegg self-assigned this Apr 21, 2023
@jkronegg jkronegg linked a pull request Jun 6, 2023 that will close this issue
7 tasks
@jkronegg jkronegg removed the good first issue Good for newcomers label Jun 6, 2023
jkronegg pushed a commit that referenced this issue Aug 4, 2023
@CharlesLgn
Copy link

Does someone made a study to use another mock library to see if there will be better performance ? (EasyMock/JMockit/...)

@jkronegg
Copy link
Contributor Author

I didn't tried with other mock library. The proposed solution #2767 replaces mockito by stub and fake implementations, which leads to a 7% improvement of the build time (that is, much more than the 2% expected).

@CharlesLgn
Copy link

I didn't tried with other mock library. The proposed solution #2767 replaces mockito by stub and fake implementations, which leads to a 7% improvement of the build time (that is, much more than the 2% expected).

I understand the principle of using fake implementation. However, I think that if you have an interface with 25 methods, the test class will be poluate with a lot of fake implementation, with also a lot unuse code.

That is why I questioned the use of other library

@jkronegg
Copy link
Contributor Author

I agree that readability is generally lower when using stub classes with a lot of methods... This may be improved by having specific stub classes that are shared between test cases, but @mpkorstanje wanted stubs implementations to be declared in the test class.

I didn't found a lot of mocking libraries performance comparisons, but according to mockito/mockito#3288, mockito and Easymock have more or less the same performance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏦 debt Tech debt 🙏 help wanted Help wanted - not prioritized by core team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants