Memory Leak in hypersistence-optimizer itself?

[MichaelZett]

Hi Vlad,

we're using the optimizer for a few weeks now and encountered a memory leak in our load-test-area.

Before 09:30 we run with optimizer enabled, after 09:30 without.
optimizer-version: 2.3.1
Our Bean:
@Bean public HypersistenceOptimizer hypersistenceOptimizer(EntityManagerFactory entityManagerFactory, Set<Class<? extends Event>> knownAcceptedEvents, Set<AcceptedEvent> acceptedEvents) { return new HypersistenceOptimizer( new JpaConfig(entityManagerFactory) .setEventFilter(event -> applyFilters(event, knownAcceptedEvents, acceptedEvents)) ); }

The optimizer seems to collect a lot of events but does not drop them - is there a config that we missed for that?

[vladmihalcea]

The Optimizer collects all events and never clears them as it doesn't know if the client application consumed them.

However, you can call clear on the events after you consumed them and exported somewhere else:

hypersistenceOptimizer.getEvents().clear(); 

[MichaelZett]

Ok, thanks, we will try something with that clear-method.

[vladmihalcea]

That's a good idea, and the List is also thread-safe, so you can clear it from a worker Thread:

private List<Event> events = new CopyOnWriteArrayList<Event>();

By the way, are those runtime events that capture long-running queries and Sessions?

[MichaelZett]

Nice, thanks.
We do not have long-running queries but long-running sessions. Have to look into it soon.

[vladmihalcea]

If you can provide me with more info, I'd gladly look into it. Normally, it shouldn't be any leak as other than the Event objects all temporary objects are bound to the Session and get cleaned up when the Session is closed.

[MichaelZett]

We'll look into it and put some data together for you. Perhaps tomorrow. Than you for your quick response.

[vladmihalcea]

Looking forward to it.

[MichaelZett]

Hi Vlad,

the mass of events seems to consist of these:
{ "connectionUsageTimeNanos": 163244, "transactionStatus": "COMMITTED", "connectionUsageTimeMillis": 0, "description": "The JDBC Connection was borrowed for [163] μs for management purposes while the transaction status was [COMMITTED].", "priority": "MAJOR", "infoUrl": "https://vladmihalcea.com/hypersistence-optimizer/docs/user-guide/#TransactionReconnectionEvent" },
.
Perhaps it would be better to sum up the count and overall borrow time instead of adding each single event?

Is there a way to see the context of this event? We are using spring-boot (2.5.2), spring-data, hibernate, hikari and postgresql. We use your suggestions for hikari.auto-commit: false and provider_disables_autocommit: true.

[vladmihalcea]

You can filter the TransactionReconnectionEvent out using an EventFilter if your setup cannot be changed to avoid acquiring the DB connection multiple times per transaction.

[MichaelZett]

Hi,
yes we will do that for this event and SessionFlushTimeoutEvent and FlushModeAutoEvent. Perhaps you could think about adding a feature for such events, so that they are counted and not re-added every time that happens.
Another feature request would be for these events and SessionTimeoutEvent as well to get more context information where/why it happens or which queries ran. If that is possible.

For now we look at the other many proposals, your nice tool made for our hibernate usage. Thank you for that.

[vladmihalcea]

I created the #144 issue for the runtime event aggregation.

The query events contain the query info, but the session events don't contain such info. One solution would be to store a Session summary, containing the queries that were executed and the flush time info.

Thanks for using it and for giving me feedback about it.

[vladmihalcea]

Should have been fixed by #146. If it wasn't, then I'll reopen it.