Massive AE2 slow down with Wooden Crates and Shipping Containers

[tehMorag]

Using over 80 Wooden Crates (maxed upgraded) and 20 Shipping Containers (fill with non upgraded Wooden Crates) all connected via AE2 "ME Storage Bus"

Have about 80 hopping bonsai's dumping it's items into the system via interfaces.
ME storage bus priority has them checking to see if there is room in the locked Wooden Crates, if it doesn't exist or have room, it them checks all the Shipping Containers for the same thing, then moves on to the ME Storage Cells as a last resort.

The issue... Having everything going, I end up getting the server timing out.
Crash log: https://pastebin.com/7bEchD0L
Laggoggles before crash:

Disconnected the bonsai's so nothing going into the system (system still auto crafting via AP2):

Hooked up bonsai's, but disconnected the Shipping Containers:

Disconnected bonsai's, Shipping Containers, and Wooden Crates:

Hooked up bonsai's but disconnected Shipping Containers, and Wooden Crates:

I hope this will help. If need be, I can send a world file for testing.
For more detailed look at the system, check out a VoD here from a few days ago debugging.
Finally got to checking out crates/shipping containers today.

Hope that helps!

[tehMorag]

An update
With some feedback, I was told to whitelist items ('Partition Storage' button) on each ME Storage Bus.
It has come down quite a bit!

[Funwayguy]

You need a proper JVM profiler sample for debugging this. LagGoggles is notoriously bad for blaming the wrong mods in these cases.

I suspect this has more to do with how AE is searching available crates rather than the crates themselves.

[tehMorag]

This the correct file? (used https://minecraft.curseforge.com/projects/sampler to get it)
http://puu.sh/DorHl/adbf882589.nps

---

LagGoggles does show AE2 with the slowdown, as well the crash log has your name in the first line.
Was also directed to post here from the SF4 discord after trying to figure out what the issue was.

Could do a cross post to AE2 if needed in the future.

[Funwayguy]

Found something that may help but it has more to do with shipping containers than the crates. Will give it to the SF4 debug team soon to test my theory. The actual issue may have more to do with item network scaling for the shear number of individual containers. If so then there's only so much I can do on my end.

[Funwayguy]

I've put version 0.0.24 up on Curse Forge and so far testing has been going well on the SF4 test server. Not sure when it will be released in a SF4 update but if you can test it yourself if you like.

[PiggiesGoSqueal]

An update
With some feedback, I was told to whitelist items ('Partition Storage' button) on each ME Storage Bus.
It has come down quite a bit!

I know this is an old post but would someone be willing to tell me: is there a way to reduce lag without requiring players to whitelist items? Generally server-side methods are better for servers than expecting players to do things in an anti-lag way as most will be unlikely to choose efficiency over laziness/convenience no matter how small the effort is to do so.