Help needed with memory leak


#1

Hello.

I suspect that my 1.12.2 server is having memory leaks. I can launch my server with 10 GB ram and it will experience no lag at all. After a few hours it will start to experience delays/hiccups when the garbage collector kicks in. The free memory will go near zero and stay at values getting closer to zero. Every time the GC runs to free up memory, it becomes less and less. Eventually we experience constant lag spikes. Server timings is not helpful, I am very certain that this is a memory issue. I can see that the lag spikes correlate to the time the free memory goes to a value near zero and attempt to free up memory.

These are my startup flags: (Used those suggested by Aikar)
java -Xms10G -Xmx10G -XX:+UseG1GC -XX:+UnlockExperimentalVMOptions -XX:MaxGCPauseMillis=100 -XX:+DisableExplicitGC -XX:TargetSurvivorRatio=90 -XX:G1NewSizePercent=50 -XX:G1MaxNewSizePercent=80 -XX:G1MixedGCLiveThresholdPercent=35 -XX:+AlwaysPreTouch -XX:+ParallelRefProcEnabled -Dusing.aikars.flags=mcflags.emc.gs -jar craftbukkit.jar

I collected some heap dumps. One around the launch time of the server and one about 5 hours later when the issues started to occur. I used eclipse memory analyzer to look at the heap dumps.

Image album: (Couldn’t post more links because new user)
https://imgur.com/a/iY45o3w

The object that caught my attention is:
“Sun.misc.Launcher$AppClassLoader @ 0x54a9a6e88”c

This object went from 285.4 MB in size to 2.2 GB in about 5 hours. All the other objects didn’t grow significantly like this one did.

So, I look at the dominator tree to see what it is

“class net.minecraft.server.v1_12_R1.CriterionTriggers @ 0x553cdadb8” takes up 65.07%
Notice how it creates a large amount of HashMaps.

Is this issue caused by the advancement and statistic mechanics in Minecraft? How can I avoid this?

I tried to delete all advancement and statistics from the world folder. And I set my spigot.yml to these values:

advancements:
  disable-saving: true
  disabled:
  - minecraft:story/disabled
stats:
  disable-saving: true
  forced-stats:
    achievement:
      openInventory: 1

Now advancement and statistics don’t save but the issue still occurs. Is the source of the problem really caused by Minecraft advancement/statistics? How can I disable those mechanics? Is it possible to disable the creation of those HashMaps? Am I using the java memory analyzer correctly? Any help would be highly appreciated.

Thank you for your time.