1.14.4 CPU Utilization (Low TPS)

I’ll start by posting my Spigot forums post here which is what I’ll carry over (and take out the most important bits of data)

I run a small server with an averaging playbase around 15~ people, everything is perfectly fine looking until we start hitting that 15-17 player mark then everything goes downhill.

I’ve collected dozens of timings and got rid of any plugins I saw using a lot of CPU (MythicMobs, HologramAPI, TreasureChests) and have configured ClearLag on my server. It’s no longer entities or plugins taking up large portions of my timings, but instead it’s just chunks which I have done research on Spigot’s configuration forum posts to optimize my server.

I’ve done all I can, but then I noticed when I looked into my server’s task manager and noticed that yes while it does use other threads occasionally when loading chunks, there’s 1 core that always sits above 50% when with only 4-5 people on. Once there’s 11~ people on it sits at 90-100% and that’s where the TPS starts feeling pain.

I’ve moved my server off any Pterodactyl Docker containers and any restrictions that may have been set and ran it directly as a .sh with launch parameters gathered from https://aikar.co/2018/07/02/tuning-the-jvm-g1gc-garbage-collector-flags-for-minecraft/

image
https://timings.aikar.co/?id=3ddd89a60d8f4cb38a35429c03f46e61

There’s an image and the timings at the same time that had been going on.
The chunks are being loaded on other threads as you can see in that image, but 1 core is pegged.

Specs:
Xeon 1241v3 (dedicated machine, I just run my MC in a hypervisor with 6 threads allocated)
16GB of RAM allocated to the VM, I have 32GB in total I can spare
1TB Harddrive

I also observed that the pegged core (in this picture the core is number 5) will randomly split up and become multi-threaded and then the server goes back to 20TPS. Then it’ll randomly go back to going single-threaded and peg a single core. Rinse & Repeat.

Plugins:
AntiVPN, AutoRestart, BigDoors, Citizens, CombatLogX, CompatNoCheatPlus*, ConditionalCommands, ConsoleSpamFix, CoreProtect, CustomDrops, CustomStructures, DailyRewards*, Essentials, EssentialsSpawn, ExtraContexts*, Factions, Harbor, Herochat*, Holograms*, HorseTpWithMe, LuckPerms, mcMMO, MineableSpawners, NoCheatPlus, ObsidianBreaker*, PlaceholderAPI*, Plan*, PNCP*, ProtocolLib*, Quests, RandomTeleport, Rankup, renametool*, Shopkeepers, Sickle, spark, TAB, TimTheEnchanter, Vault, ViaBackwards*, ViaRewind*, ViaVersion*, WorldEdit, WorldGuard, XrayInformer*, ClearLagg

Any help will be appreciated, thank you.
I am running on Paper-159, I will upgrade it tonight when the players log off.
Edit: I have updated to the latest paper, problem remains.

Hey,

I actually had the same problem with high cpu usage, and no clue where it came from.
Until i switch from my sshd to a ssd, the problem disappeared like it was never there.

Now i have a constant 20 tps, and no cpu spikes.

1 Like

Thank you, I was looking into upgrading to an SSD under the same suspicion but I wasn’t sure how that would’ve reduced CPU utilization.
I’m assuming the CPU spends less time loading/processing chunks as it doesn’t have to wait on the HDD?

At any rate, this advice is extremely valuable to verify efforts to try to get an SSD on the machine, apparently there is one but not being detected but that can be fixed I’m sure.
Thank you MaouKami, I appreciate it :grinning:

An SSD is practically required if you want to run any recent version of Minecraft acceptably, especially 1.13 and 1.14 where synchronous chunk loading (i.e. chunks loading in the main thread) is much more common.

This isn’t actually the case. What is happening here is that the server (on the main server thread) is trying to load chunks from the disk. However, rotational hard drives must first seek to the location of the data on the disk and then read the data from the disk. This process is slow, given that we live in a world where 1 second of wait time is enough to drive people insane. As a result, the server thread has to wait while data is loaded off the disk.

A solid-state disk, however, is able to read data off a specific point on the disk very quickly. In that sense, it’s able to return the chunk data to the server thread more quickly. But even with an SSD, you will notice an impact from sync chunk loads - it’ll just be smaller.

Ultimately, the real solution is going to be when PR #2308 gets merged (or Mojang delivers massive optimizations in 1.15, but I’m not optimistic), but that’ll take a while.

1 Like

Thank you for this lovely explanation, I really appreciate this.

Is there any estimated timeframe for #2308 to come out? The post makes it look like it may be soon, I’ll be patient as I’m grateful with all the work that has been put into Paper thus far.

I’ll still be grabbing that SSD whether it gives great performance uplifts or not, can only make my server and not break it.
I did notice that the main thread gets pinned the most, but when chunks are being loaded it does distribute it quite well across multiple threads and occasionally that 1 pinned thread splits into multiple threads at lower utilization. Is that supposed to happen?

Thank you!
Edit: I see a paperclip discord attachment at the bottom of that page, it looks like that may have the ASync review loaded onto it? I’ll give that a try and see how that goes.