Yesterday, I went ahead and upgraded to 1.14.4, from 1.14.3. This is what I found.
Note: I have a dedicated server, with 32 Cores, 128GB of RAM, and it’s all on NVMe drives. This server is a powerhouse and never had TPS issues with 1.13, regardless of the number of players on.
First, let me start with y background with 1.14.3 and previous 1.14 releases, had been a constant daily trial and error of disabling and enabling plugins, changing settings, to see what I could do to get the TPS up. While some things did help, even if just by 1 TPS, that is a big help in the scheme of things. Because a gain of 1 TPS here and there starts to add up.
Here are the highlights that I noticed with 1.14.2 - 1.14.3:
- The server handled 9 players just fine, but after 9 players the degradation started. The lag started to become slightly annoying at 15 players and over 20, it was unbearably slow. The TPS typically ran between 5-9, and chatting was about the only thing that was going on since breaking blocks was annoying.
- Timing reports always pointed at Entities as being the issue. But managing entities is hard and I could never see a correlation between lag when people were at horrendously large mob farms or when they weren’t. However, I did see a correlation between entities active on the server and TPS performance.
- I also disabled and modified the configuration files of plugins that came up with the highest count numbers in the Timings Report, regardless of whether it was shown to be a lag source. My theory was that less activity is better overall. However, I ignored plugins like CMI that were basically the core of the server functionality, even if they had a count.
Here is what helped to various degrees:
- view-distance: 6 // In 1.13 and earlier, I typically would run 10-12, depending on the world. This alone made having 15-22 players on bearable and playable. Oddly, the only person complaining about the view-distance was me.
- Keeping world spawns in memory has always been set to true for me in Multi-Verse Core. However, I could see with the /lag command that 80-90% of the players were always in the main world and each world had usually had a number of entities for the server to manage. I change this setting to false for all but my main world, and this made a big step toward usability with more players on.
- In bukkit.yml, I lowered the spawn-limits of monsters from 70 to 40. In fact, I could probably move this lower now that I have seen the result of 1.14.4, which I will get to shortly.
- Disabled the Citizens plugin. It always had the highest counts and oddly, when I would unload it, the performance was always slightly better than with it enabled. I don’t know why, I maybe only had 20 NPCs setup.
- Disabling floating items (aka hidden armorstands) in QuickShopsRemake also made a big difference. Armorstands are entities and if you have a lot of chest shops, this just added to the pain. One bug that allowed someone to create ChestShop with a stack of 16 items. This created 16 armorstands rotating above the chest with the item. The TPS went down to 1, and the server was unplayable. But they fixed that bug. Regardless, disabling that functionality was a huge help.
What happened after 1.14.4?
I was hoping that all my woes would be history, that TPS would go back up to 19.9/20 and life would be grand again. With 9 players online, I was able to use flyspeed 5 and fly around with a view-distance of 6 and it didn’t create any lag. But my hopes quickly faded when view-distance of 8 and 10 both created lag doing the same thing.
It seemed that even when I was in 1.14.3, that when connecting the server with a 1.14.4 client via ViaVersion plugin, that the falling through the world when you TP and waiting for chunks to load went away. In fact, overall performance of the server improved with the 1.14.4 client release. So, I decided to force everyone to 1.14.4 by upgrading the server.
Last night, at one point, we had 28 players online. TPS was around 6-9, but there was little to no noticeable lag, breaking blocks, placing them and playing. It was very surreal. The TPS didn’t seem to have a correlation between its value and player experience. Only when it dropped below 5 did anyone notice anything.
I looked at the timings report over 2 hours long the first 70 mins of it, TPS was great, even with 22 players online, and then the TPS went drastically south as we climbed over that up to 28 players. Aside from the player count itself, the real issue was obvious and it is what the Timings reports have pointed at all along, Entities.
Once our server crossed 3000 entities, TPS tanked. At 2222, it was good, at 2750, it was showing signs of weakness and at 3000, it was over. Entities peaked out about 4750. But again, the server was playable and usable and functional and noticeable gameplay lag was almost non-existent.
Knowing all this, I can see why some of things I did to get better performance from 1.14.3 worked:
- A lower view-distance means less entities for the server to manage
- Removing floating item entities in QuickShopRemake lowered that number down
- Lowered mob numbers in bukkit.yml, lowers entities
- Removing spawns from memory in rarely used worlds
I have yet to take action directly against anyone’s massive sheep, cow farm or the mob grinders like Enderman, Pigman, Blaze, etc. While I knew they contributed to the madness, they were honestly fine with lower player counts and only become issues at the higher numbers when more entities are exposed. I will likely lower all my spawn-limits more in bukkit.yml and that should allow us to get higher TPS with more players on.