TPS Drops drastically

We have had TPS issues in the past when a certain amount of players join the TPS drops to atleast 14-15.
Now we are experiencing more TPS drop with low amount of players online!
My timings report:
NOTE: At times there is 20TPS but it does drop, even tho there are same amount of players! Sometimes it even runs fine with more players and sometimes it drops with less players.

Atm we think its the NPC’s with Holograms in spawn that are causing the issue but would like to get your opinions on this.
If anyone have any suggestions or tell us what is the main issue causing the TPS to drop.
We would gladly appreciate it!