So we had a Pre Launch / Test Server running for GTNH for almost 1 week. This was considered a soft launch server if things went fine and a total Alt + F4 if things failed miserably and sadly that they did…
The good news is we were prepared for worst case scenario by warning everyone ahead of this that this might happen. Not only that, but GTNH 18.104.22.168 is being delayed a bit due Real Life Issues outside of the GTNH universe. Because of this it feels like we are not being hit as hard and we felt the issues for sure during our 1 week of testing.
The First 3 Crashes:
The first biggest issue that came up was a really rare bug that apparently only we have reported it twice in the history of GTNH that caused our server to crash none stop when a user loaded up a certain chunk in the world.
java.lang.ClassCastException: gregtech.common.blocks.GT_TileEntity_Ores cannot be cast to net.minecraft.tileentity.TileEntityChest
So what happen here is during World Generation at some point a GT ore and a Vanilla Chest from a Spawner Room decided to have a fight and unfortunately visually the chest won but functionally neither won at all. In Crucible / Thermos the server seems to crash when the chunk loads and with Forge itself it seems to only causes kick from the server via Interaction with a possible crash if you interact with it in other ways like a hopper.
We learned that this bug exists in Vanilla with a VERY VERY rare chance of say a spawner and a chest colliding and causing strange interactions like Pig Spawner forming due to damaged NBT Data. In Our case GregTech Ore’s are actual title entities. You can have Million+ GT Ore’s in a single Region all acting like Tile Entities and all it takes is a rare chance of a chest / spawner fighting over the same spot..
We also think Chunk-Pregenerator might of has somethings to do with it as well. I don’t understand the ins and outs of generation layering, etc. But due to the Brute force nature of using Chunk-Pregenerator it might of had a play in this as well.
Say No To Cotton:
Our next issue took us most of the week to sort of figure out the issue that was causing our server to lag the players out and even stall the entire instance. The Closest thing we figured it out and were able to repeat was one the players Cotton Farms and picking said Cotton.
The Farms Specs were quite simple. It was a 9×9 Cotton farm with Water in the Center of the farm using Natura Cotton. When the user picked their Cotton Farm the server in many times would stall out without giving us any kind of details. We ended up running a bunch of profilers, debuggers, etc. Just to come up with pretty much the same answer.
Entity Collision Box Issues… Most people outside our server said someone is spamming too many items, etc and acting like it was a normal day in the Minecraft Server world… Me sitting here thinking WTH! I been running Minecraft server for now over a decade and I know the difference in someone blowing up massive chest array of items and destroying the TPS vs a full out stall. Not only that our entity counts never actually went that high.
You can see with the above timings graph with the Orange Spike being our TPS that we only had 2997 Entities world wide at the time. Even then that list did not correspond to the 5 players online at the time who had under 100 entities at most showing up. So I 100% no it was not due to, too many items on the ground in the world to cause said stall. Not only that it was always a instant hitting stall and not a build up to the stall it just hit hard and many times would never recover from the stall. If it did recover it would drop back down instantly and not slowly recover.
I still can’t tell you exactly what causes it or even how to fix it. My guess is the Cotton farm interaction and something in the area help produce a flaw with the world gen and the right conditions to make the world go nuts.. I personally had issues on my land where I could not place a sapling down on Dirt if there was missing block under that dirt block. Not saying it’s related in anyway to this other issue, but there was some strange things going on. I don’t know if will even fully understand the issue.
The Inside Out World
Our Last and maybe biggest issue when it comes to the visual pleasing world and places to Mine is our World Generation.
We tend to always use Chunk-Pregenerator by Speiger and he does an amazing and fine job with it. That being said even though he is constantly working on it even rewriting big portions of the code, it never seems to play correctly with 1.7.10 Minecraft.
The above image is a perfect example of what is happening from time to time. Now we spoke before that World Generation happens in different layers. Like I said I don’t know the ins and out on how it works and what order but just imagine the base world is created without caves and ore. Then another layering happens to add say caves and then one to add the final details like ores and trees.
So from the above image we are finding that those regions / chunks are damaged beyond repair and usage. Where we thought is was surface cosmetic damage also ended up breaking the GT Ores under them as well preventing the land from being useful…
We also fear this might of created other issues and possible even affected our Crash & Stalling issues that had plagued our server. We might not full know all the answers and almost everyone told us it was this issue or another, but we could never find an actual solution. One of the Crucible devs suggested our world is just bad and we should regenerate it. That being said honestly I think with Chunk-Pregenerator the problem will just happen all over again…
So what is the Solution and Conclusion?
We start fresh and wipe the server. This time we don’t use Chunk-Pregeneration nor do we use any other tool that progenerates the player outside of the player entity. We believe this alone will prevent any major issues and bugs from happening with the world Generation or at least resolve the massive cosmetic scarring of the worlds surface and ore generation issues under the scars.
We did our best debugging and testing and repeating these issues. Sadly we don’t have all the answers and a lot of developers we spoke too are kind of puzzled at it as well. It seems we like to attract the more rare issues when it comes to Minecraft bugs and we don’t mind that normally as long as we can find a solution to our problems.
So our Solution is this handy program that DvDmanDT had modified for his own personal use and is also one the GTNH Developers. The Program is called PregenTool and has quite a few nice options and uses rcon to teleport an actual player around the world.
The above image is an example of PregenTool live in action as I’m writing this blog post out. What it does is by default start the character at 0,0 and starts teleporting them every second 80 blocks away in a circular pattern. If the TPS increases past 50ms it will then pause the teleporting for 10 seconds and check the tps again and try again. If there has not been a response from the server after a while maybe a Minute? it will go ahead and stop the server and resume again after.
The tool is not perfect. For example if the player stalls out it keeps running even though there is no player existing. There are mods to auto log the character back into the server, but I have had nothing but bugs and issues with those as well. The one makes you click the auto relogin button physically and another one actually just breaks and goes to a relog loop even if you set to 120 seconds and are in the server it will actually relog again into the server and disconnect you haha.
The best settings we are finding so far and are constantly adjusting them is 2 second teleport jumping vs the default 1 second. This helps it catch up a bit faster then normally but is never perfect and can still stall out. Thankfully the tool also has a Start / Start Rect option so you can just restart the entire tool and set it back to your last location that you had mapped with jmap on the client side. The picture above actually shows these settings.
Another way that Might work better and I will try it if I still have issues and maybe before bed is to use a shorter jump distance say 40 blocks or even 20 blocks, but increase the speed of the jumps themselves.
In the end so far we generated out to 4,000 x 4,000 and the terrain is perfect without any visual errors at least via the map. The best part is even the Witchery Walls are showing up now that you normally see in single player or servers that don’t pregen their worlds.
I hope you enjoyed reading my broken English and enjoyed the little history of our server. We hope to have our new server up and running by the weekend though it might be delayed at were about 1/4 of the way there in the generation and we still will need to build spawn / bunkers and do a few other checks and changes. That being said will be ready when we are ready and will do the best to our abilities.
Thank you to the 13 players who checked our server out during this week of now what we call testing!