Welcome to Keen Software House Forums! Log in or Sign up to interact with the KSH community.
  1. Hello Guest!
    Welcome to the Bug Report forum, please make sure you search for your problem before posting here. If you post a duplicate (that you post the same issue while other people have already done that before) you will be given a warning point which can eventually lead into account limitations !

    Here you can find a guide on how to post a good bug report thread.
    Space Engineers version --- Medieval Engineers version
  2. You are currently browsing our forum as a guest. Create your own forum account to access all forum functionality.

[DS] Recurring simspeed freezes after three hours

Discussion in 'Bug Reports' started by gesp_shad, Aug 4, 2017.

Thread Status:
This last post in this thread was made more than 31 days old.
  1. gesp_shad Apprentice Engineer

    Messages:
    222
    Howdy!

    Currently our DS is suffering from the simspeed getting locked to 0.08 after about three hours uptime, which makes the game unplayable.
    I was browsing through the log files, but there is just nothing about it. The problem occured after yesterday's hotfix, though.
     
  2. Sir_Moodz Apprentice Engineer

    Messages:
    377
    My server has this for months now, it happens after x amount of time and x becomes smaller the more people play on the server.
    Possibly related to the size of your save
    --- Automerge ---
    Best way to finally solve this annoying bug, is for them to start a server with a world known to cause this slowdown and then have their diagnostics running. Then they let a lot of players join and wait until it "happens"
     
  3. gesp_shad Apprentice Engineer

    Messages:
    222
    Odd, we didn't have had this problem before the hotfix. It just happened again after 3h30min with 5 players online. But in our case the player count is negligible.
    Our savefile is currently 31MB in the download.
     
    Last edited: Aug 4, 2017
  4. Erebos89 ME Tester

    Messages:
    1,137
    Hi,

    thank you for report. I will ask few things.
    Sim speed drop happened even when there was small number of people? For example 1-5?
    Restarting DS is fixing it until sim speed drop happen again?
    What mods are you guys using on DS?
    This start happening after update/hotfix or was it happening even before but its now much worse?
    Is possible to send us your DS save for testing?
     
  5. gesp_shad Apprentice Engineer

    Messages:
    222
    Yes, it drops regardless of the player count.

    Yes, restarting the server will fix it temporary for another 3 hours +/- 30min.


    The problem started with yesterday's hotfix 0.5.13.FB743A. We had no problems at all before that fix.

    You guys should already have a save file of our server. I sent it to you regarding this thread: Odd GPU load/performance behaviour.
    But if you need a fresh, more recent one, I can send you another one via email.
    Btw, Erebos, we are talking about the same server you started playing on yesterday ;).
     
    Last edited: Aug 4, 2017
  6. Sir_Moodz Apprentice Engineer

    Messages:
    377
    Interesting thing I just noticed: I joined my server and the bug happened. It might happen after x players join your server.
    @gesp_shad I am 95% certain the hotfix was not the cause, but I believe it could be related to the age of a world. How old is your world and exactly how big is it on disk?

    I have had this bug for several months, but it has most of the time not been noticeable because of the low player count on my server. These last days the player count has gone up again and its happening again. 2 times already this week.

    Ultimately there can be many different causes to this and finding out what will be very hard.
     
  7. gesp_shad Apprentice Engineer

    Messages:
    222
    I am not sure how old the world is exactly, but at least older than 3 months. The size of one backup folder is 240MB.
    But I dont get why this problem arroused right after the hotfix? That can't just be a coincidence...
     
  8. gesp_shad Apprentice Engineer

    Messages:
    222
    I set up a dedicated server with our latest save file on my pc today. It is now running for 9 hours without any problems, but also with no players online (pause on empty is disabled). Meanwhile the public server keeps freezing after 3 hours :p
     
  9. whyiamhere Trainee Engineer

    Messages:
    6
    Erebos89:
    Hello, any news on this problem?
    Dozens of people who are currently playing on this server would like to hear any information. Thanks
     
  10. Erebos89 ME Tester

    Messages:
    1,137
    Hi,

    I am still testing it.... Now I need to wait because I crash it. I hope that tomorrow I will know more.
     
    • Like Like x 2
  11. Sir_Moodz Apprentice Engineer

    Messages:
    377
    Best way to test this is by having several bots join and leave your server. Then you will have it happen after like an hour or so
     
  12. gesp_shad Apprentice Engineer

    Messages:
    222
    Our test scenario with a new world and the same mods is now running for 4 hours without any problems. There were constantly 15-20 players online so far.
    --- Automerge ---
    After 8 hours of testing on a new world, I would say, the old world is dead, unless you guys can come up with a miracle-kind-of-fix.
     
  13. gesp_shad Apprentice Engineer

    Messages:
    222
    Our new world is now running for 10 days and has a size of 140MB already :eek:.
    Anyway. Last night, and 20 min ago, we had the freezing issue again after about 5 hours. Today, I was on the server myself and were noticing the following:
    The simspeed freeze happened right after one player disconnected. I was browsing through the log file and noticed, that at this very moment, the memory usage went up to 4GB and dropped to 3.5GB right after he disconnected. Average memory usage is 3.5-3.7GB.
    Any thoughts about this?

     
  14. Erebos89 ME Tester

    Messages:
    1,137
    Hi,

    thank you for info. I will pass it to programmers. I hope that 0.6 version will fix some these issues.
     
  15. gesp_shad Apprentice Engineer

    Messages:
    222
    Here is some more input for you.
    Our server froze again while autosaving. While saving, a connection to a player timed out.
    I am not quite sure, if that has something to do with it, but I am most certain, that the freeze happened during the autosave.

     
  16. gesp_shad Apprentice Engineer

    Messages:
    222
    I talked to @Deepflame and it seems, some mods are causing memory leaks (see post #13).
    I was watching our logfiles, and every time the freezes happen, memory usage goes up and/or drops dramatically.
    Can anybody confirm this behaviour with @Nejispo 's mods Wooden Gallery and/or Roof sytem slope 30° and more?


    --- Automerge ---
    So we removed the mentioned mods and.... it happened again.
    Nothing suspicious in the log but one player connected and another disconnected when it happened.
     
  17. Nejispo Trainee Engineer

    Messages:
    31
    Hi,
    thank you for information. And advice where to look for the problem would help.
     
  18. gesp_shad Apprentice Engineer

    Messages:
    222
    We removed peaceman's Apple Tree mod and it is running straight so far since yesterday evening.
    I'll keep this thread updated, if anything changes.
    --- Automerge ---
    Removing the apple trees was a slight success: Freezes still happen but less often.
    My idea is, that the combination of these mods are key here. We are a very populated server and all of the three mods are heavily used and may cause this issue to happen at some point (Wooden Gallery and 30° Roofs still have huge memory leaks).
    We will continue testing.
     
  19. gesp_shad Apprentice Engineer

    Messages:
    222
    I really don't know what to do anymore. I loaded a backup with all mods back in place and the dedicated server task is now using up to 10GB after 5-6 hours. The log file does not report more than 4GB, though.
    Today we had no freezes yet, but that might be because of the early daytime. This will probably change later today (scheduled restarts are every 6 hours).
    Last thing I want to test is to run the current savefile on my own machine, which has a little bite more *oompfh*, to see if it is a problem in our server environment. But if it runs straight it could also just be a matter of time until this issue starts to come back.

    I personally think, the game/net code just cannot handle the world size with its player activity anymore. I mean the mods may be not helping it, but I don't think the issue itself lies somewhere in the mods.
     
  20. Erebos89 ME Tester

    Messages:
    1,137
    Hi,

    on 0.5 there are lots of issues with multiplayer. Right now we are working on some fixes on 0.6 that should help. Can you send me again your save for more testing.... Programmers want me to test changes on your save.
    Sorry for inconvenience.
     
  21. gesp_shad Apprentice Engineer

    Messages:
    222
    Here is our latest backup: Click

    Nice to hear you are working on the multiplayer, but I guess we won't get any information on when you will probably release 0.6 ;)? Would be good to know if a server wipe in the near future is reasonable or if we should wait for 0.6...
     
  22. gesp_shad Apprentice Engineer

    Messages:
    222
    No freezes reported during the last two days, but also very low traffic on our server (max. 5 players at the same time).
    Seems like the same issue on the old save file: world size reached a certain threshold and starts freezing depending on the traffic on the server (although, the old world was more than double the size...)
     
    • Informative Informative x 1
  23. gesp_shad Apprentice Engineer

    Messages:
    222
    Happened again today, while I was connecting to our server. I was downloading the world when the download speed plummeted at 50%.
    Also it happened more often these days. We had 10 players online average.
     
  24. Anteramian Trainee Engineer

    Messages:
    3
    We are having the EXACT same problem. Scheduled or forced restarts bring our sim speed back up to (1.00 - 1.05.) Three hours in we hit a wall. (.17 - .20) server sim speed is stuck. We dropped ALL mods and wiped the world, started in vanilla. After about 3 weeks we encounter the same issues. The current era we are running a banner expansion mod for more designs, the Trade Post mod and FT by road. I am guessing it could possibly be related to the trader as is spawns in non-native game items. I can tell you this is our third era running into this issue. We have one of the most populated servers - [US Cebtral] Valanor.net 08/27/17, The date does NOT reflect our last wipe accurately. Is there any information you can provide to give us any idea what is being done to resolve these issues? My solid player base of around 30 - 40 players are getting disenfranchised with the game and leaving ME all together. Please don't let this die on the vine as Keen is so apt to do.
     
  25. gesp_shad Apprentice Engineer

    Messages:
    222
    Finally someone else with this problem. It is the curse of highly populated servers, it seems.
    Have you tried to run the world with NO mods? Was it still freezing? Because we are using the Trader and Fast Travely by road mod, too and I wasn't testing them yet. But I doubt that it is caused by them.
     
  26. Anteramian Trainee Engineer

    Messages:
    3
    Ok so wiped and had the hosting services move our game to another physical machine with new install so there is NO possibility of old files finding their way back in a mucking it up. So far we are one week in and all is going well. Pure vanilla only adjustments are settings in the base game. Larger inventory, limited fast travel with a cool down. We get lag when 15+ players are on, but it isn't crippling. 21-25 players and the door/container lag become horrid. I will give an update at the 2 week mark, but so far so good.
     
    • Like Like x 1
  27. ibisgrunk Apprentice Engineer

    Messages:
    184
    the hosting company is where the devil in the detail is imho, Keen no doubt has a role - im not being naive but let me tell you about your hosting company.... you are being stacked in a multi-tenant VM host rack space. your CPU* usage is determining the resources available, when the server isnt being used it gets put on low resources. we hosted a PC in-home and ran the server better than the hosted company we use now HOWEVER concurrent users and logging in i believe is too much for a home PC with a single network card dealing with the server traffic. whereas we can host on a better system we need their networking resources to handle the users, again imho. the tech for my company said the i7 is a better CPU than they are using their Xeons, stuff like that...

    we used pingperfect because it was the only hosting company that had upgrades for putting the ME server on SSD (+$5) and Fast Performace (ambiguous term implying to me a max throttle +$15). Most hosted servers are charging $1 per user and folks get a 20 pop server. There is limited server energy in those basic packages, and imho this game needs a lot of horse power when siege and folks are building clustered villages. i am not endorsing pingperfect but just saying after due diligence they were the ONLY company hosting ME that offered these upgrades... even with this Max package i had to contact the host company first week because the service was awful, they in turn said they moved me onto a more powerful rack (note here is where your comments rang the same bell - if you complain they will move you to a less busy VM stack to host you, more to that? even with my Max package i too had to complain for better performance).

    the best speed global server is PlanetsNoLag out of Belgium! see prior thread talking with the server owner, and creator of the Trading Post mod, and how to host a fast server... peaceman was kind of nice twice to discuss maximizing ME server hosting options. his server has never lagged or shown any signs of too many active users, just saying, and we had folks in NAmer and SAmer logged in to his server. so it can be done!

    if i could leave with an opinion, imagine if you can that Keen is half the problem the other half is your online hosting company's weakness. as the chef is still cooking in the kitchen called Keen, we the early game adapters can focus on the hosting side of it all.

    we have autorestart and log-in-daily and see the server drift to a unnatural 0.13 SIM speed range. we are going to walk away from ME for a while due to the SIM speed issues and that fact that not more than 2-4 folks can actually be seiging. when Keen solves their performance issues i will gladly put a DS onto a $100 a month hosting rig.

    (*nobody murder me on details of hosting an application, just being general here as i have configured, architected and worked in the server hosting space)

    [ref. https://forum.keenswh.com/threads/t...in-servers-is-fixed.7395879/#post-1287073093]
     
    Last edited: Oct 16, 2017
  28. Anteramian Trainee Engineer

    Messages:
    3
    Hit the same wall in the last two days. Lots of siege/battle equipment in the world but with 7 players on the sim speed slumps to .7 and after about 4 hours tanks and sticks at .13. Set the server to reset every 4 hours now. not gonna put my community through another wipe before the update... which, do we have an ETA for that?
     
  29. gesp_shad Apprentice Engineer

    Messages:
    222
    @Anteramian, I found out how to reproduce this bug, at least in our case. One of our players was building a castle as single grid wich counts more than 7000 blocks. Somehow it reached a certain threshold, and every time you add another block to this grid, the simspeed freezes. But that could only be one of the issues. Voxel data and grid/player densitiy in a small area could also be part of the problem.
    We just wiped our server today and are doing siege equipment and castle siege tournaments while we wait for 0.6
     
    • Agree Agree x 1
  30. Ghostickles Senior Engineer

    Messages:
    2,077
    Playing on a DS solo for many hours I found the same thing happens. After placing many blocks I would set a single block and sim would suddenly tank. Had to then restart the service. This happened a couple times a day in some cases.
     
Thread Status:
This last post in this thread was made more than 31 days old.