Welcome to Keen Software House Forums! Log in or Sign up to interact with the KSH community.
  1. Hello Guest!
    Welcome to the Bug Report forum, please make sure you search for your problem before posting here. If you post a duplicate (that you post the same issue while other people have already done that before) you will be given a warning point which can eventually lead into account limitations !

    Here you can find a guide on how to post a good bug report thread.
    Space Engineers version --- Medieval Engineers version
  2. You are currently browsing our forum as a guest. Create your own forum account to access all forum functionality.

[all versions] [all modes] Keen SWH internal Testteam/Tests not efficient

Discussion in 'Bug Reports' started by scorpnoire, Feb 4, 2017.

Thread Status:
This last post in this thread was made more than 31 days old.
  1. scorpnoire Apprentice Engineer

    Messages:
    142
    The Keen SWH internal Tests seems to be inefficent reg. delivery quality.
    Bug reports of critical issues for stable branch after the 172 relase exceed a sensible amount of bugs which should have been found in dev branch versions before.
    Several issues found seem to be based on changes which had not been released before on the dev branch but got released directly to stable branch.

    Possible solutions:
    - more rigid policy reg. code freeze/release of commits/changes on stable branch (only after decent checks/testtime in dev branch)
    - raise unit test coverage/manual testcase coverage for both branches
    - specific test time period on stable branch before official release


    Open for discussion/brainstorming to find more reasons and solutions to ensure stable branch stability and quality.
    Lessons learnd for each stable release?
     
  2. Tylus Apprentice Engineer

    Messages:
    427
    Well there were bugs like the turrets shooting at nothing bug that was found in dev branch but still made its way to stable.

    I think we need more people and especially servers playing on dev branch so we can find bugs more recent.

    but that of course only works when the bugs will be fixed in stable then.
     
  3. scorpnoire Apprentice Engineer

    Messages:
    142
    Hmm... I already work professionally as Defect Manager. I really do not want to be a professional tester/testmanager in my spare time too. ;)

    Also: a known issues list for not fixed bug like mentioned turret bug would be nice as an addendum to the change list, too.
     
  4. DaDj:D Trainee Engineer

    Messages:
    11
    1. EA -> we are the testers.
    2. Stable doesn't mean that its "has less bugs" it just means that the version is stable(as in less updates)
     
    • Disagree Disagree x 2
  5. MajorTom Apprentice Engineer

    Messages:
    299
    The whole point of the separate builds is by the time it hits Stable, the weekly Dev builds have crushed all the major bugs that have turned up.

    That has not been the case this past month at least, and is simply unacceptable. The issue is it's not even the hard-to-reproduce or complex bugs arising from new features that would obviously take some time and effort to fix, it's the simple ones that players report over and over for weeks without a fix (turrets locking Shoot to On, small-ship merge blocks, etc) and are blatantly apparent when tested in-game for more than five minutes.
     
    • Agree Agree x 4
  6. Vicizlat Apprentice Engineer

    Messages:
    174
    I am sorry but I have to disagree on your second point. There is nothing "stable" about it if it doesn't have less bugs. Right now it's more like "less frequently updated" than "stable". What's the point of having a DEV version if all the stuff gets blindly pushed to Stable, including the newly created bugs? Aren't the old ones enough that we need to push the new ones as well?
     
  7. NikolasMarch Junior Engineer

    Messages:
    927
    the problem with people playing DEV version is there is ALWAYS a few things broken with it.... and sometimes they dont get fixed then get more bugs the next week.... good grief just thinking about it while typing.... Not interested in the slightest.... played on a couple of dev servers of some twitch streamers.... now im ok with going on DEV to 'test' things and to actually 'bug hunt' but when im playing.... nah not interested in playing alongside the bugs, some can be gotten around, like the current batteries not working, dont use them :D easy... landing gears however, BOOM.... again... another work around... leave the ship on a connector... wheeled vehicles are hit n miss for various reasons, server quality and ping/MP etc but for the most part in MP they r not gonna work.... and ships still crashing glitching through the ground for 9 months now? its getting a bit too much for many people. so you cant use batteries, cant use wheeled vehicles cant crash a ship and do a survival on that, cant use landing gears....

    but yeah lotsa bugs recently have made their way to stable in the last few weeks... things are going downhill at the moment, but its still 'in the first week' since the Stable update so i expect stable to be broken.... but i also expect or hope for it to be fixed by the next thursday or shortly after... as is the whole point of 'stable' being called 'stable'

    im looking into testing out the older 32bit compatibility version for its MP playability, as its in the sweet spot After they fixed the 3 month long planet mining crash bug, and Before adding that oh so game killing Client side prediction that is still screwing us over now, but lack of compatible mods for it is a bit off-putting...
     
  8. NoThanks Apprentice Engineer

    Messages:
    458
    http://forum.keenswh.com/threads/a-heartfelt-plea-to-developers.7392077/

    Join the club. Getting really sick of them doing as little testing as possible ( None?) before porting changes to the STABLE branch. Especially changes that leave it difficult/impossible to play for a month at a time. They keep doing it like they are somehow magically unaware of the multitudes of bug reports about that very specific thing. Seriously? What is the point of the STABLE branch if you are just going to port well documented and game ruining bugs straight into the STABLE branch forcing those of us who don't want to opt into the weekly mishmosh of nonsense to deal with game breaking bugs for weeks at a time.

    I get how important it is to make the blocks LOOK good. And how critical it is to keep the MOD community happy with these innumerable tweaks and additions that cater solely to them... But some of us just want the CORE game to work. It has worked in the past. It has been downright playable in the past. So we know its possible. Perhaps it's just not profitable.

    Why cure cancer when cancer treatments are a gold mine?
     
    • Agree Agree x 1
  9. NikolasMarch Junior Engineer

    Messages:
    927
    while i agree with some of this, and some i dont, there are occasions when bugs elude them, like the 3 months it took to fix the planet mining client and server crash bug, i think there should have been a roll back in this case.... but thats a whole separate story, its possible they havent been able to fix them in time.

    But things do need to change, like if there are more bugs/broken game elements in DEV than there are in stable (which is usually the case every week) at the time of Stable's update... then dont update it, dont push this broken version to stable, it's supposed to be stable, hence the name... nice n simple method of preventing more bugs reaching stable!
     
  10. FlakMagnet Senior Engineer

    Messages:
    1,551
    Then...really...you should stay away from early access titles, because that is exactly what it means.

    Now....as far as stable/dev goes, my understanding was that updates from dev should only go to stable once the bugs are ironed out. There is no deadline to update stable. When new code goes into dev, however, you would expect bugs.

    Trouble is...it's never that simple in real life, and when you have a massively complex pile of code, it's hard to tell what impact each change has. If there are certain elements that caused issues in stable, 5 versions down the line it is very tricky to tell whether the code has been fixed, or if other changes to the code have rendered the bug unnoticable. There are also the stable branch players wanting the nice bits in dev, encouraging the devs to move them over earlier than they should be. Plus it increases the number of players 'testing' the code.

    My view, really, is that the stable/dev thing has had it's day. It's not helping. Just one version would be better.

    There are otrher suggestrions for handling updates, like being able to go back to previous versions, or simply making updates an option....but these make bug tracking even harder, and fragment the community into muiltiple versions. If they mess up ...say..batteries in 01_173, and I go back to 01_172....they lose testers on 173, and servers get issues. Now 01_174 comes out and there are issues with wheels.....do they make 172,173 AND 174 available? or just 172, as it was OK. Who decides?

    The game is in early access, so expect bugs. I just think calling a branch stable that is anything but stable causes more issues than it solves and raises expectations.
     
    • Disagree Disagree x 1
  11. NikolasMarch Junior Engineer

    Messages:
    927
    DEV should be fully, or mostly bug free when they push that to stable, but i think there is a problem with how it goes to stable.... DEV should be turned into stable, with the obvious alterations to its branch name and the Tag shown in the bottom right etc etc, but i dont think its being done that way, it would save having to 'support' this extra version called Stable if it was being done this way, i have seen signs of this method of Dev>stable not being done, bugs popping up on stable on its update day that werent in dev the week before etc....
     
  12. gimmilfactory Junior Engineer

    Messages:
    523
    I'm not starting fights here. But that is biggest joke ever. If I was a tester we wouldn't have half these bugs because I'd make sure they'd be heard and not ignored as they have been.
    Testing it is one thing. Getting the report together is another. Submitting it to the right person is a whole other. Then they have to care.
    It is a big process that is a joke, because you call us testers. And I can show you 3 different bugs that have been submitted for the last 4+ weeks with no official person saying "Hey thanks for finding this, we're looking into it", or anything.
    Just total ignorance and pressing the game forward no matter how buggy it is. Even the stable branch is a joke.
     
    Last edited: Feb 12, 2017
    • Agree Agree x 2
Thread Status:
This last post in this thread was made more than 31 days old.