Welcome to Keen Software House Forums! Log in or Sign up to interact with the KSH community.
  1. You are currently browsing our forum as a guest. Create your own forum account to access all forum functionality.

Refinery & Assembler Inventory Management

Discussion in 'Community Creations' started by Derek Lewis, Jul 6, 2018.

Thread Status:
This last post in this thread was made more than 31 days old.
  1. Derek Lewis Trainee Engineer

    Messages:
    28
    I've written a couple in-game scripts for managing the inventory of ingots and components by controlling refineries and assemblers. I know there are others out there, but I like the simplicity of this approach. It's low effort/cost to get going. For each of them, all you need on top of at least one refinery or assembler is a programmable block, and 1-2 LCDs.

    Refinery manager: https://steamcommunity.com/sharedfiles/filedetails/?id=1432879178
    Assembler manager: https://steamcommunity.com/sharedfiles/filedetails/?id=1426872621

    You tell it how many of each ingot or component you want to keep "in stock", and it takes care of the rest.
    For assemblers, it queues production orders. For refineries, it transfers ore into their input inventory.
    If there are more refineries or assemblers than there are things needing to be refined/assembled, it'll double-up on some. eg. It'll use multiple refineries for refining iron.
    The status is shown on wide LCD screens.

    Configuration instructions are in the top of the code. Just add this script to a programmable block, copy-paste the sample config (in the code comment) into the "Custom Data", specify your assembler, refinery, and LCD block names, and adjust the desired quantities to whatever you want.

    Refinery manager LCD display:
    [​IMG]

    Assembler manager LCD display:
    [​IMG]
     
  2. Ronin1973 Master Engineer

    Messages:
    4,468
    How is it on server play? Have you run any stress tests? How many grids running each script does it take to start denting the server sim speed?

    There are definitely other scripts to do this... but if yours is really low CPU intensity, it would be a great alternative to TIM managed production.
     
  3. Derek Lewis Trainee Engineer

    Messages:
    28
    Unfortunately I don't know any of those yet. I haven't run stress tests. I would like to find out and improve it. I have a few ways in mind I could improve it's performance if necessary.
    Is there an easy way to tell how much execution time scripts take?
     
  4. Ronin1973 Master Engineer

    Messages:
    4,468

    Create a sample space ship and start copying it.

    I would copy the ship with the programmable block off and see when your sim speed starts to dip. Use that as your baseline. Then repeat the process with it on and see how many ships it takes to get the same dip. It's not exactly accurate but if the number is closely comparable then it's a great script for multiplayer.
     
  5. Derek Lewis Trainee Engineer

    Messages:
    28
    That sounds pretty straightforward. I'll try that when I get home next week. I'm away right now and the laptop I have can barely play SE with all the options turned down/off. ;)
     
  6. Ronin1973 Master Engineer

    Messages:
    4,468
    Perfect potato for Xbox testing. :)

    Thanks for your work. If it's multiplayer friendly I'm definitely going to use it on a server.
     
  7. Derek Lewis Trainee Engineer

    Messages:
    28
    hahaha :D No really, it's bad. I have to turn shadows off, which then makes flashlights not work, so I have to install an increased ambient lighting mod... and it barely even breaks 25 FPS anyway. Without any fancy scripts running.

    I'm not sure what's involved in being "multiplayer friendly", so I can't really say whether it is. Are there any resources you could point me at regarding that?
     
  8. Ronin1973 Master Engineer

    Messages:
    4,468
    How CPU intensive the code is. I would consult @Malware as he's the subject matter expert around here. He may even be able to give you some pointers if there are potential issues or some optimization. He's pretty friendly, helpful, and has years of experience. I believe I just summoned him. :)
     
    • Agree Agree x 1
  9. Malware Master Engineer

    Messages:
    9,032
    Hello. I'd be willing to take a gander, but I'd prefer not to have to actually subscribe to and set up a world just to look at the code. Do you have an upload somewhere else?
    --- Automerge ---
    Otherwise I can point you here for more general performance advice, but it's by no means exhaustive.
     
  10. Derek Lewis Trainee Engineer

    Messages:
    28
    Oh, hi. I've been meaning to try out your dev environment. Been doing everything in-game so far. Yuck.

    I tossed my current in-progress code for the assembler manager here:
    The refinery manager is very similar, but does inventory transfers of ore instead of queuing production.

    A couple points maybe worth mentioning:
    - I'm a software dev, been doing Java for 18 years, but very little C# ever.
    - I'm reading your "Do's and Don'ts" now, I haven't applied any of it to the code I posted above.
    - I already have a general plan for a rewrite/refactor that I think is likely to drastically help performance, summary here:
    --- Automerge ---
    Oh, I also meant to ask: other than performance concerns, are there any particular things that shouldn't be done in code that's meant to run in MP or DS?
     
  11. Malware Master Engineer

    Messages:
    9,032
    First of all, you don't need FindThisPB. There's already a property for that called Me. Secondly... You really should read that page I linked to you, because you're committing some of those sins ;)

    Be careful with string manipulation. That causes allocation galore. You should avoid your lpad/rpad things. I mean, for the most part, since you're doing work over 100 ticks, it's not too bad. But imagine this hitting a server, and many people starts using it on many builds... things add up. It's just better to make tools that works directly on the StringBuilder (and reuse that stringbuilder, just clear it after it's written to the text panel)

    Text panels can write StringBuilders directly. No need to allocate a string (with ToString).

    Finally... you're trying to do one of the things that's the most difficult to do in ingame scripts: Inventory management. That becomes performance heavy fast. Any good inventory manager must divide their work over multiple ticks.
    --- Automerge ---
    You are aware of the Discord, yeah? Plenty of folks there that can help you with this stuff, in the #programming-in-game channel. Including me.
     
  12. Derek Lewis Trainee Engineer

    Messages:
    28
    Yeah, I figured I would be. :) I'll sort that out in the rewrite/refactoring.

    Ah yeah, I should be more heavily relying on StringBuilders. I'm used to Java, that does some nice optimizations to make it StringBuilders underneath anyway. ;) I didn't know text panels could take StringBuilders directly, that's handy.

    Huh. I was thinking about doing that, but hoped that with the new plan I had, it wouldn't be necessary. I'm going to try it without, and if it ends up too heavy, I can add a state machine in and spread it over multiple ticks.

    Huh, I wasn't actually. I'll check it out. Thanks!
     
  13. Malware Master Engineer

    Messages:
    9,032
    Let's just say, lots of people have been trying it without... it's not so much the script as the inventory system itself. So it will depend heavily on the build and on the number of inventories and inventory systems being worked with. Which is something you can't really predict.
     
  14. Derek Lewis Trainee Engineer

    Messages:
    28
    What I plan to do to minimize (and _maybe_ eliminate) the need to split it across ticks is:
    1. Only scan the grid for blocks with inventories every now and then, since the grid doesn't _really_ change that often.
    2. Queue enough production/refining that it'll last approximately 10 seconds.
    3. If there are no idle assemblers/refineries, return from the script right away.
    4. If there were idle assemblers/refineries, compute current inventories, queue jobs, etc.

    Combined with having separate independent scripts for refinery management, assembler management, and inventory sorting, and each of those being optional, and restricted to the grid they're on, I'm hoping it'll be manageable without any spreading across ticks. But, I won't know till I've done all that, and tested performance. I've already updated my assembler management script to use MKD-SE, and follow your do's and don'ts.
     
  15. Malware Master Engineer

    Messages:
    9,032
    Using separate independent scripts is irrelevant, I'm sorry. Doesn't matter where the code is run, it'll have the same performance impact. More programmable blocks vs more code in one has little difference.
     
  16. Derek Lewis Trainee Engineer

    Messages:
    28
    Sorry for the essay :/ I'm just trying to make sure I have a good understanding of the performance characteristics here.

    Ok I get that more code has more impact, regardless what block it's in. At some point, the overhead of the script (due to bad code or just # of blocks it has to interact with) will necessitate splitting it across ticks, I know. But, I think I may be completely misunderstanding where the performance issues with PB code are though.

    My understanding has been that basic C# stuff like List/Dictionary manipulation will be quite fast, as it's being natively compiled. So most of that (assuming there's no object allocation that leads to more frequent GC) is probably a non-issue for performance, unless taken to the extreme. Is that roughly correct?

    So that leaves interactions with the game classes, the grid, blocks, etc, as the performance issues. I would guess that most of the properties are probably low overhead. Some of the methods are obviously going to be higher overhead. The ones that I've heard specifically mentioned as the worst are to do with transferring inventory.

    If a game has 1000 assemblers and 10000 inventory-holding blocks, and there is a script, or a collection of scripts, trying to keep the assemblers busy, then there are a number of ways I think it could play out performance wise.

    1. Everything on one grid, with one PB.
    Without spreading across ticks, it could take a long time to iterate the inventories and queue tasks on the assemblers, which would have a large impact on performance. When the script runs, the first thing it does it check if any assemblers have empty queues. Presumably that's a low overhead call, and even doing 1000 of those won't be too bad? However, with that many assemblers, it's likely that every time the script runs there's at least one empty, which means the script needs to check inventory to determine what needs to be produced. Clearly this is going to be slow. This would likely need to be spread across ticks.

    2. 100 grids each with 10 assemblers, 1 PB, and 100 blocks
    Assuming that invocations of all PBs with Update100 are "spread out" (ie. all the blocks don't run on the _same_ tick) then this already effectively spreads the work across ticks. Each script, each invocation, it's only checking 10 assemblers. Since they're queued up with 10 seconds worth of work, that means on average, every second each script will need to do some actual work (check the inventory, queue jobs on an assembler.) With 100 PBs, that means each second there will be 100 of those running. That could still be too much, even spread across ticks as it is. The only way I could see to fix that would be to queue up more than 10 seconds of work, so that it's having to do the "heavier" work of checking inventories and queuing tasks less frequently.

    I'm not sure how much of this makes sense without deeper context about the script itself. I've tossed the latest version here, if you fancy a look:
     
  17. Malware Master Engineer

    Messages:
    9,032
    You're getting the idea. Just keep in mind that there's lots of people who loves to build like #1. I don't, but many people do. Me, I don't get what's so great with huge ships, but to each their own :)

    Your scenario #2 isn't really a big issue. #1 though, is.
     
  18. Derek Lewis Trainee Engineer

    Messages:
    28
    Ok great, glad I'm on the right track. :) For large installations, I think larger production batches (more than 10 seconds) would help quite a bit.
     
  19. Malware Master Engineer

    Messages:
    9,032
    Maybe, but every time you're moving stuff between inventories, you need to move smaller batches and not everything at once. This is the important bit. Otherwise you might stall the server - and you get a simspeed hitch.
     
  20. Derek Lewis Trainee Engineer

    Messages:
    28
    I'm actually kind of curious how slow it would be to iterate over all the blocks in a 10000 block grid, get all the IMyInventories from them, and iterate over all the GetItems() in those, to build a Dictionary of inventory quantities. The sheer number of blocks is going to have an impact obviously, but my gut-feel is that none of those calls are particularly expensive, since none of them are doing "isConnected" checks or anything like that.
    --- Automerge ---
    Ok, so that's for moving stuff into refineries. I recognize that's going to be tricky.
    For assemblers, my script isn't moving anything, just queuing an assembler task for a given number of items. That on it's own is low cost, right? And then the game will deal with transferring the necessary ingots, and spreading that across ticks?
     
Thread Status:
This last post in this thread was made more than 31 days old.