Welcome to Keen Software House Forums! Log in or Sign up to interact with the KSH community.
  1. You are currently browsing our forum as a guest. Create your own forum account to access all forum functionality.

Custom Importer/Editor tool - SEToolbox

Discussion in 'Modding Guides and Tools' started by midspace, Nov 24, 2013.

Thread Status:
This last post in this thread was made more than 31 days old.
  1. DuneD Junior Engineer

    Messages:
    948
    We really need the 3D Importing, being able to import your own asteroids its one of the biggest features and unless they add a tool, similar to mwmbuilder, that let us create our own asteroids, im afraid your tool wont be obsolete, so if you can spare the time I would really appreciate if you can keep that feature.

    Rigth now, the importer works, I can still import my own asteroids but for some reason it doesnt get the center position right, one of the axis is a bit off and it takes some trial and error to add something to the center.

    In another different matter, I noticed that the load time with really big asteroids ( more than 800m of diameter ), has lowered dramatically. Merging "spheres with terrain", I managed to create a 10km of diameter asteroid, and the load time was less than 1 minute, plus the render distance of objects/asteroids has increased to more than 30km. This 2 planets are around 20km away from each other:

    [​IMG]

    We still have the issue of big download times, this map is around 12mb (could be easily avoidable if you only had to download the voxel data once, and once you are in the server update the changes, but w/e), right now both planets are made out of stone, if I try to add more materials to them, those 4-5mb asteroids, go up to almost 100mb (using the procedural generetion trick, of changing the name of one of the base asteroids, creating a new game... ) so any alternatives there would be really useful.
     
    Last edited by a moderator: Dec 15, 2014
  2. Don Jacobs Junior Engineer

    Messages:
    657
    And things like Merging roids, stations and roids, ships to each other (outside of creative mode), Throwing a ship at the player (put a suggestion in the admin script discussion that could be used to do this, as well as create custom world drifting ships/wrecks).

    The ease which we can add items to containers isn't in SE yet, cept to spawn it then put it in, which is too many steps for me :).

    SET does a lot of things that I doubt Keen will implement, or even partially implement, in the game itself. So a tool or Script would have to be used.

    I like SET. Plain and simple.

    As for roids in each sector, they could go with a naming convention similar to the sectors themselves, so there is no complications of 2 sectors using the same rock.
     
    Last edited by a moderator: Dec 15, 2014
  3. snapplepeach Trainee Engineer

    Messages:
    6
    Thank you for this remarkable tool!

    I was wondering if there currently exists an easy way for me to batch export the position coordinates for stations and asteroids within a world using SE Toolbox. I know that you can copy the full detail text for everything one at a time, but this is tedious and includes a lot of information that I don't need for what I intend to use the data for: a 3D map. It would be wonderful to be able to select a number of important stations and asteroids and obtain only the XYZ positions and names of each in a copy/paste-able list (or some exportable file).

    I'm a server admin and faction leader, and the main reason I'm looking for this is so that I can create a simple 3D map with points of interest and their relative locations. I think it would be a lot of fun to have a custom map for the world that can be rotated/zoomed and such. Most of the work in creating a bare-bones map will be done using MATLAB (for now) to generate the plot. If I had even just a text file with the xyz coordinates, I could make a general scatter plot that could be used to easily designate territories to my buddies. It would also be extremely helpful if the coordinate data had the ship/station name listed beside it so that all the important data clusters could be kept labelled on the map.

    If there already exists a way to do this (or a better way that you can suggest to easily make a 3D territory map for a SE world, please let me know).

    Thanks!
     
    Last edited by a moderator: Dec 16, 2014
  4. midspace Senior Engineer

    Messages:
    2,224
    I had started to add position information to the World Report.
    I'm not sure how far I got with it, as it was sidelined for a while.

    Voxel editing is definitely coming in game. The current public release has the MyObjectBuilder_VoxelHandDefinition object.
    In the MinerWars Mod codebase, this was used for direct editing of asteroids, moving asteroids, reseeding asteroids.
     
  5. midspace Senior Engineer

    Messages:
    2,224
    This is really where it should be (at least from the KeenSWH perspective).

    The new 3d converter, or the old one?

    Yeah, this is more of a result of changing the way they are storing the materials in the octree. I'm glad they took my suggestion, and stopped storing them as strings.

    Now this looks awesome!
    I played a little with the view distances of ships, and found they were more realistic.
    That sparkling speck in the distance? That's a ship!

    Steam still limits workshop items to 100Mb. So that will be a persistent issue for sharing stuff.
    The way Keen are headed at the moment, I'm hopeful we will be able to upload asteroids as workshop items.
    And this idea of fractal or some sort of algorithm for asteroids that you see but don't touch looks like it will become reality too.
     
  6. kittle Senior Engineer

    Messages:
    1,086
    One feature that would be very handy is a birds eye view, or 10,000ft view of where things are in the game in 3D space. So you can easily see where things are in relation to each other.

    You dont need to attempt to render the full picture of anything (even though people will ask for this) -- just a colored dot will be fine (mabye larger dots for asteroids). Then I can bring up a zoomable, panable window of colored dots to see where everything is in my world. Makes it easier to see who has lots of "stuff" and where gaps in your asteroid field are. Then add the ability to click on a dot and have it selected in the left-hand list.

    I tried something like this on my own (not in C#), but the zoom & pan part of my 3d library left much to be desired.
     
  7. MarlDaeSu Trainee Engineer

    Messages:
    58
    Just came here to say this: Midspace you are the unsung hero of every single dedicated server out there. Your hard work has made administrating a DS somewhat possible.

    You are a legend.

    That is all.
     
  8. midspace Senior Engineer

    Messages:
    2,224
    Development documentation for SEToolbox:
    https://setoolbox.codeplex.com/wikipage?title=Working%20with%20and%20developing%20SEToolbox
     
  9. DS_Marine Apprentice Engineer

    Messages:
    494
    [​IMG]
    I think there are some missing files in the latest released zip file.
    Oh and of course, thanks for your work. I tried to compile the code in VS2013 but had some trouble with that custom installer project, seems it's not so straightforward to use in VS2013.
    /Edit also I'm runing SE 01.060.023 and SEToolbox won't start because it detects version 1.59.11
     
    Last edited by a moderator: Dec 18, 2014
  10. midspace Senior Engineer

    Messages:
    2,224
    The .zip file is not released with the Space Engineers files.
    These are copied from your Space Engineers installation.
    If it cannot find the installation, you may have to start it up while holding the Shift key, to reset its paths.


    If it's finding 1.59, then you have an old copy of Space Engineers on your system somewhere.
    As above. Hold down the Shift key when you start up....
    or edit the registry directly.
    https://setoolbox.codeplex.com/wikipage?title=Frequently%20Asked%20Questions#other
     
  11. Dereknor Trainee Engineer

    Messages:
    81
    aah I'm sad to hear that you stop to develope the Toolbox :( I have no idea how to set up a server without it! It is the only tool to create a custom world.

    Yes, the game gives us some tools to put in ships and asteroids, but they are veery raw and you have to manually fly everywhere to place them. I like to play on 100km big maps with asteroids. How should I ever do that in the game! Or how should I delete unused ships or delete things that are just not used. SEToolbox has become an essential server tool!
     
  12. DS_Marine Apprentice Engineer

    Messages:
    494
    These vrage dlls I think were included in previous versions of SEToolbox. But since I always erase the content before unzipping the new version, maybe I'm the only one that has found this problem (or maybe I just forgot that I put these files in there to fix the problem by myself). The registry is ok, yet SET won't find the dll from the game's folder (dedicated 64bit server). Does SET read some registry also from steam? because that DS had steam and the full game installed, then I deleted the game and just unzip the dedicated server files that come in the tools folder in SE.

    I don't have that old version stored anywhere on my hd. Also happens on 2 diff computers (1 is dedicated server). According to processmonitor at the moment of the messagebox, the program has not accessed my game's executable.

    I'm trying to help you fix some bug, don't give me rtfm cr... :)

    /Edit oh and in one computer the version problem disappeared by a combination of running as admin and/or running the .bat file that uses the old dll, so I have only 1 computer with version error and I don't need to solve it so I can just forget about this bug.
     
    Last edited by a moderator: Dec 19, 2014
  13. midspace Senior Engineer

    Messages:
    2,224
    Not since June. That was the last time I included them in the release.

    The Toolbox does read the Steam registry, but only initially when it doesn't yet know where the game is located.

    Hm....

    Sorry. Some people seem to be totally ignorant of the obvious. It drives me :crazy:
    And, I do occasionally update the documents when replying to people, so you might find something new you didn't see before you posted. :D


    The batch file will invoke the old method of reading the dlls.
    What it does is copy the common and vrage files into the Toolbox directory. It will check to make sure it has the same version as in the game before copying them.

    Without the batch file, it will simply copy the common and vrage files to a temporary path, and use them from there.

    The difference is, copying them to the temporary directory doesn't require elevated UAC. But it's harder to program for and has the occasional issue on some computers.
    Copying to the files to the same directory as the Toolbox requires UAC, and may on occasion not copy the files correctly, thus presenting you with older versions.
     
    Last edited by a moderator: Dec 20, 2014
  14. dbudde Trainee Engineer

    Messages:
    33
    midspace, I too want you to know you are appreciated. DS would not be possible currently without you.
     
  15. jargen Trainee Engineer

    Messages:
    4
    I added around 30 asteroids but none of them are appearing when i load the game. I deleted all the old asteroids first and now i just have empty space.

    Edit: Just found the reason why. If i use the old asteroid format when SE converts it will it keep to the ore amounts i chose?
     
    Last edited by a moderator: Dec 21, 2014
  16. DS_Marine Apprentice Engineer

    Messages:
    494
    First of all, thanks for the detailed explanation. It helps me understand things a little more.

    Today I was playing with the computer that still had some problem (win2008server 64bit)
    I don't need help on this, I'm just reporting what happened in my case for what is worth.
    Startin SET normally (double click) would give this errro:
    Code:
    2014-12-21 12:55:00,511 [1] WARN  SEToolbox.Support.DiagnosticsLogging [(null)] - The Space Engineers content file 'C:\Program Files (x86)\Steam\steamapps\common\SpaceEngineers\Content\Data\CubeBlocks.sbc' could not be loaded, either because it is corrupt, or it is newer than expected.
    System.InvalidOperationException: There is an error in XML document (7280, 3). ---> System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.
       at Sandbox.Common.ObjectBuilders.Serializer.MyObjectBuilderSerializer.GetSerializer(String serializedName)
       at Sandbox.Common.MyAbstractXmlSerializer`1.ReadXml(XmlReader reader)
       at System.Xml.Serialization.XmlSerializationReader.ReadSerializable(IXmlSerializable serializable, Boolean wrappedAny)
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read256_MyObjectBuilder_Definitions(Boolean isNullable, Boolean checkType)
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read617_Definitions()
       at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
       --- End of inner exception stack trace ---
       at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
       at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle)
       at SEToolbox.Interop.SpaceEngineersApi.ReadSpaceEngineersFileRaw[T](Stream stream) in d:\Development\SpaceEngineers\Codeplex\Main\SEToolbox\SEToolbox\Interop\SpaceEngineersApi.cs:line 154
       at SEToolbox.Interop.SpaceEngineersApi.ReadSpaceEngineersFile[T](String filename, Boolean& isCompressed, Boolean snapshot) in d:\Development\SpaceEngineers\Codeplex\Main\SEToolbox\SEToolbox\Interop\SpaceEngineersApi.cs:line 105
       at SEToolbox.Interop.SpaceEngineersResources.LoadAllDefinitions(String stockContentPath, String modContentPath, Dictionary`2& contentData) in d:\Development\SpaceEngineers\Codeplex\Main\SEToolbox\SEToolbox\Interop\SpaceEngineersResources.cs:line 176
    
    Running as admin didn't help. Running the old dll bat as admin didn't work (The current directory gets moved to somewhere else and the exe just didn't start).
    Running the .bat normally and accepting all warnings/uac did make the program start without error, but will give permission denied on a vx2 file when I try to load a save.
    Running the exe normally aflter having running the bat, the exe starts ok so the previous run did some permanent modification that fixed the 1st error. vx2 permission error is still there.
    Right-clik and run as admin, the exe runs ok and I can load the save without problems.
    Code:
    2014-12-21 13:18:07,719 [1] FATAL SEToolbox.Support.DiagnosticsLogging [(null)] - Unhandled Exception
    Application: C:\Users\%USERNAME%\Desktop\SEToolbox\SEToolbox.exe
    CommandLine: setoolbox.exe  /OLDDLL
    CurrentDirectory: C:\Users\%USERNAME%\Desktop\SEToolbox
    SEBinPath: C:\Program Files (x86)\Steam\steamapps\common\SpaceEngineers\DedicatedServer64
    SEBinVersion: 1.61.14
    ProcessorCount: 16
    OSVersion: Microsoft Windows NT 6.1.7601 Service Pack 1
    Version: 4.0.30319.17929
    Is64BitOperatingSystem: True
    IntPtr.Size: 8
    IsAdmin: False
    CurrentUICulture: en-US
    CurrentCulture: es-AR
    
    Files:
    2014-12-18T23:00:55.6987715-03:00    498.176    2014.2.17.1    HelixToolkit.Wpf.dll
    2014-12-18T23:00:55.7057719-03:00    200.704    0.86.0.518    ICSharpCode.SharpZipLib.dll
    2014-12-18T23:00:55.7147724-03:00    301.056    1.2.13.0    log4net.dll
    2014-12-19T20:55:17.0250620-03:00    746.424    1.0.0.0    Sandbox.Common.dll
    2014-12-19T20:55:17.0440631-03:00    1.821.112    1.0.0.0    Sandbox.Common.XmlSerializers.dll
    2014-12-18T23:00:55.7177726-03:00    40        SEToolbox -OLDDLL.bat
    2014-12-18T23:00:55.7367737-03:00    1.131.008    01.060.013.1    SEToolbox.exe
    2014-12-18T23:00:55.7407739-03:00    747        SEToolbox.exe.config
    2014-12-18T23:00:55.7447741-03:00    27.648    1.0.0.0    SEToolbox.ImageLibrary.dll
    2014-12-18T23:00:55.7487743-03:00    7.680    1.0.0.0    SEToolbox.ImageShaders.dll
    2014-12-18T23:00:55.7767759-03:00    2.258.432        SEToolbox.pdb
    2014-12-18T23:00:55.7807762-03:00    48.640    1.0.0.0    SEToolboxUpdate.exe
    2014-12-18T23:00:55.6767702-03:00    39.936    2.0.20525.0    System.Windows.Interactivity.dll
    2014-12-19T20:55:17.4030836-03:00    422.328    1.0.0.0    VRage.Common.dll
    2014-12-19T20:55:17.4140842-03:00    486.328    1.0.0.0    VRage.Library.dll
    2014-12-19T20:55:17.4230848-03:00    410.040    1.0.0.0    VRage.Math.dll
    2014-12-18T23:00:55.6817705-03:00    64.512    2.2.0    WPFLocalizeExtension.dll
    2014-12-18T23:00:55.6867708-03:00    26.624    1.1.6.3    XAMLMarkupExtensions.dll
    
    BinCachePath: C:\Users\%USERNAME%\AppData\Local\MidSpace\SEToolbox\__bincache
    2014-12-19T20:55:17.0250620-03:00    746.424    1.0.0.0    Sandbox.Common.dll
    2014-12-19T20:55:17.0440631-03:00    1.821.112    1.0.0.0    Sandbox.Common.XmlSerializers.dll
    2014-12-19T20:55:17.4030836-03:00    422.328    1.0.0.0    VRage.Common.dll
    2014-12-19T20:55:17.4140842-03:00    486.328    1.0.0.0    VRage.Library.dll
    2014-12-19T20:55:17.4230848-03:00    410.040    1.0.0.0    VRage.Math.dll
    
    System.UnauthorizedAccessException: Access to the path 'C:\ProgramData\SpaceEngineersDedicated\EspacioArg\Saves\Espacio Argentino\Small_Pirate_Base_3_1.vx2' is denied.
       at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
       at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
       at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy)
       at System.IO.FileStream..ctor(String path, FileMode mode)
       at SEToolbox.Support.ZipTools.GZipUncompress(String sourceFilename, String destinationFilename) in d:\Development\SpaceEngineers\Codeplex\Main\SEToolbox\SEToolbox\Support\ZipTools.cs:line 275
       at SEToolbox.Interop.Asteroids.MyVoxelMap.Load(String filename, String defaultMaterial, Boolean loadMaterial) in d:\Development\SpaceEngineers\Codeplex\Main\SEToolbox\SEToolbox\Interop\Asteroids\MyVoxelMap.cs:line 230
       at SEToolbox.Interop.Asteroids.MyVoxelMap.GetPreview(String filename, Vector3I& size, BoundingBoxD& contentBounds, Int64& voxCells, Boolean& isValid) in d:\Development\SpaceEngineers\Codeplex\Main\SEToolbox\SEToolbox\Interop\Asteroids\MyVoxelMap.cs:line 139
       at SEToolbox.Models.StructureVoxelModel.ReadVoxelDetails(String filename) in d:\Development\SpaceEngineers\Codeplex\Main\SEToolbox\SEToolbox\Models\StructureVoxelModel.cs:line 342
       at SEToolbox.Models.StructureVoxelModel..ctor(MyObjectBuilder_EntityBase entityBase, String voxelPath) in d:\Development\SpaceEngineers\Codeplex\Main\SEToolbox\SEToolbox\Models\StructureVoxelModel.cs:line 70
       at SEToolbox.Models.StructureBaseModel.Create(MyObjectBuilder_EntityBase entityBase, String savefilePath) in d:\Development\SpaceEngineers\Codeplex\Main\SEToolbox\SEToolbox\Models\StructureBaseModel.cs:line 364
       at SEToolbox.Models.ExplorerModel.LoadSectorDetail() in d:\Development\SpaceEngineers\Codeplex\Main\SEToolbox\SEToolbox\Models\ExplorerModel.cs:line 436
       at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
       at MS.Internal.Threading.ExceptionFilterHelper.TryCatchWhen(Object source, Delegate method, Object args, Int32 numArgs, Delegate catchHandler)
     
    Last edited by a moderator: Dec 21, 2014
  17. Lewtz Trainee Engineer

    Messages:
    13
    I am having this same problem. Anyone know the fix? And yes I am making sure to save my SET before loading. The map file size changes each time, but none of the asteroids actually show up.
     
  18. midspace Senior Engineer

    Messages:
    2,224
    Please see the following thread if you have not already.
    http://setoolbox.codeplex.com/workitem/1576

    Using the old asteroids should allow you to specify the ores.
     
  19. voicesdark Senior Engineer

    Messages:
    2,208
    Still think it was incredibly short sighted and a poor management decision on the part of Keen to not fill you in on how the new asteroids system works so that you could modify SEtoolbox to work with it considering SEToolBox is more valuable an asset than anything Keen would develop.
     
  20. Don Jacobs Junior Engineer

    Messages:
    657
    Might not BE shortsighted. Remember, Midspace said the API, the one THEY wrote, can't handle the rocks yet, which is why they are only producing rocks with 1-2 ores in them. Once they fix that issue she MAY, not guaranteed, but MAY, be able to decode the format.
     
  21. Don Jacobs Junior Engineer

    Messages:
    657
    Midspace, SET still works (as you may have already known). I did interact with one of the Procedural rocks, and did a save, only it was saved, which means less file size for all those extras.

    Still, Keen ought to tell you how the new vx2 format works. Would only be polite.

    And your admin scripts are a necessary now, with the ranges involved. :)
     
  22. DuneD Junior Engineer

    Messages:
    948
    Well, the importer stopped working properly :(
    Now when I try to import a rock it imports it hollow, even if I set different ore types. I hope they eventually give us a tool to import our own asteroids, since right now the procedurally generated ones look like crap.
     
  23. Thedevistator Senior Engineer

    Messages:
    1,942
    Well to be fair to keen SE Toolbox is a 3rd party thing so it's not bad managing to not fill mispace in on the new vx2, but I will say they should inform everyone on this and how to work with it.
     
  24. Whiplash Trainee Engineer

    Messages:
    52
    I know this is currently 'unsupported' but if anyone has some working larger asteroids with single fills I would appreciate the sharing of said files. The common complaint with the procedural asteroids is that some ores are extremely difficult to locate, to that end this was going to be my temporary solution to the problem:

    I would like to have an asteroid for each rare material, preferably a large asteroid so I don't have to replace it daily, something fairly solid (IE not a bunch of large holes, tunnels etc) that is like 75%+ of a single material. I would need one for each material except Stone 1/2 and Iron.

    If anyone has some asteroids that still work properly in the current game I would greatly appreciate a link to a download please. As it appears I am unable to create any form of custom asteroid within SET or creative.

    Thanks.
     
  25. DuneD Junior Engineer

    Messages:
    948
    Btw, the importer still works, was a mistake on my end :p

    [SIZE= 13.3333330154419px]
    [/SIZE]You can always import one of the old asteroids stored in voxelmap folder, as long as you import it untouch, it should have procedural generation ore as always.
     
    Last edited by a moderator: Dec 29, 2014
  26. ScottMajor Apprentice Engineer

    Messages:
    236
    I hope this is the right place to post this...

    The Cyrrent SE Toolbox version 1.60.13.1, currently places asteroids in the savegame that appear incisible in-game. I've placed it using multiple ways (asteroid field, import, etc)... Can you help?
     
  27. midspace Senior Engineer

    Messages:
    2,224
    Using the old asteroids should allow you to specify the ores in SEToolbox.

    Please see the following thread if you have not already on how to do this, and to get copies of the old asteroid files.
    http://setoolbox.codeplex.com/workitem/1576
     
  28. ThatOneEngie Trainee Engineer

    Messages:
    3
    Hey, I can't tell if i'm just an idiot or not, but i cannot find a tutorial on how to copy an asteroid AND the base connected to it ((( Like the lone survivor platform )) into a new world and keep them connected.
     
  29. midspace Senior Engineer

    Messages:
    2,224
    http://setoolbox.codeplex.com/wikipage?title=Copying&referringTitle=Documentation
     
  30. ThatOneEngie Trainee Engineer

    Messages:
    3
    Oh. Thank you for the help!
     
Thread Status:
This last post in this thread was made more than 31 days old.