New! ModAPI and PB API changes Nov 2017

Hi everyone! I've burned through two keyboards in the last month working on the ModAPI, Programmable Block, and the PB API. These changes are almost ready for release, and you should see them in the next major update! This is all live now!

There are some big changes in here, and we're posting this information before release to give scripters and modders some time to update their work before release. There's a lot here, so let's dive right in!


Guides here!
https://forum.keenswh.com/threads/modding-guide-new-features-nov-2017.7398266/
https://forum.keenswh.com/threads/pb-scripting-guide-how-to-use-self-updating.7398267/

BREAKING CHANGES!
Nobody likes it, but sometimes we have to break things. This time it's mostly scripts which are broken.

The rotor API has been overhauled. The old properties for setting rotation limits was completely broken, so we replaced them with new properties. The break here was intentional, so that your script doesn't accidentally nuke your ship by setting rotation limits to insane values.

IMyGyro.Pitch was inverted previously, we un-inverted it. You'll need to update your scripts if you use this property.

IMyShipController.GetMass was inaccurate, and returned integer values. I fixed the logic and changed it to floats. You're welcome.

The ModAPI IMyConfig was very, very, very outdated. I scrapped the entire file and generated a new one. Some of the types are slightly different, and may cause breakages in some cases.

There was a complex issue where mods wouldn't compile from the workshop if you had cs files in subfolders. I fixed this issue, but it might cause your mod to lose any files that were in global or local storage.

We've stopped the "Midas exploit". PB used to have direct access to MyObjectBuilder_Base, which allowed for some serious security issues. We've solved the problem by partially whitelisting the type, which is where scripts will start breaking.
Currently, many inventory scripts will get the name of an inventory item by using the SubTypeIdName property. This property has been removed, and you must use SubTypeId instead.

PHP:

string itemType = $"{item.TypeId}/{item.SubTypeId}"; //New way
string itemType = $"{item.TypeId}/{item.SubTypeIdName}"; //Old way


The projector API had LoadBlueprint and LoadRandomBlueprint methods. These directly accessed disk on the server, and have been removed. These methods shouldn't have been available in the first place, and didn't really work properly to begin with. They still exist in ModAPI, however.

IMyGridProgramRuntimeInfo.MaxMethodCallCount and CurrentMethodCallCount are obsolete and have been removed. They haven't worked in quite some time as we haven't been using that kind of performance counter for about a year. To replace these methods, we've added MaxCallChainDepth and CurrentCallChainDepth, which are actually used by the protection system now.

NEW FEATURES!!!
Lots of fun stuff in here!
New SBC additions from the lovely @Phoenix84

• ensure the Public flag is checked everywhere, to make it useful for hiding things like fonts, blueprints, etc

PB additions from the enigmatic @Digi

• add BuiltBy info to slimblocks
• retreive pilot name from ship controller

Digi also added the ability for mods to retrieve server simspeed through IMyPhysics

Few things from the man, the myth, the legend, the @Malware

• fix air vent returning oxygen level when vent is disabled
• fix IMyAirVent.GetOxygenLevel doesn't return planetary oxygen

Additions from your friendly neighborhood modding overlord (me)

• allow access to the oxygen provider system (what planets use to create oxygen atmospheres)
• Ingame.IMyLargeTurretBase.GetTargetedEntity ( ͡° ͜ʖ ͡°)
• allow access to current game version
• add compilation symbols for game version, one for major, one for minor
• allow planet mods to use custom cloud layers
• add access to GridGroups
• IMyMultiplayer.SendMessage* now has no data length limit! (terms and conditions apply)
• fix MyAPIGateway not being initialized soon enough
• all terminal controls now have properties in the Ingame interfaces. No more GetValue calls!
• fix Ingame GridTerminalSystem changing instance during runtime

There's more than this, but these are the highlights. Full details are at the bottom of the post!

The most exciting new feature is that Programmable blocks no longer require timers! This feature will get its own guide, but basically you can configure the PB so that it runs its script every 1, 10, or 100 ticks. This allows for some very flexible scripts, and actually improves performance! To be completely honest, I was so excited to implement this, I got the ticket on a Friday morning and worked through the weekend to turn it in that Sunday evening. This is something a lot of people have wanted for a long time and it's finally here!


Bugfixes and Improvements
Thanks to everyone who has reported issues in RexBot, on the forum, and the three or four people who threw bricks through my window with bugreports tied to them. These player reports represent the bulk of this update.

I fixed an extremely annoying issue with the programmable block where the Program constructor was run before the grid group was fully spawned, or when the PB was damaged, or unpowered. This shouldn't cause any problems, but if you notice any strange behavior, please contact me and I'll fixit ASAP

For a long time, modders have been complaining about the 4kB limit on network messages. To the point that a lot of people designed message segmenting systems to split huge messages into 4kB chunks that get reassembled on the other side. As it turns out, sending 100 4kB messages in one tick is much worse than sending a single 400kB message. Our multiplayer programmer had done some backend work a few months ago to our netlayer to remove any length limit on reliable messages, so we decided to just completely remove the 4kB limit for mods. However, unreliable messages are limited by Steam to 1200B, so we decided to lower that limit for mods to 1024B, to account for overhead from our system. This limit has always been there, but it just failed silently before. This change allows mods to have more flexible networking while causing less strain on the network.

Another long-time annoyance is that MyAPIGateway was not initialized when mod code actually starts running. After a lot of investigation and caffeine, I found a more optimal place to initialize MyAPIGateway and moved it there. You should now be able to rely on MyAPIGateway being not-null when you need it.

Grid groups are an awesome feature of our engine that mods now have access to. Basically a grid group is a collection of grids that are connected in some way. There's Logical, Mechanical, Physical, and NoContactDamage groups, each represent a different kind of connection and have different uses. Logical connections are grids which have connected terminal systems. Mechanical are grids connected by rotor/piston/suspension, including Logical connections. Physical connections include Mechanical, as well as grids connected by landing gear. NoContactDamage groups are only grids connected by landing gear. You can find this system at MyAPIGateway.GridGroups.

The old terminal property system for PB is now unnecessary! I went through every block, and ensured that all terminal controls are now represented inside the block's Ingame interface. Everything now has a pretty get/set method, or a function call for any actions. Changing over to these properties will make your PB script lighter, better, faster, stronger.

Compilation symbols allow your code to compile differently based on the version of the game. There's two symbols defined, VERSION and BUILD. For example, version 1.184.700 will define VERSION_184 and BUILD_700. You can also access the current game version at MyAPIGateway.Session.Version.


I will post more detailed guides for these new features in the future. Just consider this a very comprehensive changelog


Full Changelog
PB API:

• PB can make itself run every tick, 10 ticks or 100 ticks, no longer requiring timer blocks (RuntimeInfo.UpdateFrequency).
• Added Main(string, UpdateType) to determine caller type (self-update, terminal, timer, antenna, mod, etc).
• All block interfaces now have properties, making Set/GetValue() unnecessary for vanilla blocks.
• Refactor Ingame rotor API. Member names are changed on purpose to break scripts.
• Add Ingame.IMyLargeTurretBase.GetTargetedEntity
• Fix PB constructor executing too early.
• Fix IMyAirVent.Status (also affects modAPI)
• Fix index out of range exception in programmable block Echo
• Fix GridTerminalSystem changing for PB scripts
• Fix MyCubeGrid.GetCurrentMass is inaccurate.
• Change MyGridMass struct to use float instead of int
• Change GetCurrentMass to use float
• Add list of usable sounds for Ingame SoundBlock interface
• Add System.Buffer to script whitelist
• Fix wrong getter in IMyGyro.Pitch (also affects modAPI)
• Allow MyCubeSize for programmable block
• Fix IMyMergeBlock.IsConnected not working correctly. Fix IMyMergeBlock.Other not working correctly. (also affects modAPI)
• Fix spelling error BLinkLength, in a backwards compatible way.
• Added HasPilot, PilotName and GetUserRelationToPilot() for Ingame.IMyShipController.
• Added OwnerName, BuiltById and BuiltByName to IMySlimBlock and IMyCubeBlock for PB.
• Move IMyLargeTurretBase.SetTarget(IMyEntity) from Ingame interface to ModAPI

SBC Modding:

• Add checks for MyDefinitionBase.Public in various areas. So mods can have hidden definitions, and dynamically show or hide them in scripts (maybe also VS?).
• Fixed modded gas types (removed static ctor from MyEntityThrustComponent which initialized the ResourceDistributor too early).
• Fix modded ammo can go backwards (SpeedVariance tag).
• Fix modded blocks can't load textures if 'models' appears in the texture path

ModAPI:

• Fix MyAPIGateway members being null during startup.
• Fix pathing in script compiler
• Change limits of mod multiplayer messages. Remove limit for reliable messages, lower limit to 1024B for unreliable.
• Fix IMyMultiplayer.SendMessageToOthers
• Fix MyFactionCollection.AddPlayerToFaction
• Whitelist members in SessionComponents
• Whitelist MyBillboard.BlendTypeEnum
• Rewrote and updated IMyConfig.
• Fix GPS markers not being removed after their DiscardTime
• Fix mods unable to add custom controls to basic doors.
• Corrected checker for finalizers in MyScriptCompiler
• Fixed MyCubeBuilder.GetBuildBoundingBox() returning inconsistent BBox center when aimed at a grid
• Add option to specify a PreviewModel for the cubebuilder.
• Fix issue where modded planets can't use custom cloud layers
• Implement IMyGridGroups, to allow mods limited access to the grid group system (MyAPIGateway.GridGroups).
• Added to IHitInfo the hit normal vector and fraction (useful for computing distance between start and hit position faster).
• Added IMyPhysics.ServerSimulationRatio which returns the server's sim speed even on clients.
• Remove IMyGravityGeneratorBase.IsPositionInRange, as it conflicts with IMyGravityProvider, and both have the same implementation
• Wrap MyOxygenProviderSystem for ModAPI
• Allow System.ObsoleteAttribute for mods.
• Add IMyTerminalBlock.CustomDataChanged event
• Add colorable triangle billboard overload
• Add mod compilation symbols for game version
• Add Version property to IMySession
• Add implicit conversion for MyVersion to System.Version for the IMySession property
• Change some definition classes to be public
• Add ColorMask property to IMySlimBlock
• Add IMyFactionCollection.FactionStateChanged
• Add IMyFactionCollection.CreateNPCFaction
• Add more ProtoBuf items to whitelist

 

Oh sweet mercy!

Love u rexxar <3

I think I need new pants...

I think I should contact my doctor... this isn't going away within 4 hours...

 

Hop on board the change train!

 

Incoherent burbling noises

Rexxar you are a god amongst mortals.
Thank you for your service and we salute your efforts!

 

No more timers!

 

This is just amazing

 

yes, yes, very good API works guys!!

 

The most exciting new feature is that Programmable blocks no longer require timers! This feature will get its own guide, but basically you can configure the PB so that it runs its script every 1, 10, or 100 ticks. This allows for some very flexible scripts, and actually improves performance! To be completely honest, I was so excited to implement this, I got the ticket on a Friday morning and worked through the weekend to turn it in that Sunday evening. This is something a lot of people have wanted for a long time and it's finally here!


ohgodyes

 

Poor Timers.

 

Bless you, sir.

 

Very happy for these changes, thank you a great deal for the time you've put into it!
I know us programmers are a small group but with these changes we can make a lot of fun things for a lot of the community to use.

 

Running a script once per tick and a performance at the same time? What unholy arcane rituals did you perform?
You guys are doing a great job and service to the coding crowd! This is awesome!

 

Forgive my ignorance, I'm not a programmer, but I like to keep informed on changes. I use the looping computers mod, because I hate having to have a timer for every programmable block I use, so the change to allow PBs to loop themselves is wonderful news. However, I find most of the scripts I run work best with a timer of either 1, 3, 10, or 30 seconds, depending on what they do. Of course that depends on sim speed. So with an ideal sim speed of 1.0, what does 1, 10, and 100 ticks translate to?

Also, will this be a setting in the block itself (similar to how the 'Looping Computers' mod does it. Or will the loop timer be set as part of the script? I'd think that, while having it as part of the script may be handy for some, having it as a block setting would be more user-friendly.

 

Before seeing this thread I had absolutely zero hype for the next update.

Now I'm the most hyped I've been since planets.

*shovels hype into the hyp train's engine*

Let's get this hype train rolling!!

 

At sim speed 1 there are 60 ticks in a second.

 

There's always 60 ticks in a game second. That's why there's such a thing as simspeed. The number of ticks doesn't change, the second does...
--- Automerge ---

It's part of the script because it's intrinsicly part of the script's logic. For many scripts, having the end user change the interval would actually break the logic of the script. Not to mention that the repeat interval can and will change during runtime. It can't really be compared to the timer block at all, because that wasn't designed to run scripts: this is.

Also; for those scripts where such customization is a thing - well, there's nothing stopping them from still using timers. It all depends on the need

 

In that case, I may still be using the Looping Computers mod then. I know there's some scripts I use that eat up a lot of system resources when they run, and if they're running every second and a half, that'll make those scripts unusable for me.

 

It's part of the script because it's intrinsicly part of the script's logic. For many scripts, having the end user change the interval would actually break the logic of the script. Not to mention that the repeat inb

Well I'm afraid it's not really up to you as the user of a script, you can't just choose to use a mod if the script is written for this. They would conflict - or worse. If a script is that slow you should complain to the author, because then it's poorly written. Harsh but true.

 

So if there is a dropdown or similar that allows choosing between None (run only on Run), 1, 10, or 100 - it just runs after that time immediately without any kind of start button? The code would start executing before the player has a chance to exit the console for the PB, and would similarily just start running on game load? I assume there is permanent code in the block before or after update for 1, 10, or 100 ticks. If the script in the block is buggy or not compiled, what happens?

About 0.0167 seconds, 0.1667 seconds, and 1.6667 seconds respectively.


I remember in SE multiplayer years ago that players with higher sim speeds would have ships that travelled faster, despite all ships having the same speed shown on the HUD. It works differently now, but at the time velocity was effectively metres/tick rather than metres/second.

 

One tick is always 1/60th of a second. Additionally, you can simply keep a running sum of your ticks since last run and only run the code logic once a threshold value has been reached.

 

It's controlled by the scripter, not the end user. The idea is to lessen the setup need for the end user, but also to allow the scripter to fully control how the script runs, even to the level of changing frequency during runtime to adjust for need - thus allowing for a lesser performance impact on when that power is not required. Imagine a script running every 100 ticks normally, until a condition occurs that means it needs a higher sampling rate, then it ups its frequency itself.

 

This is getting better and better. I can already see certain states in my stuff that might need faster control vs certain states that could also do with multiple seconds in between each run.

 

60 ticks = 1 second

 

It could be so cool if runtime exceptions in Ingame scripts included info about location where that exception was thrown. In large scripts it may become total pain trying to find where that null is popping in instead of instance..

 

Oh right, I was thinking 60 ticks = 60 updates / second rather than 1 update / second.

Can't you just wrap the main code in a try block, catch the exception and print the stack trace?
Or is there something preventing that?
(It's been a while.)

 

Performance. You do not want your script to be running in debug mode, trust me

This helps a little though:
https://github.com/malware-dev/MDK-SE/wiki/Debugging-Your-Scripts

 

@rexxar Though properties will be available for the PB, the default, minimum (mininum ) and maximum values we'll still have to obtain via the TerminalPropertyExtensions (GetMinimum, GetMaximum, GetDefaultValue), right ?

 

Thanks for tip. I should be stupid that didn't manage to come up with this myself. Maybe getting too old. Had solved my problem within 5 minutes and couldn't find solution for days..
--- Automerge ---

Yep, Malware, I had made exactly same thing on my side and that was enough. I think Keen could include stack trace not just error message. That does not require debug mode, but helps much more than generic error message.

 

Not entirely, for simple push button actions they are useful (i.e. hangar doors, pressurization etc.) I use them all the time for setting dog conditions on bulkhead doors for airtightness compartmentalization.

 

Thanks Rexxar! Which game version and build will contain these changes, and when should we expect to see it available for testing on steam?