1. This forum is obsolete and read-only. Feel free to contact us at support.keenswh.com

New Camera Raycast and Sensor API (Update 01.162 DEV)

Discussion in 'Programming (In-game)' started by Drui, Nov 17, 2016.

Thread Status:
This last post in this thread was made more than 31 days old.
Page 1 of 5
  1. Drui

    Drui Keen Update Guy Staff

    Messages:
    1,415
    by Rexxar:
    --------------------------------------------------------------------

    This week brings several big changes to the programmable block API. The major features are:
    • IMyRemoteControl.GetFreeDestination has been removed
    • IMySensorBlock.LastDetectedEntity no longer returns an IMyEntity reference, instead returns MyDetectedEntityInfo
    • IMyCameraBlock has new raycast functions to replace GetFreeDestination
    • New MyDetectedEntityInfo struct to contain information about entities detected with sensors
    GetFreeDestination was never meant to be available, and causes many problems. It's been removed from the ingame API to make programmable blocks less dangerous to servers. If you really want to use it, it's still available in ModAPI, and it's been optimized slightly to make it a bit faster.

    LastDetectedEntity used to return a reference that you could hold onto forever. This opened the door for many exploits, and has been changed to instead return a struct with info about the detected entity. There is also a new DetectedEntities property that returns a list of entities that the sensor is currently detecting. The list that DetectedEntities returns is filtered by the sensor settings.

    Both the sensor and camera functions return a MyDetectedEntityInfo struct that looks like this:
    Code:
    public enum MyDetectedEntityType
{
  None = 0,
  Unknown,
  SmallGrid,
  LargeGrid,
  CharacterHuman,
  CharacterOther,
  FloatingObject,
  Asteroid,
  Planet,
  Meteor,
  Missile,
}

public struct MyDetectedEntityInfo
{
  /// <summary>
  /// The entity's EntityId
  /// </summary>
  public readonly long EntityId;

  /// <summary>
  /// The entity's display name if it is friendly, or a generic descriptor if it is not
  /// </summary>
  public readonly string Name;

  /// <summary>
  /// Enum describing the type of entity
  /// </summary>
  public readonly MyDetectedEntityType Type;

  /// <summary>
  /// Position where the raycast hit the entity. (can be null if the sensor didn't use a raycast)
  /// </summary>
  public readonly Vector3D? HitPosition;

  /// <summary>
  /// The entity's absolute orientation at the time it was detected
  /// </summary>
  public readonly MatrixD Orientation;

  /// <summary>
  /// The entity's absolute velocity at the time it was detected
  /// </summary>
  public readonly Vector3 Velocity;

  /// <summary>
  /// Relationship between the entity and the owner of the sensor
  /// </summary>
  public readonly MyRelationsBetweenPlayerAndBlock Relationship;

  /// <summary>
  /// The entity's world-aligned bounding box
  /// </summary>
  public readonly BoundingBoxD BoundingBox;

  /// <summary>
  /// Time when the entity was detected. This field counts milliseconds, compensated for simspeed
  /// </summary>
  public readonly long TimeStamp;

  /// <summary>
  /// The entity's position (center of the Bounding Box)
  /// </summary>
  public Vector3D Position
  {
  get { return BoundingBox.Center; }
  }

  /// <summary>
  /// Determines if this structure is empty; meaning it does not contain any meaningful data
  /// </summary>
  /// <returns></returns>
  public bool IsEmpty()
  {
  return TimeStamp == 0;
  }
}
    This gives you a ton of information about the detected entity without giving you a reference that you can exploit.

    On to the camera: The camera raycast is a very powerful tool. It doesn't exactly replace GetFreeDestination, but you can do similar sorts of things.

    Here's IMyCameraBlock:
    Code:
    public interface IMyCameraBlock:IMyFunctionalBlock
{
  /// <summary>
  /// Does a raycast in the direction the camera is facing. Pitch and Yaw are in degrees.
  /// Will return an empty struct if distance or angle are out of bounds.
  /// </summary>
  /// <param name="distance"></param>
  /// <param name="pitch"></param>
  /// <param name="yaw"></param>
  /// <returns></returns>
  MyDetectedEntityInfo Raycast(double distance, float pitch = 0, float yaw = 0);

  /// <summary>
  /// Does a raycast to the given point.
  /// Will return an empty struct if distance or angle are out of bounds.
  /// </summary>
  /// <param name="targetPos"></param>
  /// <returns></returns>
  MyDetectedEntityInfo Raycast(Vector3D targetPos);

  /// <summary>
  /// Does a raycast in the given direction.
  /// Will return an empty struct if distance or angle are out of bounds.
  /// </summary>
  /// <param name="distance"></param>
  /// <param name="targetDirection"></param>
  /// <returns></returns>
  MyDetectedEntityInfo Raycast(double distance, Vector3D targetDirection);

  /// <summary>
  /// The maximum distance that this camera can scan, based on the time since the last scan.
  /// </summary>
  double AvailableScanRange { get; }

  /// <summary>
  /// When this is true, the available raycast distance will count up, and power usage is increased.
  /// </summary>
  bool EnableRaycast { get; set; }

  /// <summary>
  /// Checks if the camera can scan the given distance.
  /// </summary>
  /// <param name="distance"></param>
  /// <returns></returns>
  bool CanScan(double distance);

  /// <summary>
  /// Returns the number of milliseconds until the camera can do a raycast of the given distance.
  /// </summary>
  /// <param name="distance"></param>
  /// <returns></returns>
  int TimeUntilScan(double distance);

  /// <summary>
  /// Returns the maximum positive angle you can apply for pitch and yaw.
  /// </summary>
  float RaycastConeLimit { get; }

  /// <summary>
  /// Returns the maximum distance you can request a raycast. -1 means infinite.
  /// </summary>
  double RaycastDistanceLimit { get; }
    The raycast function has several moddable values to allow server admins to balance it to fit their servers. Here's an excerpt from CubeBlocks.sbc that shows some values you can change:
    Code:
    <RequiredPowerInput>0.00003</RequiredPowerInput>
<RequiredChargingInput>0.001</RequiredChargingInput>
<RaycastConeLimit>45</RaycastConeLimit>
<RaycastDistanceLimit>-1</RaycastDistanceLimit>
<RaycastTimeMultiplier>2.0</RaycastTimeMultiplier>
    By default the camera raycast has no distance limit (denoted by -1). If you put 0 for the distance limit, it disables raycasting altogether. RaycastConeLimit lets you change the available scanning range, by default it's plus or minus 45 degrees in any direction relative to the direction the camera is pointing.

    The scanning range is a function of time. By default you gain two meters of range for every millisecond the camera is idle (and raycasting is enabled). Since each game tick takes roughly 16ms, you can do a 32m scan every tick with no penalty. Or you can do a 2km scan every second. This value may be tweaked in the future as we find a good balance of usability vs performance.

    Here is a sample script I used while developing the raycast functionality. It's set up to do a 100m raycast every tick and dump the results to an LCD:
    Code:
    double SCAN_DISTANCE = 100;
float PITCH = 0;
float YAW = 0;
private IMyCameraBlock camera;
private IMyTextPanel lcd;
private bool firstrun = true;
private MyDetectedEntityInfo info;
private StringBuilder sb = new StringBuilder();

public void Main(string argument)
{
  if (firstrun)
  {
  firstrun = false;
  List<IMyTerminalBlock> blocks = new List<IMyTerminalBlock>();
  GridTerminalSystem.GetBlocks(blocks);

  foreach (var block in blocks)
  {
  if (block is IMyCameraBlock)
  camera = (IMyCameraBlock)block;

  if (block is IMyTextPanel)
  lcd = (IMyTextPanel)block;
  }

  camera.EnableRaycast = true;
  }

  if (camera.CanScan(SCAN_DISTANCE))
  info = camera.Raycast(SCAN_DISTANCE,PITCH,YAW);

  sb.Clear();
  sb.Append("EntityID: " + info.EntityId);
  sb.AppendLine();
  sb.Append("Name: " + info.Name);
  sb.AppendLine();
  sb.Append("Type: " + info.Type);
  sb.AppendLine();
  sb.Append("Velocity: " + info.Velocity.ToString("0.000"));
  sb.AppendLine();
  sb.Append("Relationship: " + info.Relationship);
  sb.AppendLine();
  sb.Append("Size: " + info.BoundingBox.Size.ToString("0.000"));
  sb.AppendLine();
  sb.Append("Position: " + info.Position.ToString("0.000"));

  if(info.HitPosition.HasValue)
  {
  sb.AppendLine();
  sb.Append("Hit: " + info.HitPosition.Value.ToString("0.000"));
  sb.AppendLine();
  sb.Append("Distance: " + Vector3D.Distance(camera.GetPosition(), info.HitPosition.Value).ToString("0.00"));
  }

  sb.AppendLine();
  sb.Append("Range: " + camera.AvailableScanRange.ToString());
  lcd.WritePublicText(sb.ToString());
  lcd.ShowPrivateTextOnScreen();
  lcd.ShowPublicTextOnScreen();
}
    Additionally, if you turn on debug draw in the F11 menu, the game will draw the raycast and valid scanning area.
     
    Last edited by a moderator: Nov 17, 2016
    Drui, Nov 17, 2016
    #1
    • Like Like x 7
  2. Roxette

    Roxette Senior Engineer

    Messages:
    1,547
    Do we have to wait another three weeks for this to hit the 'stable' branch, or will that also be updated today ?
     
    Roxette, Nov 17, 2016
    #2
  3. Malware

    Malware Master Engineer

    Messages:
    9,867
    Of course you have to wait.
     
    Malware, Nov 17, 2016
    #3
    • Agree Agree x 1
    • Informative Informative x 1
  4. Georgik

    Georgik Apprentice Engineer

    Messages:
    215
    Will be there some backward compatibility with Stable branch API? Or scripts containing LastDetectedEntity will crash during compilation in DEV now?
     
    Georgik, Nov 17, 2016
    #4
  5. Cryo2005

    Cryo2005 Trainee Engineer

    Messages:
    6
    Double Rainbow!

    GetFreeDestination() was a pain. I can't wait to rebuild a F/A-18 radar display :) Awesome stuff devs :)
     
    Cryo2005, Nov 17, 2016
    #5
  6. jonnytaco

    jonnytaco Apprentice Engineer

    Messages:
    228
    alright, time to get to work on small grid camera array layouts :D
     
    jonnytaco, Nov 17, 2016
    #6
  7. Malware

    Malware Master Engineer

    Messages:
    9,867
    Use conditionals. Scripts using the old API will not work, this is a breaking change and a necessary one. Sometimes that's the only solution.
    https://forums.keenswh.com/threads/u...d-improvements.7386481page-$1.html#post-1286997636
     
    Malware, Nov 17, 2016
    #7
    • Like Like x 1
    • Agree Agree x 1
  8. Georgik

    Georgik Apprentice Engineer

    Messages:
    215
    Yeah, I agree, just asked if there is some possibility. And there it is, thanks for showing me compilation conditionals, I absolutely missed this feature (maybe some official announcement about this ? :) )
     
    Georgik, Nov 17, 2016
    #8
  9. CheeseJedi

    CheeseJedi Apprentice Engineer

    Messages:
    382
    These changes sound great! We've needed an officially supported way of detecting objects at range. I expect many existing scripts will need to be modified, but such is the price of progress.

    Presumably the Pirate 'AI' PB scripts have been updated to reflect these changes? I'd hate to think they were forgotten about and the pirate faction ending up being 'broken' again. :eek:ops:
     
    CheeseJedi, Nov 17, 2016
    #9
    • Agree Agree x 1
  10. Malware

    Malware Master Engineer

    Messages:
    9,867
    As far as I know they don't use any of these features.
     
    Malware, Nov 17, 2016
    #10
    • Agree Agree x 1
  11. CheeseJedi

    CheeseJedi Apprentice Engineer

    Messages:
    382
    Good catch. It's been a long time since I last looked at that code so my memory must have faded. Having just looked they use GetNearestPlayer instead - so hopefully they won't be affected by this.

    Nothing worse than dead pirates that you didn't make dead. :p

    EDIT:
    --- Automerge ---
    Apologies for the double post - can't edit my previous one... Thanks, Automerge!

    I Just checked the cubeblocks.sbc from the develop branch and the new moddable settings are missing from the small camera (but are there for the large block camera):

    Code:
          <RaycastConeLimit>45</RaycastConeLimit>
      <RaycastDistanceLimit>-1</RaycastDistanceLimit>
      <RaycastTimeMultiplier>2.0</RaycastTimeMultiplier>
    Also missing is

    Code:
          <RequiredChargingInput>0.001</RequiredChargingInput>
    Which I'm guessing is to do with the 'cooldown' time of the raycast?

    Any thoughts on whether this is likely to be an XML omission, or are small grid cameras excluded from this new raycast behaviour?
     
    Last edited: Nov 17, 2016
    CheeseJedi, Nov 17, 2016
    #11
    • Funny Funny x 1
  12. halipatsui

    halipatsui Senior Engineer

    Messages:
    1,253
    How long vanilla maximum raycast range will be?
     
    halipatsui, Nov 17, 2016
    #12
  13. MisterSwift

    MisterSwift Apprentice Engineer

    Messages:
    367
    I've been waiting for this for months! Looks like it's time to get back into SE :D
     
    MisterSwift, Nov 17, 2016
    #13
  14. Inflex

    Inflex Developer Staff

    Messages:
    397
     
    Inflex, Nov 17, 2016
    #14
  15. Sanfard

    Sanfard Apprentice Engineer

    Messages:
    109
    Nice. I'll be looking forward to some fancy camera scripts in the near future. My fighter's GFD aimbot script is now obsolete, however. :p
     
    Sanfard, Nov 17, 2016
    #15
    • Funny Funny x 1
  16. Acolyte

    Acolyte Apprentice Engineer

    Messages:
    109
    I like the look of this new api function, but I am surprised it was put onto a camera instead of a new radar dish model - Keen seem to be in a modeling mood at the moment, what with all those (redundant imho) teasers they put into their update videos at the moment.

    Still, we can code now (well not exactly now for me since I'm stable :p) and worry about models later. I never was very good at being artistic with this game anyway....
     
    Acolyte, Nov 17, 2016
    #16
  17. rexxar

    rexxar Senior Engineer

    Messages:
    1,532
    I just missed it when I was changing the sbc. However, the ObjectBuilder serializer is set up in such a way that if the values don't exist in the sbc, it will automatically use the default values, which are the same as used in the large camera block. (this means all modded cameras will work with default raycast limits!)

    The charging input is the amount of power the camera uses when the raycast is turned on an charging.
     
    rexxar, Nov 17, 2016
    #17
  18. CheeseJedi

    CheeseJedi Apprentice Engineer

    Messages:
    382
    Thanks for the clarification @rexxar - good to know all cameras are included. Fantastic work on this by the way! :tu:
     
    CheeseJedi, Nov 17, 2016
    #18
  19. Wicorel

    Wicorel Senior Engineer

    Messages:
    1,263
    So I made some quick tests and ended up with this modified version of Rexxar's example code.

    It scans a box of settable width and height with 'focus' on the center area. (the central area gets more tests than the outter areas).

    It starts scanning out from the center and gets a wider scan by quadrants until it reaches the limit.

    If nothing was found, it will increase the current scan distance.

    If something is found, it sets the scan distance to the center of the found object.

     
    Wicorel, Nov 18, 2016
    #19
  20. Loues S. Cat

    Loues S. Cat Apprentice Engineer

    Messages:
    143
    Ummmm...
    "public readonly Vector3D? HitPosition;"

    Is Vector3D? a typo?
    Because I already know it is implemented into the game... so uhh... is "Vector3D?" a special case or was it meant to be "Vector3D"?
    right now it is screwing up some otherwise lovely calculations
     
    Loues S. Cat, Nov 18, 2016
    #20
  21. rexxar

    rexxar Senior Engineer

    Messages:
    1,532
    Vector3D? is short for Nullable<Vector3D>. Vector3D is a struct and normally cannot be null, so we have to wrap it in Nullable if we want to assign it a null value. Read the example script, it shows you how to deal with Vector3D?.
     
    rexxar, Nov 18, 2016
    #21
    • Agree Agree x 1
  22. Wellstat

    Wellstat Apprentice Engineer

    Messages:
    212
    From your struct, it looks like we can get target center position, target orientation, and ray hit position. You can lock on to specific block position with that.
     
    Wellstat, Nov 18, 2016
    #22
  23. Malware

    Malware Master Engineer

    Messages:
    9,867
    Because then they had to add a new model, adding to the workload... besides, this isn't a radar, it's a raycaster. It's not far-fetched to imagine the cameras have a double function imo. I like imagining it has a simple rangefinder laser, and once it finds "something" it invokes technology similar to the sensor at that location to retrieve details.

    The replacement models happening right now were always gonna happen. Planned from day-1. And I gotta say, for me the teasers are in no way redundant. I love getting a sneak peek.
     
    Malware, Nov 18, 2016
    #23
    • Agree Agree x 1
  24. Wellstat

    Wellstat Apprentice Engineer

    Messages:
    212
    Having been taken a long holiday from scripting, now comes the crazy experiments back in SE.

    Since we can identify friend or foe with the new beam, upcoming scripts will be able to coordinate all the cameras and sensors on dronw swarms, creating sensor area networks to power 3D proximity radar displays and coordinate attacks.
     
    Wellstat, Nov 18, 2016
    #24
    • Like Like x 1
  25. sepen_

    sepen_ Trainee Engineer

    Messages:
    52
    I'm a bit worried, but not positive, that this allows tracking of a grid beyond a doubt? As in, I can always tell 100% the entity I detected is the same, or different, than the last one, even when I'm not friendly?

    That seems overpowered given that holding on to a reference was axed. I couldn't even deploy decoy grids to confuse a camera guided missile, it being bad enough the decoy has to be the same Type to begin with.

    Edit: That said, awesome addition I'm already playing around with! Good job. :)
     
    sepen_, Nov 18, 2016
    #25
  26. Cryo2005

    Cryo2005 Trainee Engineer

    Messages:
    6
    Is there a way to get the actual Entity ref info with the ID? Anyhow, if it's an enemy entity you won't even get a name, only a generic destriptor. The ID is great to log all the found entries and make sure you don't log duplicates, And since you don't get an actual IMyEntity reference, there's not much harm you could do. Long distance scanning with this method might be challanging, due to the cooldown mechanic...which is awesome, because it will force us to implement different types of ladar systems.
     
    Cryo2005, Nov 18, 2016
    #26
  27. sepen_

    sepen_ Trainee Engineer

    Messages:
    52
    So, here's the question: Will you repeatedly receive the same EntityId for the same detected entity?
     
    sepen_, Nov 18, 2016
    #27
  28. Malware

    Malware Master Engineer

    Messages:
    9,867
    How would they be an ID if not? :p
    --- Automerge ---
    I don't understand what you're saying here. In order to track you with this new system, you must either be within range of sensors, or in visual range of a camera. If something gets between you and the tracking camera, all the tracker has is the last position you were detected.

    So you can make a decoy to block an incoming missile, for instance. Effectively blind it.

    Can it identify you? Yes, sure. But only if it can see you. And the greater the range, the lesser the resolution a possible scan will have.
     
    Malware, Nov 18, 2016
    #28
    • Agree Agree x 1
    • Disagree Disagree x 1
    • Funny Funny x 1
  29. sepen_

    sepen_ Trainee Engineer

    Messages:
    52
    They would be an ID within a result set of one scan event. (I know they aren't here, but it's not as outlandish as you made it out to be.) :p right back.

    I'm saying, I could dump a truck load of ore into a missile, and it would not have to go to any length at all to discern those things aren't the ship. No motion analysis, no size, no prediction, no nothing. They are of a different type. No explosions of missiles into countermeasures. It can just hang around, and at any later point determine with 100% accuracy a new 'signal' is the exact same thing it saw 45min ago, no doubts. Edit: It gets even worse with networks of detection systems that all share one ID and wouldn't need effort to coordinate at all.

    In my opinion, an exact ID is too much. Even two identical ships could be accurately told apart. Every. Single. Time. That's fair, if you have a friend and do some air traffic control with transponders, but not if you are foes and aim with missiles e.g.
     
    Last edited: Nov 18, 2016
    sepen_, Nov 18, 2016
    #29
  30. Malware

    Malware Master Engineer

    Messages:
    9,867
    Then they wouldn't be called entityId... You're not getting entities, you're getting info about entities, which the name clearly states... it's quite obvious to me, but I've been working with structures and APIs for so long that maybe it just seems that obvious to me. I didn't intend any offense.

    Personally I'd never call anything ID if it wasn't persistable. IDs within one result set would simply be their index... this is a read-only call, after all.

    Ah, I see your problem.

    But... um... There is nothing wrong with my statement... There's nothing wrong with yours either, but mine does not conflict with yours... so it's like you saying "water is wet" and me disagreeing with that...
     
    Malware, Nov 18, 2016
    #30
Page 1 of 5
Thread Status:
This last post in this thread was made more than 31 days old.