How to create custom marker shape and control their orientation on charts?

[PBR]

I have a data table with Time, Long, Lat, and orientation columns which represents a vessel position and its direction at different times. I am looking to create a scatter plot with a simple custom shape for my scatters. Also, how to show the direction of different scatter points based on the corresponding data read from orientation column.

Can someone please help me on this? I found the below link on Mapchart but no instruction or code is available there as a starting point.

https://www.youtube.com/watch?v=4ognnrmMCP8&t=470s

Thank you

[David Boot-Olazabal]

Hi PBR,

I have checked the video and it seems like this example is using map charts with different layers to portray the location of the vessel.
I will ask my colleague, who presented this demo, to have a look at your request. Due to the holiday season, you may expect a delay in response though.

Kind regards,

David

[PBR]

Hi David,
I really appreciate that. Yes it is used Mapchart in the video and it is probably going to work for me as well. It was mentioned in the video it is used ironpython to form the custom shape and its direction over time.
Thank you

[Kirsten Smith (she/her)]

You can vote for this suggestion here:

https://ideas.spotfire.com/ideas/SPF-I-5123

 

There is also a workaround in the conversation that you might find helpful.

[Mathew Lee]

Hi PBR

I am preparing a more detailed response so please bear with me.

However, let me give you some high level ideas in the meantime.

• The MapChart is required because of its ability to render polygons as geometric shapes on a feature layer
• In this case, the geometric shapes are not regions of land but the shape of vessels at a specific orientation
• These geometic shapes need to be stored in a geometry column in a format called WKB (Well-Known Binary).
• The geometry column can be in the same table as the rest of the vessel data.  This can be pre-generated at the data source if WKB is supported e.g. Oracle Spatial Database.
• The workaround is to have Spotfire generate the WKB data as a separate GEO datatable and use the Map Chart's feature layer to assoicate this GEO datatable as the Geocoding Hierarchy (Map Chart Documentation).  
• In order for the Geocoding Hierarchy association to work, there needs to be an ID column on both the Vessel datatable and the GEO datatable
• To generate the GEO datatable, a program needs to read the ID, Latitude, Longitude and Orientation from each row of the Vessel datatable, and then create a WKB data with some trigonometry.  This can be done as a IronPython script or as a Data Function.  I will show you how to do this in IronPython script in my later post.
• You may also want to use the size of the vessel (if it is available) to render the shape based on size with a bit more trigonometry.
• If the analysis you aim to do requires zooming in from a higher zoom level then the vessel shapes would be too small to be shown at a continental zoom for example.  In that case, the generation script needs to take the zoom level into account and generate shapes that are scale by zoom level.

This is response for now.  As I said, please bear with me...

Mathew

[Mathew Lee]

Here is the details of the vessel shape generation script.

First of all, the imports

# Imports
import clr
clr.AddReference('System')
import System, math
from System import Array, Byte, Convert, BitConverter
from System.IO import StreamWriter, MemoryStream, SeekOrigin,BinaryWriter
from Spotfire.Dxp.Data.Import import StdfFileDataSource
from Spotfire.Dxp.Data import DataTableSaveSettings, DataValueCursor

Then define the parameters and data value cursors in preparation for reading.  These can be hardcoded in the script or pass in from the script parameters.  The Front, Back, Left and Right columns are the distance in meters Longitude and Latitude location to the edges of vessel.  They specifie the size of the vessel and are optional.  You may use Length and Width instead depending on the your data.  You may also use constant length and width values across all vessels in your dataset.

# Define table name and column name parameters
VesselDataTableName = "VESSELS"
IdColName = "VESSEL_ID"
LatColName = "LAT"
LonColName = "LON"
HeadingColName = "HEADING"
FrontColName = "DIM_A"
BackColName = "DIM_B"
LeftColName = "DIM_C"
RightColName = "DIM_D"

# Prepare data value cursors for reading
vesselDataTable = Document.Data.Tables[VesselDataTableName]
idCursor = DataValueCursor.CreateFormatted(vesselDataTable.Columns[IdColName])
latCursor = DataValueCursor.CreateNumeric(vesselDataTable.Columns[LatColName])
lonCursor = DataValueCursor.CreateNumeric(vesselDataTable.Columns[LonColName])
frontCursor = DataValueCursor.CreateNumeric(vesselDataTable.Columns[FrontColName])
backCursor = DataValueCursor.CreateNumeric(vesselDataTable.Columns[BackColName])
leftCursor = DataValueCursor.CreateNumeric(vesselDataTable.Columns[LeftColName])
rightCursor = DataValueCursor.CreateNumeric(vesselDataTable.Columns[RightColName])
headingCursor = DataValueCursor.CreateNumeric(vesselDataTable.Columns[HeadingColName])

Before data reading, initialise the sourceString variable to store all as a STDF format.  This already includes the necessary metadata configuration for the geometry column to be used on the Map Chart.  This is no need to perform additional configuration as described here.

# Prepares STDF file header.  This already includes the necessary metadata configuration for
# the geometry column to be used on the Map Chart.
sourceString = "\! filetype=Spotfire.DataFormat.Text; version=2.0;\r\n"
sourceString +="\! property=mapchart.columntypeid; category=Column; type=String;\r\n"
sourceString +="\! property=ContentType; category=Column; type=String;\r\n"
sourceString +="\! property=Name; category=Column; type=String;\r\n"
sourceString +="\! property=DataType; category=Column; type=String;\r\n"
sourceString +="\?;Geometry;XCenter;XMax;XMin;YCenter;YMax;YMin;\r\n"
sourceString +="\?;application/x-wkb;application/x-wkb;application/x-wkb;application/x-wkb;application/x-wkb;application/x-wkb;application/x-wkb;\r\n"
sourceString +=IdColName+";geometry;XCenter;XMax;XMin;YCenter;YMax;YMin;\r\n"
sourceString +="String;Binary;Double;Double;Double;Double;Double;Double;\r\n"

Then reads vessel data by row, generate the shape as WKB and add it along with other information to the sourceString variable.  We will go into the details of the drawVesselWKB() function later on but the result is a Base64 encoded string of the vessel shape in WKB polygon.  As mentioned previously, the front, back, left and right values are optional and you can use constant values, for example, front = 120, back = 40, left = 15 and right = 15 if you wish.

for row in vesselDataTable.GetRows(idCursor,latCursor,lonCursor,frontCursor,backCursor,leftCursor,rightCursor,headingCursor):
    rowIndex = row.Index
    id = idCursor.CurrentValue
    lat = latCursor.CurrentValue
    lon = lonCursor.CurrentValue
    front = frontCursor.CurrentValue
    back = backCursor.CurrentValue
    left = leftCursor.CurrentValue
    right = rightCursor.CurrentValue
    heading = headingCursor.CurrentValue
    wkb = drawVesselWKB(lon, lat, heading, front, back, left, right)
    vesselString = id+";\#"+wkb+";"+str(lon)+";"+str(lon)+";"+str(lon)+";"+str(lat)+";"+str(lat)+";"+str(lat)+";\r\n"
    sourceString += vesselString

Finally writes the sourceString as a separate data table (with the "_Geo" suffix which can be changed) using the StdfFileDataSource.  Replaces the data table if one already exists.

# make a stream from the string
stream = MemoryStream()
writer = StreamWriter(stream)
writer.Write(sourceString)
writer.Flush()
stream.Seek(0, SeekOrigin.Begin)

dataSource = StdfFileDataSource(stream)

# add the data into a Data Table in Spotfire
geoDataTableName = VesselDataTableName+"_Geo"
if Document.Data.Tables.Contains(geoDataTableName):
    Document.Data.Tables[geoDataTableName].ReplaceData(dataSource)
else:
    newTable = Document.Data.Tables.Add(geoDataTableName, dataSource)
    tableSettings = DataTableSaveSettings (newTable, False, False)
    Document.Data.SaveSettings.DataTableSettings.Add(tableSettings)

---

DrawVesselWKB Function Details

The vessel shape is made up of a rectangle to represent the body and a triangle to represent the head.  Five points p1 to p5 are to be generated as illustrated in the diagram below.  These points are traversed from the origin point which is specified by the lon and lat parameters at an angle specifed by the heading parameter with the vessel size specified by the front, back, left and right parameters.  The headLength is set as 1/4 of total length.

The function to traverse from one point to another point using an angle in degrees and distance in meters is translate(). 

# Translate source coordinates by angle in degrees and distance in meters
# This returns the new coordinates in longitude and latitude degrees
def translate(source, angle, distance):
    sourceLon = source[0]
    sourceLat = source[1]

    distanceInLon = distance * math.sin(math.radians(angle))
    distanceInLat = distance * math.cos(math.radians(angle))

    # constant ratio calculated by (pi/180) * earth_radius
    metersPerDegree = 111320.0

    # Calculate new Lon Lat coordinates based on distances in meters
    # https://stackoverflow.com/questions/7477003/calculating-new-longitude-latitude-from-old-n-meters
    newLon = sourceLon + (distanceInLon / metersPerDegree) / math.cos(sourceLat * math.pi/180)
    newLat = sourceLat + (distanceInLat / metersPerDegree)
    return newLon, newLat

The implementation of the DrawVesselWKB() function is listed as below.  Note that to draw a polygon, the end point needs to be the same as the starting point, therefore 6 points are required to draw a polygon of 5 points.  Change this function accordingly to your desired vessel shape.  You may create more complex shapes with polygons and multi-polygons as specified in WKB.  Here are some references
https://markyourfootsteps.wordpress.com/tag/wkb/
https://en.wikipedia.org/wiki/Well-known_text_representation_of_geometry
Make sure the WKB header (e.g. no of parts and no of points) is changed accordingly to align with the coordinates being included in the WKB.

# Renders a vessel based on Lat Lon coordiantes, heading and size
# This returns the WKB representation (encoded in Base64) of the vessel 
# shape which is made up of a rectangle (3/4 of length) and 
# a triangle (1/4 of length) 
def drawVesselWKB(lon, lat, heading, front, back, left, right):
    # Starts a binary stream
    stream = MemoryStream()
    writer = BinaryWriter(stream)

    # Writes the WKB header for a filled polygon with one part
    # and 6 points.
    #                +--------+---------+--------+--------+
    # Description  : | Byte   | Type    | No of  | No of  |
    #                | Order  | Polygon | Parts  | Points |
    #                +--------+---------+--------+--------+
    # Size         : | 1-Byte | 4-byte  | 4-byte | 4-byte |
    #                +--------+---------+--------+--------+
    # Value        : |    1   |  3000   |  1000  |  6000  |
    #                +--------+---------+--------+--------+
    writer.Write(Array[Byte]([1,3,0,0,0,1,0,0,0,6,0,0,0]))
    headLength = (front+back) / 4.0 * 1.0

    # Generate each of five points of the vessel shape
    origin = lon, lat
    p1 = translate(translate(origin, heading+90.0, right), heading, front - headLength)
    p2 = translate(translate(p1, heading, headLength), heading-90.0, (left+right)/2)
    p3 = translate(p1, heading-90.0, left+right)
    p4 = translate(p3, heading+180.0, front-headLength+back)
    p5 = translate(p4, heading+90.0, left+right)
    
    # Write to the binary stream of the generated five points and then back to the 
    # starting point to close the polygon. 
    writer.Write(BitConverter.GetBytes(p1[0]))
    writer.Write(BitConverter.GetBytes(p1[1]))
    writer.Write(BitConverter.GetBytes(p2[0]))
    writer.Write(BitConverter.GetBytes(p2[1]))
    writer.Write(BitConverter.GetBytes(p3[0]))
    writer.Write(BitConverter.GetBytes(p3[1]))
    writer.Write(BitConverter.GetBytes(p4[0]))
    writer.Write(BitConverter.GetBytes(p4[1]))
    writer.Write(BitConverter.GetBytes(p5[0]))
    writer.Write(BitConverter.GetBytes(p5[1]))
    writer.Write(BitConverter.GetBytes(p1[0]))
    writer.Write(BitConverter.GetBytes(p1[1]))
    writer.Flush()

    # Return as Base64 string
    return Convert.ToBase64String(stream.ToArray())

---

Map Chart Configurations

Last but not least, this part suggests the configuration required on the map chart.

You may include both the Marker layer (with standard marker shapes) and Feature layer (with rendered vessel shapes) and leverage the "Zoom Visbility" feature on the Map Chart as shown below.  This allows markers to be shown at a high zoom level and the vessel shapes to be shown at a lower zoom level.

Edited August 20 by Mathew

[PBR]

Thank you @Mathew Lee.
I really appreciate the time and effort you put into preparing such a detailed response. I’ll try out the solution soon and will get back to you with my results.