Jump to content

How to load JSON format data to LiveView tables

Krishna Chaitanya 4

Recommended Posts

I was getting the below data in a format of jason consuming from kafka topic. How do i do this , convert this data and load into my live view table

If there's more than one approach, I would like to know them...please brief about the approach and steps in detail.

[12544:OperatorThread(Publisher.KafkaFedTableProvider.ReadFromBus.KafkaConsumer)]INFO com.streambase.sb.adapter.kafka.KafkaConsumer: Deserializing tuple for topic 'zer.kamel.wbs.extract' from string:{"rountenbr":"FX77","routedt":{"dayOfMonth":5,"dayOfWeek":"TUESDAY","dayOfYear":126,"monthValue":5,"nano":0,"hour":0,"second":0,"minute":0,"year":2020,"month":"MAY","chronology":{"id":"ISO","calendarType":"iso8601"}},"pieces":2,"declnval":553.53,"mdetmstp":{"dayOfMonth":30,"dayOfWeek":"THURSDAY","dayOfYear":121,"monthValue":4,"nano":0,"hour":23,"second":29,"minute":26,"year":2020,"month":"APRIL","chronology":{"id":"ISO","calendarType":"iso8601"}},"cifirstmstp":null,"cilastmstp":null,"awbfirstmstp":null}

Link to comment
Share on other sites

  • 2 weeks later...

We have created some tutorial videos to show how to automatically generate an application that discovers JSON format of a Kafka message payload, and generate a corresponding LiveView data table: https://community.spotfire.com/wiki/spotfire-data-streams-quick-start-video-series.

Watch the SFDS-101 Introduction and then SFDS-103 that is specific to Apache Kafka topics as a data source. Spotfire Data Streams and LiveView are the same underlying technology, so these video also apply to a LiveView application that's not being used with Spotfire.

That said, the automated schema generation in the Connectivity Wizard only completely handles completely flat JSON schemas (because LiveView data table field types are only simple types and are always flat schemas as well). So, while I would still use the Connectivity Wizard to get started to set up my Studio projects, I would then go into the generated publisher .sbapp and add some parsejson_loose() function calls or JSONToTuple operators to handle those date/time objects in the schema you've presented above and convert them to a native LiveView timestamp value.

This is just an outline of a way to handle this situation. It's been a while since you posted the question so if you are still interested, please comment on this answer if you'd like more detail about how to do this.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Create New...