Jump to content

Need your help in Iterate, Aggregation of String [StreamBase...

Manoj Chaurasia

Recommended Posts


Need your help in Iterate, Aggregation of String [streamBase 7.5.0]

to convert the input Hawkfeed string into desired output schema.

Input Schema:

_data_ (DataType String)

Input Record:

_data_=HostName:TIBXCEPDEV220,CPUID:22,UpTime:2015-02-02 15:34:34.456+530,DiskUsage:30,DiskFree=70

Output Schema:

HostName (DataType String) (Example: TIBXCEPDEV220)

CPUID (DataType int) (Example: 22)

UpTime (DataType timestamp) (Example: 2015-02-02 15:34:34.456+0530)

DiskUsage (DataType int) (Example: 30)

DiskFree (DataType int) (Example: 70)

Link to comment
Share on other sites

  • 5 years later...

I won't work the conversion logic out in detail, but the general approach I might take would be to convert that string into valid JSON with string operations (using perhaps Iterate and/or maplist(), split(), regexsplit() invocations), and then pass both the schema and the resulting JSON string to the parsejson() or parsejson_loose() function.


I think there's a lot of ways this could go wrong in practice, so the parsing logic would need quite a bit of validation for missing fields, bad values, nulls, etc. Note also the use of two different delimiters (: and =) in different strings that has to be accounted for.

The larger risk here is that we have just one sample string to parse -- if the other strings in the stream have a different complement of embedded field value pairs to parse or field names show up that aren't in the StreamBase schema, then the task becomes rather difficult to predict.


Another approach would be to parse the string into a list of sub-tuples with two fields like name and value to handle an arbitrary set of fields. This approach is more flexible but certainly less convenient, and also doesn't automatically deal with field types.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Create New...