Need your help in Iterate, Aggregation of String [StreamBase...

[Manoj Chaurasia]

Hi,

Need your help in Iterate, Aggregation of String [streamBase 7.5.0]

to convert the input Hawkfeed string into desired output schema.

Input Schema:

_data_ (DataType String)

Input Record:

_data_=HostName:TIBXCEPDEV220,CPUID:22,UpTime:2015-02-02 15:34:34.456+530,DiskUsage:30,DiskFree=70

Output Schema:

HostName (DataType String) (Example: TIBXCEPDEV220)

CPUID (DataType int) (Example: 22)

UpTime (DataType timestamp) (Example: 2015-02-02 15:34:34.456+0530)

DiskUsage (DataType int) (Example: 30)

DiskFree (DataType int) (Example: 70)

[Steve Barber]

I won't work the conversion logic out in detail, but the general approach I might take would be to convert that string into valid JSON with string operations (using perhaps Iterate and/or maplist(), split(), regexsplit() invocations), and then pass both the schema and the resulting JSON string to the parsejson() or parsejson_loose() function.

 

I think there's a lot of ways this could go wrong in practice, so the parsing logic would need quite a bit of validation for missing fields, bad values, nulls, etc. Note also the use of two different delimiters (: and =) in different strings that has to be accounted for.

The larger risk here is that we have just one sample string to parse -- if the other strings in the stream have a different complement of embedded field value pairs to parse or field names show up that aren't in the StreamBase schema, then the task becomes rather difficult to predict.

 

Another approach would be to parse the string into a list of sub-tuples with two fields like name and value to handle an arbitrary set of fields. This approach is more flexible but certainly less convenient, and also doesn't automatically deal with field types.