How to set text data with linebreak to a datatable in spotfire using ironpython

[Bruno Martins]

I'm getting data from an http request using ironpython to feed the datatable in spotfire but I'm having trouble with data with linebreaks. The data I'm getting from the HTTP request comes as tsv, so the linebreaks are represented as rn. That data has a description field that has linebreaks as well. So when parsing the data it breaks into newlines that doesnt exist. I'm retrieving currently one result but after parsing it it becomes several lines in the datatable.

I need to show the description text with linebreaks in spotfire.

The service I'm getting the data from was made by myself. I was sending a JSON response with the data but I had to parse it to a tsv in ironpython so I transfered this job to the server.

If doing this is not possible is there a way to spotfire to read data from a request and pass it do the datatable without converting to tsv

there is a parameter for the separator readerSettings.Separator = "t" but I couldn't find one for the linebreak. Is there a parameter for TextDataReaderSettings() that sets the linebreak

 

import clr

clr.AddReference('System.Data')

clr.AddReference('System.Web.Extensions')

import System

from System import DateTime

from System.Data import DataSet, DataTable

from System.IO import StreamReader, StreamWriter, MemoryStream, SeekOrigin

from System.Net import HttpWebRequest

from System.Web.Script.Serialization import JavaScriptSerializer

from Spotfire.Dxp.Data import DataType, DataTableSaveSettings

from Spotfire.Dxp.Data.Import import TextFileDataSource, TextDataReaderSettings

 

def getDataTable(tableName):

try:

return Document.Data.Tables[tableName]

except:

return None

 

uri = "http://localhost/comentarioindicadores"

webRequest = HttpWebRequest.Create(uri)

response = webRequest.GetResponse()

streamReader = StreamReader(response.GetResponseStream())

data = streamReader.ReadToEnd()

 

# build a string representing the data in tab-delimited text format

textData = "idtindicadortmstanotcomentriotmotivotaotcriado emtatualizado emrn"

textData += data

 

# make a stream from the string

stream = MemoryStream()

writer = StreamWriter(stream)

writer.Write(textData)

writer.Flush()

stream.Seek(0, SeekOrigin.Begin)

 

# set up the text data reader

readerSettings = TextDataReaderSettings()

readerSettings.Separator = "t"

readerSettings.AllowNewlinesInQuotedFields = True

readerSettings.AddColumnNameRow(0)

 

readerSettings.SetDataType(1, DataType.String)

readerSettings.SetDataType(2, DataType.Integer)

readerSettings.SetDataType(3, DataType.Integer)

readerSettings.SetDataType(4, DataType.String)

readerSettings.SetDataType(5, DataType.String)

readerSettings.SetDataType(6, DataType.String)

readerSettings.SetDataType(7, DataType.DateTime)

readerSettings.SetDataType(8, DataType.DateTime)

 

# create a data source to read in the stream

textDataSource = TextFileDataSource(stream, readerSettings)

 

# add the data into a Data Table in Spotfire

dt = getDataTable("comentarios")

if dt != None:

dt.ReplaceData(textDataSource)

else:

Document.Data.Tables.Add("comentarios", textDataSource)

[Sayali Patil]

Hello,

 

Can you please share a sample dataset (TSV output which you receive from your service) for reference here

 

Thx!

[Bruno Martins]

Example data, single row:

 

1tIASMt1t2018tLorem ipsum dolor sit amet, consectetur adipisicing elit. <br><br>Aperiam sequi nulla assumenda fugiat temporibus facilis, deserunt harum asperiores veritatis sunt, architecto velit minima alias recusandae autem! Excepturi minus at quosrn

 

where the <br> tags were the replaced rn the text had due to the user hitting enter key to break lines. I needed to do add the tag in order to have proper distinction between rows. If you replace the <br> with rn this single row will become two rows after the text processing at ironpython, which is not correct. I still have not managed to show this text with html tag <br> so I'm still searching for a solution for this problem.