expression language vs. built-in TERR expression functions vs. expression functions: inconsistent results with qq-plot data

[Mark Herrmann]

Hi there,

working of this template I wanted to create an easier to use expression function. I thought this was going to be straight forward but playing with it I realized that on small data sets the results from the method proposed by Tibco and my expression function differ. To understand the behaviour I created a copy of my expression function inside a TERR_Real expression and surprisingly found I get yet another different result.

Long story short, I used three methods out of which the last two should actually be identical but the results are not identical on small data sets:

 

As per community template: -NormInv((Rank([Value]) - 0.5) / Count([Value]))

Using R's qqnorm: qqnorm(Value)[[1]] in

 

an expression function and

a TERR_Real expression

 

 

 

I have put this all in the attached dxp. Not sure this is a bug in expression functions or TERR_Real or some seed thing or me being stupid but if anyone's got an idea what's happening and what flavour is most trustworthy that'd be highly appreciated.

Thanks,

Mark

[Gaia Paolini]

Your definition of p_expression_function_10 still uses [Value_100]. Once you fix that, the expression function and the TERR_real columns are identical.

The method from the community looks more approximated in computing quantiles, so I would expect the resultsto have some differences from qqnorm forthis very small dataset.

In your test dataset, you have only 9 points, of which 3 are identical in pairs (so 6 distinct values). It is for thesepairs of identical values that you see the discrepancies.

But these differences look to me really small considering the small dataset is too small to have a statistically significant distribution, and the large dataset has 1000 values.

[Mark Herrmann]

Thanks, Gaia! I knew it was something really stupid. The most puzzling issue was the "difference" between TERR_Real and the Expression Function so thanks for spotting that!