1 Pages (5 items)
Time for calculating the variance() & stddev() from csv file - Messages
#1 Posted: 4/3/2024 8:10:07 PM
Hi SMath community,
I would like to calculate the variance() and StdDev() of the second column "V(3v3)" of the attached file with 50k datapoints. Unfortunately the calculation takes forever (aborted it after 30min).
The file is an export of a circuit simulation result (gaussian tolerance distribution in LTSpice). Originally it is in .txt-Format and it seems that I have to convert it in excel to .csv-Format before I can use is. I attached both, the original LTSpice export (.txt) and the converted one from Excel (.csv).
How can I handle this in sMath?
Archiv.zip (585 KiB) downloaded 31 time(s).
Thanks in advance,
Marc
I would like to calculate the variance() and StdDev() of the second column "V(3v3)" of the attached file with 50k datapoints. Unfortunately the calculation takes forever (aborted it after 30min).
The file is an export of a circuit simulation result (gaussian tolerance distribution in LTSpice). Originally it is in .txt-Format and it seems that I have to convert it in excel to .csv-Format before I can use is. I attached both, the original LTSpice export (.txt) and the converted one from Excel (.csv).
How can I handle this in sMath?
Archiv.zip (585 KiB) downloaded 31 time(s).
Thanks in advance,
Marc
#2 Posted: 4/3/2024 10:34:03 PM
With import file functions you might need to set the optimization to numeric to speed up the loading.
If you like my plugins please consider to support the program buying a license; for personal contributions to me: paypal.me/dcprojects
#3 Posted: 4/3/2024 11:32:05 PM
Optimizations were all set to numeric - did not help. What helped was that I substituted the statistic function variance() and StdDev with the original/native formula for the variance and standard deviation.
This speeds it up to around 1s.
Don´t know what happens inside the functions of the statistic plugin but it is very slow.

Thanks
This speeds it up to around 1s.
Don´t know what happens inside the functions of the statistic plugin but it is very slow.
Thanks
#4 Posted: 4/11/2024 6:24:17 AM
I have noticed the same thing; the built-in function for standard deviation takes forever but the calculation runs pretty quickly if I just code in the formula in the worksheet.
#5 Posted: 4/12/2024 4:53:41 PM
Confirmed, the bottleneck seems to be in the final ouput processing to return the result from the equation [f.e. 1/{n-1}*sum(((el(X,i)-mean)^2),i,1,n)) for StdDev]; no idea not sure what could be the cause...
If you like my plugins please consider to support the program buying a license; for personal contributions to me: paypal.me/dcprojects
1 Pages (5 items)
-
New Posts
-
No New Posts