1 страниц (5 вхождений)
Time for calculating the variance() & stddev() from csv file - Сообщения
#1 Опубликовано: 03.04.2024 20:10:07
Hi SMath community,
I would like to calculate the variance() and StdDev() of the second column "V(3v3)" of the attached file with 50k datapoints. Unfortunately the calculation takes forever (aborted it after 30min).
The file is an export of a circuit simulation result (gaussian tolerance distribution in LTSpice). Originally it is in .txt-Format and it seems that I have to convert it in excel to .csv-Format before I can use is. I attached both, the original LTSpice export (.txt) and the converted one from Excel (.csv).
How can I handle this in sMath?
Archiv.zip (585 КиБ) скачан 31 раз(а).
Thanks in advance,
Marc
I would like to calculate the variance() and StdDev() of the second column "V(3v3)" of the attached file with 50k datapoints. Unfortunately the calculation takes forever (aborted it after 30min).
The file is an export of a circuit simulation result (gaussian tolerance distribution in LTSpice). Originally it is in .txt-Format and it seems that I have to convert it in excel to .csv-Format before I can use is. I attached both, the original LTSpice export (.txt) and the converted one from Excel (.csv).
How can I handle this in sMath?
Archiv.zip (585 КиБ) скачан 31 раз(а).
Thanks in advance,
Marc
#2 Опубликовано: 03.04.2024 22:34:03
With import file functions you might need to set the optimization to numeric to speed up the loading.
If you like my plugins please consider to support the program buying a license; for personal contributions to me: paypal.me/dcprojects
#3 Опубликовано: 03.04.2024 23:32:05
Optimizations were all set to numeric - did not help. What helped was that I substituted the statistic function variance() and StdDev with the original/native formula for the variance and standard deviation.
This speeds it up to around 1s.
Don´t know what happens inside the functions of the statistic plugin but it is very slow.

Thanks
This speeds it up to around 1s.
Don´t know what happens inside the functions of the statistic plugin but it is very slow.
Thanks
#4 Опубликовано: 11.04.2024 06:24:17
I have noticed the same thing; the built-in function for standard deviation takes forever but the calculation runs pretty quickly if I just code in the formula in the worksheet.
#5 Опубликовано: 12.04.2024 16:53:41
Confirmed, the bottleneck seems to be in the final ouput processing to return the result from the equation [f.e. 1/{n-1}*sum(((el(X,i)-mean)^2),i,1,n)) for StdDev]; no idea not sure what could be the cause...
If you like my plugins please consider to support the program buying a license; for personal contributions to me: paypal.me/dcprojects
1 страниц (5 вхождений)
-
Новые сообщения
-
Нет новых сообщений