Performance of importData(9)

Performance of importData(9) - Сообщения

#1 Опубликовано: 19.08.2013 12:16:38
Martin Kraska

Martin Kraska

1222 сообщений из 2150 понравились пользователям.

Группа: Moderator

Perhaps not a bug, but a problem. The importdata(9) function has a quadratic dependence of time consumption versus datasets read. For large files this becomes prohibitive. Pre-allocation of the matrix does not help.

If you read the file piecewise into separate matrices, then the time consumption is
linear. However, combining them using stack() turns out to be even more ineffective. Other operations like column extraction or plotting augmented columns fast in comparison to read operation. Therefore, an efficient text file reading routine would be highly welcome.

If you want to try the example, put data and .sm file in the same directory and launch the .sm file there.
readtest.PNG
Martin Kraska Pre-configured portable distribution of SMath Studio: https://en.smath.info/wiki/SMath%20with%20Plugins.ashx
1 пользователям понравился этот пост
Radovan Omorjan 19.08.2013 12:46:00
#2 Опубликовано: 08.09.2013 06:17:20
Martin Kraska

Martin Kraska

1222 сообщений из 2150 понравились пользователям.

Группа: Moderator

Problem solved. The quadratic time dependence holds for optimization> symbolic. For optimization> numeric, the time dependence is approximately linear.
DataExchange.zip (381 КиБ) скачан 53 раз(а).
Martin Kraska Pre-configured portable distribution of SMath Studio: https://en.smath.info/wiki/SMath%20with%20Plugins.ashx
  • Новые сообщения Новые сообщения
  • Нет новых сообщений Нет новых сообщений