Performance of importData(9)

Performance of importData(9) - Сообщения

#1 Опубликовано: 19.08.2013 12:16:38
Martin Kraska

Martin Kraska

1 260 сообщений из 2 186 понравились пользователям.

Группа: Moderator

Perhaps not a bug, but a problem. The importdata(9) function has a quadratic dependence of time consumption versus datasets read. For large files this becomes prohibitive. Pre-allocation of the matrix does not help.

If you read the file piecewise into separate matrices, then the time consumption is
linear. However, combining them using stack() turns out to be even more ineffective. Other operations like column extraction or plotting augmented columns fast in comparison to read operation. Therefore, an efficient text file reading routine would be highly welcome.

If you want to try the example, put data and .sm file in the same directory and launch the .sm file there.
readtest.PNG
Martin KraskaPre-configured portable distribution of SMath Studio: https://en.smath.info/wiki/SMath%20with%20Plugins.ashx
1 пользователям понравился этот пост
Radovan Omorjan 19.08.2013 12:46:00
#2 Опубликовано: 08.09.2013 06:17:20
Martin Kraska

Martin Kraska

1 260 сообщений из 2 186 понравились пользователям.

Группа: Moderator

Problem solved. The quadratic time dependence holds for optimization> symbolic. For optimization> numeric, the time dependence is approximately linear.
DataExchange.zip (381,39 КиБ) скачан 156 раз(а).
Martin KraskaPre-configured portable distribution of SMath Studio: https://en.smath.info/wiki/SMath%20with%20Plugins.ashx
  • Новые сообщения
  • Нет новых сообщений