I am having trouble importing data from a csv file with unequal numbers of columns per line, as well as having trouble importing time and date in the following format, is anyone able to help ?

The only ways I have been able to solve this is to open in notepad, and do search and replaces on "-", "+00:00" and to remove the "T" (before the time), and then replace all these with ","
The 2nd problem is interpreting the blank entries which vary in number per line.... again using notepad to do a search and replace is slow and clunky.

2019-09-29T19:17:44+00:00,,8,513024,17.6,30,14,6.85,84,TRUE,FALSE
2019-09-29T18:50:29+00:00,,,0,0,0,15,6.975,0,FALSE,FALSE
2019-09-29T18:23:17+00:00,,,0,0,0,16,6.975,0,FALSE,FALSE
2019-09-29T17:56:08+00:00,,,0,0,0,18,6.825,0,FALSE,FALSE
........
...
..

I also have the problems of importing another type of csv file :
;;;;"";"30100f05802807d40e549a01";;;;;;"";"0000";"Limit";"N/A";"IrVT";"-139.00";"12.19";"142B";"-1.0";"3";"N/A";"2.509";"-139.00";"9.06";"2253";"-1.0";"2";"N/A";"0.822";"-143.00";"6.07";"2254";"-1.0";"2";"N/A";"1.657";;;;;;;;"1414";"2019-09-30 08:17:44"

and as par of this i need to process the hex data :"30100f05802807d40e549a01", again any help in importing and processing hex data would be appreciated.

Surely there is a way to read in such csv files and parse the data thru various rules before processing and graphing ??

Hi Mark,

I don't think you're going to be able to accomplish what you want directly within SMATH. The importData function doesn't handle text well unless it is surrounded by quotation marks. Additionally, I could not get it to work with empty columns either.

You're best bet right now is to import to excel and parse the CSV that way. They also have a built in hex converter. SMath doesn't have anything natively, but you could probably write a function that take a hex string and converts it to whichever base you'd like.

Wrote

I am having trouble importing data from a csv file

Alternately, build each data matrix manually.
Global Warming in on of those I constructed ... > ½ day.
Data set from source is *.dat, just numbers.

in my experience, its easier to manipulate in a spreadsheet program instead of SMath.

do all your cleaning up and reformatting there, even convert it to hex strings then import it into smath.

you can also cleverly make the matrix syntax in excel to copy and past into smath where the list is simply ordered of the row and column values and last two values are number of rows and columns. here is an example-

mat(r1c1,r1c2,r2c1,r2c2,r3c1,r3c2,r4c1,r4c2,4,2)

for 4 rows and 2 columns

As was already mentioned, you can import to xlsx using excel with comma as column delimiter and then import to SMath Studio.

BTW, there is a glitch in mouse handling of the matrix resizer. It seems to compute the mouse drag required for resizing based on width of the first column.

importtest.sm (3 KiB) downloaded 148 time(s).

test.xlsx (11 KiB) downloaded 89 time(s).

Thanks for all the suggestions.... I think the xlsx option will be manageable for now - thanks.
Shame the importData function is not a bit more forgiving.

Mark-
How do you mean 'more forgiving'?
What is it that you are experiencing or it not doing that you are expecting?
Some of us might be suffering from expert-blindpot.

I was meaning that the csv import is quite restrictive in the type and format of data it accepts.
The particular csv file has a different number of null entries per line, and the dataimport.csv function just ignores these rather than filling the matrix column with something like "" or 0
It also seems to dislike mixed types of data : some columns text, and some numeric
I could handle the date and time if inherently able to be imported as string.

For this data analysis job I was trying to avoid the extra step of importing the csv to excel, massaging the data in excel and then saving before bringin into SMath.

You can use the Script component to parse any file as you want. But this will require some programming knowledge.

Wrote

I was meaning that the csv import is quite restrictive in the type and format of data it accepts.
The particular csv file has a different number of null entries per line, and the dataimport.csv function just ignores these rather than filling the matrix column with something like "" or 0
It also seems to dislike mixed types of data : some columns text, and some numeric
I could handle the date and time if inherently able to be imported as string.

For this data analysis job I was trying to avoid the extra step of importing the csv to excel, massaging the data in excel and then saving before bringing into SMath.

Puzzling your comment ... maybe from punch card days.
The particular csv file has a different number of null entries per line, and the dataimport.csv function just ignores these rather than filling the matrix column with something like "" or 0

like "" or 0 ... or NaN

Hello,

I have also troubles with importing a csv file. It works fine with numbers:


But doesnt work with strings: