UNIX for Dummies Questions & Answers

Hello! Graduate student thrown to the sharks of unix and fortran77 here.

My basic question is this: what exactly is the difference between yank (ie "*yG for the whole file and "*p in a new file) and control-insert / shift-insert with respect to text formatting? One is vi and one is Windows, according to wikipedia, and one can get the whole file and the other leads to monotonous copy, paste, scroll, rinse, repeat. BUT, for my purposes, the way control-insert reformats the text upon pasting is different than yank, and it's something I actually want to reproduce in bulk for the rest of the file.

Some background: I needed to pare down the grid size of a csv data file to match new inputs, so I did it in a rather tedious but effective way in MATLAB using dlmwrite(fout, [datanames],'-append', 'newline','unix') within an if-loop for my parameters. The resulting file (let's call it problemchild.csv) looks like this:
1422,52,1,24.881,-88.113,0,300,293,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0
1423,53,1,24.85,-87.766,0,300,293,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0
and so on.

The parent data-reading program (MEGAN) refuses to read problemchild.csv, though, insisting
>>--->> WARNING in subroutine STR2INT
No digits in "^@^@^@ã6²?7 "
even though the file I made appears to be in exactly the same format, even when looking at cat -v problemchild.csv, as the larger original. I suspect because I wrote it in dlmwrite that the file is characterized as numerical, not a string, even though it looks the same. I would rather not edit the parent program, however, because it uses STR2INT in ways I'm not sure I could reproduce if I took the commands out. So, I'd like to figure out how to copy over my problemchild text into a file that will pass STR2INT. Trying to copy and paste in a quick manner (ie cat problemchild.csv >> newfile.csv OR "*yG and "*p) reproduces the same error, but control-insert and shift-insert for each line of the file produces something the parent program will actually read. Except that the problemchild file is 1800+ lines long, and I'd really prefer not to convert it manually...

I've tried to search around but I'm still pretty new at this and I haven't found much that has been helpful. I'm using VIM - Vi IMproved 6.3 (2004 June 7, compiled Jan 11 2005 10:55:42) and it's not linked to my clipboard via xterm_clipboard, if that is useful at all. I'm kind of at the end of my rope with this so any help would be appreciated. Thanks!

Marshlight,
Not sure of what you meant by paring down the grid size... but here is a easy way to reformat a delimited file (this assumes your data values do not contain embedded comma's in the data, they are just used for separators).

Given your sample data, contained in file "sampledata":
1422,52,1,24.881,-88.113,0,300,293,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0
1423,53,1,24.85,-87.766,0,300,293,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0

This example takes the first 5 fields and saves to a new "result" file. Note the delimiter is the comma, the printf command is used... 5 "%s" format strings are used, and you pass the first 5 fields: $1 through $5:

Code:

Code:

awk -F\, '{printf("%s,%s,%s,%s,%s\n",$1,$2,$3,$4,$5)}' samplefile > result

Result file contains:

Code:

1422,52,1,24.881,-88.113
1423,53,1,24.85,-87.766

By adjusting the number of "%s" formats and the field positions you want, you should be able to pare-down the data the way you want.

Thanks for the reply! I had taken a 111x147 gridmap of the above format (I do need all the columns eventually) and shortened it to 90x97 - that's what I meant by paring down. But your instructions may be useful in the future!

After I read what I wrote, I realized it sounded crazy, so I went digging for other reasons why things might not be working. It appears that there's another file that is connected to what str2int reads, and if everything (ie gridmap definition) doesn't exactly match, it doesn't work and defaults to the error above. Not a particularly enlightening system but it gives me something to go on. In short, nevermind...