Shell Programming and Scripting

Hi,

I am doing some data parsing for some economics research. I was recently exposed to shell script and am brand new to awk. I have a large csv file (like 10G) and I would like to make it a lot smaller with awk, but it is a bit tricky for me and I haven't been able to get it yet. I would like to keep the first four columns, in then in the newly created csv file have the fifth column contain whatever "cell" (separated by commas) in the original file contains the string "}".

Can anyone provide some help?

Thank you very much.

didn't get you clearly can you post some i/p and expected o/p for us

Sure. I apologize for any confusion. I will just make up these strings to simplify it, but the structure is the same as my data set. Basically, I need the first four "cells" and the last "cell" of each line but the position of the last cell varies. Thank you for any help.

Input (via csv file):


Name1, year1, day1, amount1.1, amount1.2, amount1.3}
Name1, year1, day2, amount2.1, amount2.2, amount2.3}
Name2, year1, day1, amount1.1, amount1.2, amount1.3, amount1.4}
Name2, year1, day2, amount2.1, amount2.2, amount2.3, amount2.4}
Name3, year1, day1, amount1.1, amount1.2, amount1.3}
Name3, year1, day2, amount2.1, amount2.2}

Output (via csv file):

Name1, year1, day1, amount1.1, amount1.3}
Name1, year1, day2, amount2.1, amount2.3}
Name2, year1, day1, amount1.1, amount1.4}
Name2, year1, day2, amount2.1, amount2.4}
Name3, year1, day1, amount1.1, amount1.3}
Name3, year1, day2, amount2.1, amount2.2}

ok gotcha try this

Code:

awk -F"," 'BEGIN{OFS=","}{print $1$2$3$4$NF}' inputfile > outputfile

Thank you! It appears to be working. As a final question, is there a way to "delete" (not use in the output file) the last character of every last cell and the first three characters of cells 1-4?

you can try something like this

Code:

 
awk -F"|" 'BEGIN{OFS=","}{print substr($1,4)substr($2,4)substr($3,4)substr($4,4)substr($NF,1,length($NF)-1)}' infile