Help in Deleting columns and Renaming Mutliple columns in a .Csv File


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Help in Deleting columns and Renaming Mutliple columns in a .Csv File
# 1  
Old 03-15-2012
Help in Deleting columns and Renaming Mutliple columns in a .Csv File

Hi All,
i have a .Csv file in the below format

Code:
startTime, endTime, delta, gName, rName, rNumber, m2239max, m2239min, m2239avg, m100016509avg, m100019240max, metric3min, m100019240avg, propValues
11-Mar-2012 00:00:00, 11-Mar-2012 00:05:00, 300.0, vma3550a, a-1_CPU Index<1>, 200237463, 0.0, 0.0, 0.0, , , , ,
11-Mar-2012 00:05:00, 11-Mar-2012 00:10:00, 300.0, vma3550a, a-1_CPU Index<1>, 200237463, 0.0, 0.0, 0.0, , , , ,
11-Mar-2012 00:10:00, 11-Mar-2012 00:15:00, 300.0, vma3550a, a-1_CPU Index<1>, 200237463, 0.0, 0.0, 0.0, , , , ,

i want to delete 3rd and 14th field and then the following field headers are to be renamed

Code:
m2239max, m2239min, m2239avg, m100016509avg, m100019240max, metric3min, m100019240avg

My expected file format should be look like this

Code:
startTime, endTime, gName, rName, rNumber, M1max, M1min, M1avg, M2avg, M3max, M3min, M3avg, 
11-Mar-2012 00:00:00, 11-Mar-2012 00:05:00, vma3550a, a-1_CPU Index<1>, 200237463, 0.0, 0.0, 0.0, , , , ,
11-Mar-2012 00:05:00, 11-Mar-2012 00:10:00, vma3550a, a-1_CPU Index<1>, 200237463, 0.0, 0.0, 0.0, , , , ,
11-Mar-2012 00:10:00, 11-Mar-2012 00:15:00, vma3550a, a-1_CPU Index<1>, 200237463, 0.0, 0.0, 0.0, , , , ,

I am newbie to Unix and sed.. Let me know if any inputs required.
This is kinda urgent.

Thanks in Advance,
Mahi

Moderator's Comments:
Mod Comment Please use code tags for program code or text data. Thank you.

Last edited by Corona688; 03-15-2012 at 12:34 PM..
# 2  
Old 03-15-2012
Hello;
I dont understand exactly which colums you want to delete.

But, format the output is easy with awk

# cat InputFile | awk -F"," '{print $2 $1 $4 $5 $6 }' > OutputFile

You can choose which colums you want or not.

Say me if it is not that you want

PS
# 3  
Old 03-15-2012
Try:
Code:
cut -d, -f-2,4-13 infile > newfile

And just edit the header by hand..
# 4  
Old 03-16-2012
@SolarMax:- Thank you for the inputs... But i want to delete 3rd and 14th field and then i want the following Field headers are to renamed

Code:
m2239max, m2239min, m2239avg, m100016509avg, m100019240max, metric3min, m100019240avg

The Code u provided will onlt delete and prints the columns but i will not rename the filed headers.

Columns deletion and Field header renaming should happen at single Go with in a script. Hope it is clear

let me know if any clarification required.

Thanks!
# 5  
Old 03-16-2012
Hello
#
#
"Delete the first line"
NBLIG=$(wc -l InputFile |awk '{print $1}')
let 'NBM1=NBLIG-1'
tail -$NBM1 InputFile > DataFileOnly
#
#
# Your new header
echo "m2239max, m2239min, m2239avg, m100016509avg, m100019240max, metric3min, m100019240avg" > NewFile
#
#
# Append
cat DataFileOnly >> NewFile
#
# 6  
Old 03-16-2012
Code:
{ echo "m2239max, m2239min, m2239avg, m100016509avg, m100019240max, metric3min, m100019240avg"
  cut -d, -f-2,4-13 | sed 1d
} < infile > newfile

-or-

Code:
echo "m2239max, m2239min, m2239avg, m100016509avg, m100019240max, metric3min, m100019240avg" > newfile
cut -d, -f-2,4-13 infile | sed 1d >> newfile


Last edited by Scrutinizer; 03-16-2012 at 04:24 AM..
# 7  
Old 03-16-2012
@SolarMax:-

Thanks for the help but my shell doesnt seem to support this kinda script. I use Bash shell. this peice of code is not being supportedSmilie
Request you to provide the code which is supported by Bash.

@Scrutinizer:- Good peice of Code but it does seem to work. the following junk has been written to newfile

\
Code:
317^E\\344\255y^^\265\346u\2011_]Uo\236wp\346S\216\371\224c>\345\230\3732^Ms\237`\356^S\314\233\323\304\374\265[\300\375^L\250\371\373KJ^W\367\301^X\326\377\2261\317s
^A^Wyg^F\235\271,8\233\257\315\373^W\354e^Gc>\346a>\346a>\346a^^9^?zS\3005I\2307I\230w\247\251yt6[\327\371\213^A5^?}\276\247$eK<\267\263\376\317\374\\300E\336\232\347Qk
^^W^X\363\325U\265\327yyt\316<\237\2409\346S\216\371\224c\356\313\264\270\316\233^Ds?\203ys\232\232Gg\263g\316\200\230\377\375Zk1\305<\227BO\371;^Q\257\341"o\315\363\25
05\257^K\214\371\352\252z\363\274\203\232\306^[\363\362\3505\307|\232\307|\3121\367e^Z\346>\301\334'\2307\247\251yt6{\346^L\250\371\275\277c\314+\227B\317s^A^Wyk\236G\2
55y]`\314WW\325\233\347^]\324^T\363)\307|\3121\237r\314}\231\206\271O0\367      \346\315ij^^\235\315\2369^Cbn\256\356\^E=^E\203\253\273\257\335"\372\373a`\245\353^B\331
<j\267\265\371\360S<\357\340\244\247^\\351)Gz\312\221\366^U^Z\322>A\332'H7\247\251t^T6{\322^L\250t\334
\374\247\2141M\314+7B\317s\373^Vy{u\347Qk^^W^X\363UT\365Ww\336\301\231O9\346S\216\371\224c\356\2334\314}\202\271O0oNS\363(l\366\314^YP\363O^Qc>^V\363\312\215\320\353\33
4\276E\336\232\347Qk^^W^X\363UT\365\346y^Gg>\345\230O9\346S\216\271o\3220\367   \346>\301\2749M\315\243\260\3313g@\314\357_\375^Mc^\271^Q\322w\355<\267\277\273$\327\276
\225Qk\236\367v\377\351tm><\267\347^]\234\371\224c>\345\230O9\346\276I\303\334'\230\373^D\363\34645\217\302f\317\234^A5^?\3670\266\211y\345F\350un\337"o\257\363<j\315\3
53^Bs\235\257\242\252\277\316\363^N\316|\3121\237r\314\247^\s\337\244a\356^S\314}\202ys\232\232Ga\263g\316\200\232?\372^[\306\274r#\364:\267o\221\267\346y\324\232\327^E
\306|^UU\275y\336\301\231O9\346S\216\371\224c\356\2334\314}\202\271O0oN\253\346\327\257\315\366\355= \346\277\337P:\3370\346g\375o^S\363:\265o\344\215y^]5\346\272\340d\
376\331\274}n\257;\234\315\347\374\277\356s\376\257^?\310\377\373'\347O^?n\337~^_\272\274\352\375\337v\346\377\372\323\324<

i tried both codes but no LuckSmilie, Request you to provide an alternate for thisSmilie

Thanks a Ton!!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Replace columns in .csv using other .csv columns

Hi, I have 2 csv files with 15 000 lines, which looks like this: Daily.csv "CODE","BRAND","DESIGNER","SIZE","TYPE","GENDER","SET","DESCRIPTION","IMAGE","COST","WEIGHT","MSRP","UPC" "M-1001","212","Caroline Her","1.7 oz","EDT... (4 Replies)
Discussion started by: olivieraz
4 Replies

2. Shell Programming and Scripting

Match columns from two csv files and update field in one of the csv file

Hi, I have a file of csv data, which looks like this: file1: 1AA,LGV_PONCEY_LES_ATHEE,1,\N,1,00020460E1,0,\N,\N,\N,\N,2,00.22335321,0.00466628 2BB,LES_POUGES_ASF,\N,200,200,00006298G1,0,\N,\N,\N,\N,1,00.30887539,0.00050312... (10 Replies)
Discussion started by: djoseph
10 Replies

3. Shell Programming and Scripting

Compare 2 csv files by columns, then extract certain columns of matcing rows

Hi all, I'm pretty much a newbie to UNIX. I would appreciate any help with UNIX coding on comparing two large csv files (greater than 10 GB in size), and output a file with matching columns. I want to compare file1 and file2 by 'id' and 'chain' columns, then extract exact matching rows'... (5 Replies)
Discussion started by: bkane3
5 Replies

4. Shell Programming and Scripting

Deleting all the fields(columns) from a .csv file if all rows in that columns are blanks

Hi Friends, I have come across some files where some of the columns don not have data. Key, Data1,Data2,Data3,Data4,Data5 A,5,6,,10,, A,3,4,,3,, B,1,,4,5,, B,2,,3,4,, If we see the above data on Data5 column do not have any row got filled. So remove only that column(Here Data5) and... (4 Replies)
Discussion started by: ks_reddy
4 Replies

5. UNIX for Advanced & Expert Users

Unix Bash: substitute columns in .csv using other .csv columns

Hi All, I have two .csv's input.csv having values as (7 columns) ABC,A19907103,ABC DEV YUNG,2.17,1000,2157,07/07/2006 XYZ,H00213850,MM TRUP HILL,38.38,580,23308,31/08/2010 output.csv having (25 columns) A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U,V,W,X,Y... (4 Replies)
Discussion started by: abhivyas
4 Replies

6. Shell Programming and Scripting

Deleting columns by list or file

Dear specialists out there, please help a poor awk newbie: I have a very huge file to process consisting of 300000 columns and 1500 rows. About 20000 columns shall be deleted from that file. So it is clear, that I can't do this by writing down all the columns in an awk command like $1, $x etc.... (5 Replies)
Discussion started by: flxms
5 Replies

7. Shell Programming and Scripting

Deleting columns from CSV file

Hi All, Am working on perl script which should delete columns in existing CSV file. If my file is : AA,BB,CC,DD 00,11,22,33 00,55,66,77 00,99,88,21 AA,BB... are all my headers can come in any order (e.g AA,CC,BB...) and rest are values. I want to delete column CC... Can anybody help... (2 Replies)
Discussion started by: darshakraut
2 Replies

8. Shell Programming and Scripting

no of columns in csv file

Hi, I would like to know how can I find the number of columns in each line in a csv file. I have a csv file wiht comma separated an dhas 10000 line in it. I want to verify the number of columns in each line. is there any way to find it out using awk or any other commands? Thanks.... (2 Replies)
Discussion started by: McLan
2 Replies

9. Shell Programming and Scripting

deleting rows & columns form a csv file

Hi , I want to delete some rows & columns from file. can someone please help me on this? Regards. (2 Replies)
Discussion started by: code19
2 Replies

10. Shell Programming and Scripting

Deleting specific columns from a file

Hi Friends, I want to delete specific columns from a file. Say my file content is as follows: "1","a","ww1",1234" "2","b","wwr3","2222" "3","c","erre","3333" Now i want to delete the column 2 and 4 from this file. That is I want the file content to be: "1","ww1" "2","wwr3"... (11 Replies)
Discussion started by: premar
11 Replies
Login or Register to Ask a Question