csv manipulation


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting csv manipulation
# 1  
Old 09-20-2010
csv manipulation

Hi,

I want to delete the null values that are present in all the columns except the last column
Code:
$ cat x.csv
baseball,NULL,8798765,Most played,0,5,12367,NULL,NULL,98,67,Reason is not sufficient
baseball,NULL,8928192,Most played,0,4,76893,NULL,RAW,54,78,NULL
baseball,NULL,5678945,Most played,9,2,1,6,NULL,6789,123,Reason not conclueded
cricket,NOTNULL,125782,Usually played,NULL,NULL,NULL,NULL,123,67,NULL
cricket,NOTNULL,678921,Usually played,NULL,NULL,NULL,NULL,563,NULL,Reason not defines
$ sed 's/NULL/ /g' x.csv
baseball, ,8798765,Most played,0,5,12367, , ,98,67,Reason is not sufficient
baseball, ,8928192,Most played,0,4,76893, ,RAW,54,78,
baseball, ,5678945,Most played,9,2,1,6, ,6789,123,Reason not conclueded
cricket,NOT ,125782,Usually played, , , , ,123,67,
cricket,NOT ,678921,Usually played, , , , ,563, ,Reason not defines
$

Can somebody help me out on this
# 2  
Old 09-20-2010
Something like this,
Code:
sed 's/NULL,/,/g' inputfile

# 3  
Old 09-20-2010
Code:
$ ruby -F"," -ane 'print "#{ $F[0..-2].each{|x| x.gsub!(/NULL/i,"")}.join(",")},#{$F[-1]}" ' file
baseball,,8798765,Most played,0,5,12367,,,98,67,Reason is not sufficient
baseball,,8928192,Most played,0,4,76893,,RAW,54,78,NULL
baseball,,5678945,Most played,9,2,1,6,,6789,123,Reason not conclueded
cricket,NOT,125782,Usually played,,,,,123,67,NULL
cricket,NOT,678921,Usually played,,,,,563,,Reason not defines

# 4  
Old 09-20-2010
Code:
Thanks Pravin.
But can you check the second row, last column.
The code will delete that also
$ cat x.csv
baseball,NULL,8798765,Most played,0,5,12367,NULL,NULL,98,67,Reason is not sufficient
baseball,NULL,8928192,Most played,0,4,76893,NULL,RAW,54,78,value is NULL,value expected
baseball,NULL,5678945,Most played,9,2,1,6,NULL,6789,123,Reason not conclueded
cricket,NOTNULL,125782,Usually played,NULL,NULL,NULL,NULL,123,67,NULL
cricket,NOTNULL,678921,Usually played,NULL,NULL,NULL,NULL,563,NULL,Reason not defines
$ sed 's/NULL,/,/g' x.csv
baseball,,8798765,Most played,0,5,12367,,,98,67,Reason is not sufficient
baseball,,8928192,Most played,0,4,76893,,RAW,54,78,value is ,value expected
baseball,,5678945,Most played,9,2,1,6,,6789,123,Reason not conclueded
cricket,NOT,125782,Usually played,,,,,123,67,NULL
cricket,NOT,678921,Usually played,,,,,563,,Reason not defines

# 5  
Old 09-20-2010
Try this,

Code:
sed 's/,NULL,/,,/g' x.csv

# 6  
Old 09-20-2010
Code:
$ ruby -F"," -ane 'print "#{ $F[0..-2].each{|x| x.gsub!(/NULL/i,"")}.join(",")},#{$F[-1]}" ' file
baseball,,8798765,Most played,0,5,12367,,,98,67,Reason is not sufficient
baseball,,8928192,Most played,0,4,76893,,RAW,54,78,value is ,value expected
baseball,,5678945,Most played,9,2,1,6,,6789,123,Reason not conclueded
cricket,NOT,125782,Usually played,,,,,123,67,NULL
cricket,NOT,678921,Usually played,,,,,563,,Reason not defines



---------- Post updated at 01:49 AM ---------- Previous update was at 01:43 AM ----------

Quote:
Originally Posted by pravin27
Code:
sed 's/,NULL,/,,/g' x.csv

Code:
# sed 's/,NULL,/,/g' file
baseball,8798765,Most played,0,5,12367,NULL,98,67,Reason is not sufficient
baseball,8928192,Most played,0,4,76893,RAW,54,78,value is NULL,value expected
baseball,5678945,Most played,9,2,1,6,6789,123,Reason not conclueded
cricket,NOTNULL,125782,Usually played,NULL,NULL,123,67,NULL
cricket,NOTNULL,678921,Usually played,NULL,NULL,563,Reason not defines

NULL still present.
# 7  
Old 09-20-2010
Sorry for previous post.

how about perl.
Code:
perl -ple 's/(?<=,)NULL(?=,)//g' inputfile

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Match columns from two csv files and update field in one of the csv file

Hi, I have a file of csv data, which looks like this: file1: 1AA,LGV_PONCEY_LES_ATHEE,1,\N,1,00020460E1,0,\N,\N,\N,\N,2,00.22335321,0.00466628 2BB,LES_POUGES_ASF,\N,200,200,00006298G1,0,\N,\N,\N,\N,1,00.30887539,0.00050312... (10 Replies)
Discussion started by: djoseph
10 Replies

2. Shell Programming and Scripting

Pax1.csv file manipulation

Hallo Team and happy Friday, I have a .csv file attached called MHG_Only_Billing_only_great_201408_SummaryDetails.csv The file is as below: $ cat MHG_Only_Billing_only_great_201408_SummaryDetails.csv... (1 Reply)
Discussion started by: kekanap
1 Replies

3. Shell Programming and Scripting

.csv file manipulation

Hall0 Team, I need you help. I have a .csv file called Pax.csv. It has 13 fields The fields that i am interested in is field 7, field 10(CallDuration),field 12 and field 13(TelkomAsif) Field 12(Cost) is affected by what is on field 7(CallType) if field7= VoIS Community Sharecall then Cost... (14 Replies)
Discussion started by: kekanap
14 Replies

4. Shell Programming and Scripting

Data Manipulation on a .csv file

Hallo Friends, I need you help. My file has 5000 or so lines and currently looks like below(sample). Service Type,Origin,Destination,Rate Per Minute,Minimum Charge,Time Based Rate,Time Based From Day,Time Based To Day,Time Based From Time,Time Based To Time,Destination Prefix List,, VoIS... (3 Replies)
Discussion started by: kekanap
3 Replies

5. Shell Programming and Scripting

CSV data format manipulation

Hi There I need a script which will pick up the data from a .CSV file and reformat it as per the requirement and write it to another .CSV file. I am using an application that will only take data in a particular format and need something that will convert without manual intervention. The... (4 Replies)
Discussion started by: rbggbr16
4 Replies

6. Shell Programming and Scripting

Awk to convert a text file to CSV file with some string manipulation

Hi , I have a simple text file with contents as below: 12345678900 971,76 4234560890 22345678900 5971,72 5234560990 32345678900 71,12 6234560190 the new csv-file should be like: Column1;Column2;Column3;Column4;Column5 123456;78900;971,76;423456;0890... (9 Replies)
Discussion started by: FreddyDaKing
9 Replies

7. Shell Programming and Scripting

Perl search csv fileA where two strings exist on another csv fileB

Hi I have two csv files, with the following formats: FileA.log: Application, This occured blah Application, That occured blah Application, Also this AnotherLog, Bob did this AnotherLog, Dave did that FileB.log: Uk, London, Application, datetime, LaterDateTime, Today it had'nt... (8 Replies)
Discussion started by: PerlNewbRP
8 Replies

8. Shell Programming and Scripting

CSV to SQL insert: Awk for strings with multiple lines in csv

Hi Fellows, I have been struggling to fix an issue in csv records to compose sql statements and have been really losing sleep over it. Here is the problem: I have csv files in the following pipe-delimited format: Column1|Column2|Column3|Column4|NEWLINE Address Type|some descriptive... (4 Replies)
Discussion started by: khayal
4 Replies

9. Shell Programming and Scripting

csv file manipulation

Hi, I generated a csv file.But i have to manipulate it to make it user friendly. This is the structure of the input csv file. partyfailurereasonnumbercvfrejected12cvfapproved45cvfonhold23mvcunknown98mvconcall17pucapproved78puchold34actunknown19 The first line is the header. The... (6 Replies)
Discussion started by: amit.behera
6 Replies

10. Shell Programming and Scripting

Need to compare two csv files values and write into another csv file

Hi all, Am new to scripting. So i just need your ideas to help me out. Here goes my requirement. I have two csv files 1.csv 2.csv abc,1.24 abc,1 def,2.13 def,1 I need to compare the first column of 1.csv with 2.csv and if matches then need to compare... (2 Replies)
Discussion started by: chinnahyd
2 Replies
Login or Register to Ask a Question