Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Trim empty fields in a given range

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Trim empty fields in a given range
# 1  
Old 09-03-2011
Trim empty fields in a given range
Is there some easy way to trim empty fields but only in a given range?
for example say I have csv data that looks like this:


Code:
apple,,,Granysmith,,2.50,,TimmysGrocers
Pear,Bartlett,,,,,Park,
peach,,,,Peento,3.00,Garden,TimmysGrocers

is there a way of getting the single field with data between 2-4 and ignoring the rest so that the output looks like


Code:
apple,Granysmith,2.50,,TimmysGrocers
Pear,Bartlett,,Park,
peach,Peento,3.00,Garden,TimmysGrocers


hope that makes sense.
Last edited by cue; 09-03-2011 at 03:30 PM.. Reason: Code tags!
# 2  
Old 09-03-2011
In the third line you just remove columns 2-4 with spaces in it, not extracting any data. And you have different "Granysmithes" in your input and output files. Anyway you can try:
Code:
sed -r 's/, +,/,/; s/, +,/,/' INPUTFILE

or (for 2-5 fields):
Code:
sed -r 's/, +,/,/; s/, +,/,/; s/, +,/,/' INPUTFILE

Last edited by yazu; 09-03-2011 at 03:21 PM..
This User Gave Thanks to yazu For This Post:
cue
# 3  
Old 09-03-2011
Quote:
Originally Posted by yazu
In the third line you just remove columns 2-4 with spaces in it, not extracting any data.
Hi yazu. thanks I'll give that a try. The capital letter was a mistake, sorry.
Is there a way of doing it for an arbitrary number of fields, I actually need it in the range 2 to 102 but because I do not understand your sed command I cannot adapt it. Could you please elaborate on it.

---------- Post updated at 02:14 PM ---------- Previous update was at 01:17 PM ----------

I tried to wrap my head around it for a while now but I don't understand it at all any further help greatly appreciated.
Last edited by cue; 09-03-2011 at 03:40 PM..
# 4  
Old 09-03-2011
Code:
sed -r 's/, +,/,/'

removes the first empty field and so on. I think it doesn't scale for your real needs. Try to reformulate your problem.
This User Gave Thanks to yazu For This Post:
cue
# 5  
Old 09-03-2011
Quote:
Originally Posted by yazu
Code:
sed -r 's/, +,/,/'

removes the first empty field and so on. I think it doesn't scale for your real needs. Try to reformulate your problem.
Well, I took your advice and tried something different (and messy), an awk script. so, for example to remove blanks between 2 and 105 i would type
Code:
./script.awk Inputfile.csv 2 105

It works but at the end of the output I want it also spits out

Code:
awk: cannot open 2 (No such file or directory)

why is it trying to open it as a file? I just want it as an argument/parameter to the script, is there a way to avoid this? The script is below

Code:
#!/usr/bin/awk -f
BEGIN { FS = ","}
FNR>1 {for (i = 1; i < ARGV[2]; i++) printf $i","; 
 for (i = ARGV[2]; i <= ARGV[3] ; i++) if ($i!="") printf $i",";
 for (i = ARGV[3]+1; i <= NF; i++) printf $i","; print ""}

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk empty fields

Hello I have a file like this a,b,c,1,2,3,d,e,f,,,,g,h,i,,,,j,k,l and using awk 'FS="'"{print $9,$10,$11}' does not work as I was hoping. I would like the empty fieds, i.e. between the two comma to be interpreted as a zero. is this possible? I would like to get f 0 0 out of the above... (1 Reply)
Discussion started by: garethsays
1 Replies

2. Shell Programming and Scripting

awk to identify empty fields in line

I am trying to use awk to identify and print out records in fields that are empty along with which line they are in. I hope the awk below is close, it runs but nothing results. Thank you :). awk awk -F'\t' 'FNR==NR ~ /^*$/ { print "NR is empty" }' file file 123 GOOD ID 45... (3 Replies)
Discussion started by: cmccabe
3 Replies

3. Shell Programming and Scripting

awk to remove range of fields

I am trying to cut a range of fields in awk. The below seems to work for removing field 50, but what is the correct syntax for removing a range ($50-$62). Thank you :). awk awk 'BEGIN{FS=OFS="\t"}{$50=""; gsub(/\t\t/,"\t")}1' test.vcf.hg19_multianno.txt > output.csv Maybe: awk... (6 Replies)
Discussion started by: cmccabe
6 Replies

4. Shell Programming and Scripting

How to parse fixed-width columns which may include empty fields?

I am trying to selectively display several columns from a db2 query, which gives me a fixed-width output (partial output listed here): --------- -------------------------- ------------ ------ 000 0000000000198012 702 29 000 0000000000198013 ... (9 Replies)
Discussion started by: ahsh79
9 Replies

5. Shell Programming and Scripting

checking csv files with empty fields..!

Hi! I need to learn that how a shell script can transverse a csv file n check if any field is empty or not. means its contains two comma or space b/w commas i.e., "" or " ". can anyone help me out how I can do that.... (10 Replies)
Discussion started by: sukhdip
10 Replies

6. Shell Programming and Scripting

awk to print range of fields

Hi file.in and file.out are in csv format. the code I have now is, cat file.in | awk -F"," '!($1$2$3$4$5$6$7$8 in a){a;print $0}' > file.out Here, I am printing entire line using $0. however, I want to print $1 to $150 and it should be in csv format. Cut -d is not good in performace.... (3 Replies)
Discussion started by: krishnix
3 Replies

7. Shell Programming and Scripting

awk ignores fields with only spaces or empty

Hi, Does any one know how to avoid the scenario where awk ignores the fields having only spaces or empty fields? for instance, Data: "a","b","c","d",""," " code: awk -F, '{ print NF }' File the output I get is 4 instead of 6 do you know how to avoid this? (6 Replies)
Discussion started by: ahmedwaseem2000
6 Replies

8. Shell Programming and Scripting

Counting non empty fields and calculating with that number

Hello all, I have a problem with a skript of mine: My input has the following format 1,33296 transcript_id"ENSRNOT00000018629" 0 1,33296 0 0 transcript_id"ENSRNOT00000029014" 0 0,907392 transcript_id"ENSRNOT00000016905" 0,907392 0 transcript_id"ENSRNOT00000053370" 0 0... (0 Replies)
Discussion started by: DerSeb
0 Replies

9. Shell Programming and Scripting

use awk to replace empty fields with the latest nonempty field

Hi suppose I have a csv file like this count,1977,1978,1979 usa, , , blue japan, red, yellow,green india, , yellow,blue china, blue, yellow, green I want the output to be(replace everything, including empty data, with the most recent data): ... (1 Reply)
Discussion started by: grossgermany
1 Replies

10. Shell Programming and Scripting

cutting fields and doing trim in awk

hi, I have this code: #!/usr/bin/ksh #SERVICES gzcat *SERVICES* | awk ' { SUBSCRIBERNUMBER=substr($0,1,20) SUBSCRIBERNUMBER=trim(SUBSCRIBERNUMBER) SERVICECODE=substr($0,22,61) SERVICECODE=trim(SERVICECODE) STARTDD=substr($0,63,72) STARTDD=trim(STARTDD) STARTDT=substr($0,74,81)... (1 Reply)
Discussion started by: naoseionome
1 Replies
Login or Register to Ask a Question