remove empty field


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers remove empty field
# 1  
Old 04-05-2012
remove empty field

Hi all !

I'm sure it is a basic question but I didn't find any threads that fit my need.

How to remove empty fields with awk?
Or in other words, how to shift all the fields after an empty field on the left?

input:
Code:
1|2||3|4|5||6

wanted:
Code:
1|2|3|4|5|6

I tried:
Code:
awk '{for(i=1; i<=NF; i++) if($i ~//) $i=$(i+1)}' input

and obtained,
Code:
2||3|4|5||6

I think the problem is I need to know the position of the empty field first, and then apply the condition to the next fields !

Is it correct?
# 2  
Old 04-05-2012
try this

Code:
awk -F"|" '{for(i=1;i<=NF;i++) {if($i != "") print $i"|";}}' input


Last edited by zaxxon; 04-05-2012 at 09:58 AM.. Reason: code tags please
# 3  
Old 04-05-2012
Code:
sed 's:||:|:g' input

# 4  
Old 04-05-2012
You would need to do something like this to shift the fields:
Code:
awk '{for(i=1; i<=NF; i++) if($i=="") for(j=i;j<=NF;j++)$j=$(j+1)}1' FS=\| OFS=\| infile

output:
Code:
1|2|3|4|5|6||

But that still leaves two empty fields at the back..
So you would need to do something like this to remove those empty fields at the back.
Code:
awk '{for(i=1; i<=NF; i++) if($i=="") for(j=i;j<=NF;j++)$j=$(j+1);NF-=2}1' FS=\| OFS=\| infile

output:
Code:
1|2|3|4|5|6

If you don't know how many field need to be shifted, you would need to calculate the number of empty fields...

But a more straightforward approach would be to just delete field separators:
Code:
awk '{gsub(/^\||\|$/,x);gsub(/\|\|/,"|")}1' infile

output:
Code:
1|2|3|4|5|6

Or if you do not need to use awk:
Code:
sed 's/^|//;s/|$//;s/||/|/g' infile


Last edited by Scrutinizer; 04-05-2012 at 10:21 AM..
This User Gave Thanks to Scrutinizer For This Post:
# 5  
Old 04-05-2012
Thanks guys !


@balajesuri
Quote:
sed 's:||:|:g' input
Works, but need awk.

@Scrutinizer
Thanks for the complete reply. It works perfectly with both awk and sed !

Just a question: what the "x" in the first gsub stand for?
Is it a variable?
# 6  
Old 04-05-2012
Yes x is a variable, and since it has not been initialized, it contains the value ""
# 7  
Old 04-06-2012
Ok, I get it !

Thanks !

---------- Post updated at 10:42 PM ---------- Previous update was at 08:17 PM ----------

Another question which is linked to this thread (I will also post that in a different thread maybe) !

How do you insert the previous command to remove empty fields,
Code:
awk '{gsub(/^\||\|$/,x);gsub(/\|\|/,"|")}1' infile

into a script where I need to specify FS and OFS?

let's say my script look like:
Code:
#!/usr/bin/gawk -f

BEGIN{FS=OFS="|"}

{
   <do my stuff 1>
}

# need to remove empty fields at this stage

{
    awk '{gsub(/^\||\|$/,x);gsub(/\|\|/,"|")}1' infile
}

# need to keep going with FS=OFS="|"

{
   <do my stuff 2>
}
1

I mean, if I don't "cancel" the "BEGIN{FS=OFS="|"} at the beginning, the command to remove field separators will not work !

How can I "suspend" the BEGIN{FS=OFS="|"}, jus the time to execute the command for removing FS ?
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Using awk to remove duplicate line if field is empty

Hi all, I've got a file that has 12 fields. I've merged 2 files and there will be some duplicates in the following: FILE: 1. ABC, 12345, TEST1, BILLING, GV, 20/10/2012, C, 8, 100, AA, TT, 100 2. ABC, 12345, TEST1, BILLING, GV, 20/10/2012, C, 8, 100, AA, TT, (EMPTY) 3. CDC, 54321, TEST3,... (4 Replies)
Discussion started by: tugar
4 Replies

2. Shell Programming and Scripting

How to remove empty field in a text file?

Hi all, I want to remove empty field in a text file. I tried to used sed. But it failed. Input: LG10_PM_map_19_LEnd 1000560 G AG AG LG10_PM_map_19_LEnd 1005621 G AG LG10_PM_map_19_LEnd 1011214 A AG AG LG10_PM_map_19_LEnd 1011673 T CT CT ... (3 Replies)
Discussion started by: huiyee1
3 Replies

3. Shell Programming and Scripting

How to detect empty field in awk ?

Hi ! programmers I have a need of detecting empty field in file my file looks like this 40.900|-71.600|1.6|20|1|1961|21.00|3.700||1|US|28035|10029370|31 40.900|-71.600|5.7|20|1|1961|21.00|3.700||1|US|28035|10029370|31 40.900|-71.600|7.8|20|1|1961|21.00|3.700||1|US|28035|10029370|31... (7 Replies)
Discussion started by: Dona Clara
7 Replies

4. Shell Programming and Scripting

print non empty column/field value

I have below file 25-09-2012 24-09-2012 19-09-2012 31-07-2012 30-04-2012 30-03-2012 ASIAEXFVNV N/A CEU 4 DMIRSOA N/A CAS 2 2 2 DMIRSOA N/A MIDMT 2 NFIAL22 N/A HVNY 11 11 11 NFIAL22 N/A NYAL3 11 11 11 NFIAL22 N/A NYCN 11 11 11 ... (4 Replies)
Discussion started by: manas_ranjan
4 Replies

5. Shell Programming and Scripting

awk - remove row if specific field is empty/blank

I have this text.filecharles darwin sam delight george washington johnson culper darwin sam delight micheal jackson penny lite and would like to remove the row, if the first field is blank. so the result would be: result.filecharles darwin sam ... (4 Replies)
Discussion started by: charles33
4 Replies

6. Shell Programming and Scripting

Using sed to remove lines where field is empty

I was just looking at this post: https://www.unix.com/shell-programming-scripting/22893-delete-multiple-empty-lines.html. and I am looking to achieve the same with sed. So the idea is to delete lines from a file where a certain field has no value. Inputfile: EMID MMDDYY HOURS JOB EMNAME 0241... (4 Replies)
Discussion started by: figaro
4 Replies

7. UNIX for Dummies Questions & Answers

Count of Field for Non-Empty

Hi Guys, I wanted to count the number of records for a particular field of a file. whose fields are separated by comma"," I fI use this command. cat "filename" cut -sd "," -f13 | wc -l This shows all the lines count including the blank values for the field number 13. I wanted to count... (2 Replies)
Discussion started by: Swapna173
2 Replies

8. Shell Programming and Scripting

How to print empty line when the a field is changed

Hi all, I have this input file .. BSS01 107 Swafieh 11/06/2008 12:06:57 BSS01 111 Ramada_Hotel 12/06/2008 11:37:20 BSS01 147 Kalha_Rep 11/06/2008 19:13:39 BSS01 147 Kalha_Rep ... (9 Replies)
Discussion started by: yahyaaa
9 Replies

9. Shell Programming and Scripting

Awk scrip to remove empty field

Hi I have a file which looks like this name: Sally group: Group4 name: Tim group: Group1 name: Dan group: Group2 name: Chris group: Group3 name: Peter group: name: Fred group: name: Mary group: Group2 Well I want to get rid of the... (4 Replies)
Discussion started by: bombcan
4 Replies

10. UNIX for Dummies Questions & Answers

Shell Script using Join, empty field help!

Deleted#### (1 Reply)
Discussion started by: tibbyuk
1 Replies
Login or Register to Ask a Question