Visit Our UNIX and Linux User Community


awk - if field is empty, move line to new file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk - if field is empty, move line to new file
# 1  
Old 10-07-2009
awk - if field is empty, move line to new file

I have a script with this statement:

/usr/xpg4/bin/awk -F"[,|]" 'NR==FNR{s["\"" $1 "\""]=$2;next}{printf "%s\"%s\"\n", $0, s[$3]}' LOOKUP.TXT finallistnew.txt >test.txt

I want to include logic or an additional step that says if there is no data in field 3, move the whole line out of test.txt into an additional file.

Any ideas?
Thanks,
# 2  
Old 10-07-2009
Code:
!$3 { print > newfile }

or

!$3 { print > newfile; next }

As you didn't give any input data sample, this is meaningless if there is a forth field!!

(and as you didn't use code-tags - or carriage returns, I didn't check whether you referenced $4)
# 3  
Old 10-07-2009
Sorry. I don't know what code tags are in Unix scripting but here is a sample data file:

"16","\\server\IP\SCAN\11119431.pdf","100-1115555-222","
00001","Docs","SMITH, JO","DC 1","FL","Administrator
","DK","DK","22222"

"16","\\server\IP\SCAN\88888892.pdf","101-2221140-009","
00001","Docs","LMN. Enterprises Inc.","DC 1","AK","Administ
rator","DK","DK",""

Each line above is one line. So there are 2 lines in the data file, for example. But in the example above, if the last field in each line does not contain a value, I want it to kick out to another file and remove it from the data file above. So in this example, the second line would be moved to a new file.

Hope that makes sense.
# 4  
Old 10-07-2009
Code-tags are a forum thing, they make what you post more readable.

So, I'll do that first, then I can read it...
Code:
/usr/xpg4/bin/awk -F"[,|]" '
  NR==FNR{s["\"" $1 "\""]=$2; next
}

{
  printf "%s\"%s\"\n", $0, s[$3]
}
' LOOKUP.TXT finallistnew.txt >test.txt

"16","\\server\IP\SCAN\11119431.pdf","100-1115555-222","
00001","Docs","SMITH, JO","DC 1","FL","Administrator
","DK","DK","22222"

"16","\\server\IP\SCAN\88888892.pdf","101-2221140-009","
00001","Docs","LMN. Enterprises Inc.","DC 1","AK","Administ
rator","DK","DK",""

# 5  
Old 10-07-2009
Quote:
Originally Posted by scriptr2be
Sorry. I don't know what code tags are in Unix scripting but here is a sample data file:

Code tags have nothing to do with scripting; they are what you put around code in posts to this forum (as I have done to your code below).

Carriage returns are not necessary, but newlines are useful when posting code; they make it much more readable (as do spaces and indentation):

Code:
/usr/xpg4/bin/awk -F"[,|]" '
  NR == FNR { s["\"" $1 "\""] = $2
  next
}
{
  printf "%s\"%s\"\n", $0, s[$3]
}' LOOKUP.TXT finallistnew.txt >test.txt

Quote:
Code:
"16","\\server\IP\SCAN\11119431.pdf","100-1115555-222","00001","Docs","SMITH, JO","DC 1","FL","Administrator","DK","DK","22222"
"16","\\server\IP\SCAN\88888892.pdf","101-2221140-009","00001","Docs","LMN. Enterprises Inc.","DC 1","AK","Administrator","DK","DK",""

Each line above is one line.

Then put it on one line.
Quote:
So there are 2 lines in the data file, for example. But in the example above, if the last field in each line does not contain a value, I want it to kick out to another file and remove it from the data file above. So in this example, the second line would be moved to a new file.

Code:
if ( $NF == "" ) print > newfile
else print

# 6  
Old 10-07-2009
I have a couple of additional questions.

You pass two files into AWK. Which file provides the data you show? Is it from both files? If not, what's in the other file?

Where would you like the output from this to go?:
Code:

{
 printf "%s\"%s\"\n", $0, s[$3]
}

# 7  
Old 10-07-2009
So if this code produces the data file above:
/usr/xpg4/bin/awk -F"[,|]" 'NR==FNR{s["\"" $1 "\""]=$2;next}{printf "%s\"%s\"\n", $0, s[$3]}' LOOKUP.TXT finallistnew.txt >test.txt


How do you insert a if statement into that to make it kick out? I can also include a line after this code to remove the line if there is no value in the last field.

---------- Post updated at 06:56 PM ---------- Previous update was at 06:54 PM ----------

scottn,

Yes it is taken from two data files. Where would I like the data to go? I would like the line that does not contain a value in the last field to go to a file called /tmp/deleteme.

Thanks,

Previous Thread | Next Thread
Test Your Knowledge in Computers #442
Difficulty: Easy
84% of the women surveyed by Cosmopolitan Magazine in 2018 indicated they would prefer to date a computer programmer than a rock star.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Printing string from last field of the nth line of file to start (or end) of each line (awk I think)

My file (the output of an experiment) starts off looking like this, _____________________________________________________________ Subjects incorporated to date: 001 Data file started on machine PKSHS260-05CP ********************************************************************** Subject 1,... (9 Replies)
Discussion started by: samonl
9 Replies

2. UNIX for Dummies Questions & Answers

Using awk to remove duplicate line if field is empty

Hi all, I've got a file that has 12 fields. I've merged 2 files and there will be some duplicates in the following: FILE: 1. ABC, 12345, TEST1, BILLING, GV, 20/10/2012, C, 8, 100, AA, TT, 100 2. ABC, 12345, TEST1, BILLING, GV, 20/10/2012, C, 8, 100, AA, TT, (EMPTY) 3. CDC, 54321, TEST3,... (4 Replies)
Discussion started by: tugar
4 Replies

3. Shell Programming and Scripting

Need a program that read a file line by line and prints out lines 1, 2 & 3 after an empty line...

Hello, I need a program that read a file line by line and prints out lines 1, 2 & 3 after an empty line... An example of entries in the file would be: SRVXPAPI001 ERRO JUN24 07:28:34 1775 REASON= 0000, PROCID= #E506 #1065: TPCIPPR, INDEX= 003F ... (8 Replies)
Discussion started by: Ferocci
8 Replies

4. Shell Programming and Scripting

How to detect empty field in awk ?

Hi ! programmers I have a need of detecting empty field in file my file looks like this 40.900|-71.600|1.6|20|1|1961|21.00|3.700||1|US|28035|10029370|31 40.900|-71.600|5.7|20|1|1961|21.00|3.700||1|US|28035|10029370|31 40.900|-71.600|7.8|20|1|1961|21.00|3.700||1|US|28035|10029370|31... (7 Replies)
Discussion started by: Dona Clara
7 Replies

5. Shell Programming and Scripting

awk - remove row if specific field is empty/blank

I have this text.filecharles darwin sam delight george washington johnson culper darwin sam delight micheal jackson penny lite and would like to remove the row, if the first field is blank. so the result would be: result.filecharles darwin sam ... (4 Replies)
Discussion started by: charles33
4 Replies

6. UNIX for Dummies Questions & Answers

An easier way to move character from field to a new line

Hi ! Is there a more convenient way to do the following: When, in $2, there is more then one letter (separated by ","), it returns the letter on the second position to a new line in $2 with the same content in $1 and $3 as the line this letter comes from. infile: aaaaa A,B 1,2,3,4,5... (6 Replies)
Discussion started by: lucasvs
6 Replies

7. Shell Programming and Scripting

use awk to replace empty fields with the latest nonempty field

Hi suppose I have a csv file like this count,1977,1978,1979 usa, , , blue japan, red, yellow,green india, , yellow,blue china, blue, yellow, green I want the output to be(replace everything, including empty data, with the most recent data): ... (1 Reply)
Discussion started by: grossgermany
1 Replies

8. UNIX for Dummies Questions & Answers

Help required on printing the line if third field is not a empty space

Hi Experts, I want to print the lines whose third field in non-empty/blank space. i.e. INPUT FILE/B] dcdccghjrry0yd cont dcdccttrk820529 cont rdekedfsSCr dcdccttrdky2d0y cont rdekedfsSC2 ... (3 Replies)
Discussion started by: krao
3 Replies

9. Shell Programming and Scripting

How to print empty line when the a field is changed

Hi all, I have this input file .. BSS01 107 Swafieh 11/06/2008 12:06:57 BSS01 111 Ramada_Hotel 12/06/2008 11:37:20 BSS01 147 Kalha_Rep 11/06/2008 19:13:39 BSS01 147 Kalha_Rep ... (9 Replies)
Discussion started by: yahyaaa
9 Replies

10. Shell Programming and Scripting

Awk scrip to remove empty field

Hi I have a file which looks like this name: Sally group: Group4 name: Tim group: Group1 name: Dan group: Group2 name: Chris group: Group3 name: Peter group: name: Fred group: name: Mary group: Group2 Well I want to get rid of the... (4 Replies)
Discussion started by: bombcan
4 Replies

Featured Tech Videos