Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Check for not null column in a pipe delimited file

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Check for not null column in a pipe delimited file
# 1  
Old 06-05-2016
Check for not null column in a pipe delimited file
Hi,

I have a requirement where I have to check whether the mandatory columns in a pipe delimited file is null and print error message.
For eg, I have to check if the 3rd,5th,6th,7th and 8th column are null and print the message "<column name> is null".
The data file will have aroung 100,000 records.
Please help!
# 2  
Old 06-05-2016
Hi,

Something like this:

Code:
awk -F'|' '{ 
   if( $3=="" || $5=="" || $7=="" || $8=="" ) 
         if ( $3=="" ) { print "Line ",NR, " Field 3: Mandatory field is null"; }
         if ( $5=="" ) { print "Line ",NR, " Field 5: Mandatory field is null"; }
         # ...
   else  
         do_something ; 
}' yourfile.txt

Last edited by stomp; 06-05-2016 at 07:45 AM..
This User Gave Thanks to stomp For This Post:
reshma15193
# 3  
Old 06-05-2016
Like so?
Code:
awk -F \| '{for(i=3; i<=8; i++) if(i!=4) if($i=="") printf "%s\n","column " i " is null at line " NR}' file


---
@stomp:
You cannot use !$i or for example!$3 as a test, since then the condition will also become true if the field equals 0 rather than "" (null value).
Last edited by Scrutinizer; 06-05-2016 at 07:41 AM..
These 2 Users Gave Thanks to Scrutinizer For This Post:
reshma15193 stomp
# 4  
Old 06-05-2016
Quote:
Originally Posted by Scrutinizer
You cannot use !$i or for example !$3 as a test, since then the condition will also become true if the field equals 0 rather than "" (null value).
Thanks. Changed That.
This User Gave Thanks to stomp For This Post:
reshma15193
# 5  
Old 06-05-2016
How about making it a bit more flexible? Try
Code:
awk '
NR==1   {MX = split(CHK, T, ",")
        }
        {for (i=1; i<=MX; i++)
           if ($(T[i]) == "") print "column " T[i] " is null at line " NR
        }
' FS="|" CHK="3,5,6,7,8" file
column 3 is null at line 1
column 6 is null at line 3
column 7 is null at line 6

Last edited by RudiC; 06-05-2016 at 04:48 PM.. Reason: Overlooked the "pipe" delimiter request.
This User Gave Thanks to RudiC For This Post:
reshma15193
# 6  
Old 06-05-2016
Quote:
Originally Posted by stomp
Hi,

Something like this:

Code:
awk -F'|' '{ 
   if( $3=="" || $5=="" || $7=="" || $8=="" ) 
         if ( $3=="" ) { print "Line ",NR, " Field 3: Mandatory field is null"; }
         if ( $5=="" ) { print "Line ",NR, " Field 5: Mandatory field is null"; }
         # ...
   else  
         do_something ; 
}' yourfile.txt

I think you're missing a pair a braces:
Code:
awk -F'|' '{ 
   if( $3=="" || $5=="" || $7=="" || $8=="" ) {
         if ( $3=="" ) { print "Line ",NR, " Field 3: Mandatory field is null"; }
         if ( $5=="" ) { print "Line ",NR, " Field 5: Mandatory field is null"; }
         # ...
   } else  
         do_something ; 
}' yourfile.txt

Without them, your code will execute the else clause only if the last last field tested is an empty string and, if the 1st if test does not find any empty strings, the remaining if tests will never succeed either:
Code:
awk -F'|' '{ 
   if( $3=="" || $5=="" || $7=="" || $8=="" ) { 
         if ( $3=="" ) { print "Line ",NR, " Field 3: Mandatory field is null"; }
   }
   if ( $5=="" ) { { print "Line ",NR, " Field 5: Mandatory field is null"; }
   # ...
   } else  
         do_something ; 
}' yourfile.txt

And, of course, if do_something; is more than one statement, you'll also needed braces around all statements in the else clause. The braces around your print statements don't hurt anything, but aren't strictly required since only one statement is being executed in those if then clauses.
This User Gave Thanks to Don Cragun For This Post:
reshma15193
# 7  
Old 06-06-2016
Thanks Don. Regarding awk, I'm only a newbie so far. Reading and then Try+Error. Smilie
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replace delimiter for a particular column in a pipe delimited file

I have an input file as below Emp1|FirstName|MiddleName|LastName|Address|Pincode|PhoneNumber 1234|FirstName1|MiddleName2|LastName3| Add1 || ADD2|123|000000000 Output : 1234|FirstName1|MiddleName2|LastName3| Add1 ,, ADD2|123|000000000 OR 1234,FirstName1,MiddleName2,LastName3, Add1 ||... (2 Replies)
Discussion started by: styris
2 Replies

2. Shell Programming and Scripting

Replacing a column in a pipe delimited file

Hi, I have a pipe delimited file as below and I need to replace the 2nd column of each line with null values. 1|10/15/2011|fname1|lname1 2|10/15/2012|fname2|lname2 3|10/15/2013|fname3|lname3 Output file: 1||fname1|lname1 2||fname2|lname2 3||fname3|lname3 I tried this ... (2 Replies)
Discussion started by: member2014
2 Replies

3. Shell Programming and Scripting

Replace pipe delimited column string to null

Hi All, I have a large dat file where each lines are pipe delimited values. I need to parse the file depending on the request. For example: sometimes I have told to remove all the values in the 7th column (this case remove values '3333' only from the first line and '3543' from the second line)... (4 Replies)
Discussion started by: express14
4 Replies

4. Shell Programming and Scripting

Find for line with not null values at nth place in pipe delimited file

Hi, I am trying to find the lines in a pipe delimited file where 11th column has not null values. Any help is appreciated. Need help asap please. thanks in advance. (3 Replies)
Discussion started by: manikms
3 Replies

5. Homework & Coursework Questions

how to show particular column from pipe delimited file

hi, I have pipe delimited flat file as below 1|ab|4.5|9| 2|ac|3|12| 3|ac|4.5|8| i want to show (display) only 3rd field between pipes. please help (1 Reply)
Discussion started by: vai15517
1 Replies

6. UNIX for Advanced & Expert Users

Check Whether File is Pipe Delimited

Can anybody help me how to check whether a file is Pipe delimited or not? (1 Reply)
Discussion started by: Allwin333
1 Replies

7. Shell Programming and Scripting

count of null in pipe delimited txt file

Hi, I have a pipe delimited txt file which contains 17 fields per line/row. 16th field contains email id. I want to count the number of lines/rows that contains null in the 16th field. Plz find attached example data file. I'm looking for a command line/script which achieves this. ... (5 Replies)
Discussion started by: Sriranga
5 Replies

8. UNIX for Dummies Questions & Answers

Need help with shell script for chekking a column in txt file - pipe delimited

Hi: I have a text file date(pipe delimited) which is loaded in to the DB using sql loader(&CTL files) after some initial validation by the shell script. Now i have a situation where the shell script needs to check a column in the text file and if it is NULL then it needs send this record/row... (12 Replies)
Discussion started by: ravi0435
12 Replies

9. Shell Programming and Scripting

How to check Null values in a file column by column if columns are Not NULLs

Hi All, I have a table with 10 columns. Some columns(2nd,4th,5th,7th,8th and 10th) are Not Null columns. I'll get a tab-delimited file and want to check col by col and generate seperate error code for each col eg:102 if 2nd col value is NULL and 104 if 4th col value is NULL so on... I am a... (7 Replies)
Discussion started by: Mandab
7 Replies

10. Shell Programming and Scripting

Grep for NULL in a pipe delimited file

hi, How can I check for a field in a pipe-delimited file having a NULL value in Unix using a grep command or any other command. Please reply (5 Replies)
Discussion started by: sureshg_sampat
5 Replies
Login or Register to Ask a Question