Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

To find record having null value

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting To find record having null value
# 8  
Old 12-22-2016
Hello grinkrf,

Could you please try following too once and let me know if this helps you.
You could dynamically put as per file like whichever field you want to check.
Code:
awk -F"," 'function verify(var){num=split(var, A);for(i in A){if($A[i]){p++}};if(num==p){print};p=""} FILENAME=="Input_file1"{verify("2");} FILENAME=="Input_file2"{verify("2,4")}'  Input_file1   Input_file2

Output will be as follows.
Code:
123,asd,fgh,nmb,pok
1235,afg,fgdf,nmbs,posk

EDIT: Adding a non-one liner form of solution now too.
Code:
awk -F"," 'function verify(var){
                                        num=split(var, A);
                                        for(i in A){
                                                        if($A[i]){
                                                                        p++
                                                                 }
                                                   };
                                        if(num==p) {
                                                        print
                                                   };
                                        p=""
                               }
           FILENAME=="Input_file1"   {
                                        verify("2");
                               }
           FILENAME=="Input_file2"   {
                                        verify("2,4")
                               }
          '  Input_file1  Input_file2

Thanks,
R. Singh
Last edited by RavinderSingh13; 12-22-2016 at 06:27 AM.. Reason: Adding a non-on liner form of solution too successfully now.
This User Gave Thanks to RavinderSingh13 For This Post:
ginrkf
# 9  
Old 12-22-2016
Quote:
Originally Posted by Aia
Please, could you provide a realistic input sample of your file?
Please, could you provide a sample of the kind of output you want based on that input file?
Quote:
Originally Posted by ginrkf
Hi Please find my sample file

File1
Code:
123,asd,fgh,nmb,pok
134,,fg,mnm,mnb

For the above file I need to reject the second record as the second column contain null value.

File2

Code:
1235,afg,fgdf,nmbs,posk
1345,,fg,,mnsb

For file 2 by requirement is different where I need to reject a record where 2nd and 4th column is null

So I want to write a unique command where I can pass the column position to check where it is null or not.For ex. I want to pass column position as 2 for the first file and column position as 2&4 for the second file
Unfortunately, you did not provide a sample of the desired output and your words leave a lot to interpretation. The input provided is quite ambiguous, as well, due to the skimpiness of the sample.
In post #1 you said:
Quote:
All my files are pipe delimited
Which make the examples not realistic neither.

Quote:
I need to reject a record where 2nd and 4th column is null
A record normally is a string of characters terminated at the new line (a full human line). Is that the same way you are interpreting it? Do you want the resulted output to exclude any lines that match your criteria?
In your comment "2nd and 4th column is null":
Is your criteria for File2 to exclude the lines that have BOTH fields 2 AND 4 empty or to exclude lines that have EITHER fields 2 OR 4 empty?
Last edited by Aia; 12-22-2016 at 07:20 PM..
# 10  
Old 12-22-2016
Thanks to all for your inputs.The below code is working
Code:
awk -F"|" 'function verify(var){
                                        num=split(var, A);
                                        for(i in A){
                                                        if($A[i]){
                                                                        p++
                                                                 }
                                                   };
                                        if(num==p) {
                                                        print
                                                   };
                                        p=""
                               }
           FILENAME=="sample_file"   {
                                        verify("2");
                               }
                                          
          '  sample_file

The result which the above code is giving is, printing all the records where the column 2 does not contain any empty value.Thanks RavinderSingh .

I need some more modification with the above code in such a way that correct records should print to one file and incorrect records to other file.Kindly help in modifying the above code to achieve this functionality also.


Also I want to parameterise the column position and file name, so that I can create a script and pass the file name and column position as values dynamically.
Last edited by ginrkf; 12-22-2016 at 11:57 PM..
# 11  
Old 12-23-2016
Hello ginrkf,

Could you please try following and let me know if this helps you, I haven't tested it though.
Code:
awk -F"," 'function verify(var){
                                        num=split(var, A);
					print FILENAME " is getting processed now..." >> "correct_records"
					print FILENAME " is getting processed now..." >> "Incorrect_records"
                                        for(i in A){
                                                        if($A[i]){
                                                                        p++
                                                                 }
                                                   };
                                        if(num==p) {
                                                        print >> "correct_records"
                                                   };
					else       {
							print >> "Incorrect_records"
						   };
                                        p=""
					print "*******************" >> "correct_records"
					print "*******************" >> "Incorrect_records"
                               }
           FILENAME=="Input_file1"   {
                                        verify("2");
                               }
           FILENAME=="Input_file2"   {
                                        verify("2,4")
                               }
          '  Input_file1  Input_file2

If you need any additional things in code then kindly show us the sample output for same too.

Thanks,
R. Singh
# 12  
Old 12-23-2016
Hi Ravinder

The code is showing me some syntax error.
Code:
awk: cmd. line:12:                                      else       {
awk: cmd. line:12:                                      ^ syntax error

Also my expectation is that I will be writing this to a test_script.sh

Code:
awk -F"," 'function verify(var){
                                        num=split(var, A);
					print FILENAME " is getting processed now..." >> "correct_records"
					print FILENAME " is getting processed now..." >> "Incorrect_records"
                                        for(i in A){
                                                        if($A[i]){
                                                                        p++
                                                                 }
                                                   };
                                        if(num==p) {
                                                        print >> "correct_records"
                                                   };
					else       {
							print >> "Incorrect_records"
						   };
                                        p=""
					print "*******************" >> "correct_records"
					print "*******************" >> "Incorrect_records"
                               }
           FILENAME=="$1"   {
                                        verify("$2");
                               }
           
          '  $1

So that while calling the test_script.sh script I will pass the file name for $1 and column position for $2 as given below


Code:
sh test_script.sh SAMPLE_FILE 2

# 13  
Old 12-23-2016
Hello ginrkf,

Could you please run following in a script form it should work then as follows.
Code:
awk -F"," 'function verify(var){
                                        num=split(var, A);
                                        for(i in A){
                                                        if($A[i]){
                                                                        p++
                                                                 }
                                                   };
                                        if(num==p) {
                                                         print $0 " with file name: " FILENAME>> "correct_lines"
                                                   };
                                        if(num!=p) {
                                                        print $0 " with file name: " FILENAME >> "incorrect_lines"
                                                   }
                                        p=""
                               }
           FILENAME=="Input_file1"   {
                                        verify("2");
                               }
           FILENAME=="Input_file2"   {
                                        verify("2,4")
                               }
          '   Input_file1    Input_file2

Output came for above is as follows.
Code:
cat incorrect_lines
134,,fg,mnm,mnb with file name: file1
1345,,fg,,mnsb with file name: file2

cat correct_lines
123,asd,fgh,nmb,pok with file name: file1
1235,afg,fgdf,nmbs,posk with file name: file2

Thanks,
R. Singh
# 14  
Old 12-23-2016
Hi,

I have modified the same script in such way that I can pass the value as an external parameter while running.

Code:
vi test_script.sh

Code:
a=$1
awk -F"|" 'function verify(var){
                                        num=split(var, A);
                                        for(i in A){
                                                        if($A[i]){
                                                                        p++
                                                                 }
                                                   };
                                        if(num==p) {
                                                         print $0 " with file name: " FILENAME>> "correct_lines"
                                                   };
                                        if(num!=p) {
                                                        print $0 " with file name: " FILENAME >> "incorrect_lines"
                                                   }
                                        p=""
                               }
           FILENAME=="SAMPLE_FILE"   {
                                        verify("$a");
                               }

          '   SAMPLE_FILE

Code:
sh test_script.sh 2

But in this case its not working.if we hard code the value there its working as expected.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Identify empty file with null record

Hi Team, I have a file abc.dat which is a empty file. But it has null record in first line. I need to identify this unique file and handle it separately. scenario 1: abc/dw> wc abc.dat 1 0 1 abc.dat abc/dw> cat abc.dat abc/dw> scenario 2: abc/dw> wc pqr.dat 0 0 0 pqr.dat... (3 Replies)
Discussion started by: kmanivan82
3 Replies

2. Shell Programming and Scripting

How to find null column?

hi Gurus, I need find the null column in a file. my file like below abc, ,cde,def abc,ded,cdd,def abc, ,ddd,ccd aaa,bbb,ccc,ddd basic, I need to find the lines which second column is null Thanks in advance (3 Replies)
Discussion started by: ken6503
3 Replies

3. Shell Programming and Scripting

Egrep ERE to find null

Hello All, My apologies if a similar question has already been posted. I searched for "grep", "egrep" and "null" I'm having an issue with egrep and an extended regular expression. I'm doing a test for a job name. I would like to have the user enter a job name with no spaces and also test to... (7 Replies)
Discussion started by: sydox
7 Replies

4. Shell Programming and Scripting

how to find null column

Hi, everyone I have a requirement as following: source file 1, abc, def, caaa 2, , cde, aaa 3, bcd, , adefefg I need find columns which contains null value, in above example, I need get two rows 2, , cde, aaa 3, bcd, , adefefg anybody has idea how to achive this ... (5 Replies)
Discussion started by: ken002
5 Replies

5. Shell Programming and Scripting

Find out if few fields in a file are null

Hi, I've a pipe delimited file where I want to find out a number of lines where 1st 2nd and last field are null using awk/sed. Is it possible? Thanks (5 Replies)
Discussion started by: rudoraj
5 Replies

6. UNIX for Dummies Questions & Answers

Using /dev/null with grep and find

Hi, I am trying to display the filename in which a string was found after using find and grep. For this after some googling I found that this works: find -name "*.java" -exec grep "searchStr" {} /dev/null \; I wanted to know the difference between the above and the following: find -name... (0 Replies)
Discussion started by: gaurav_s
0 Replies

7. Shell Programming and Scripting

How can find Null value in If condition

Hi, i wrote If Conditions in my script, it's returns null and some values. but i am unable to find when Null value getting. bec we need modification according null vales. pls help me on this. (2 Replies)
Discussion started by: koti_rama
2 Replies

8. Shell Programming and Scripting

Find null fields in file

Hi All, I have some csv files out of which i want to find records which have empty values in either the 14th or 16th fields. The following is a sample. $cut -d',' -f14,16 SPS* | head -5 VOIP_ORIG_INFO,VOIP_DEST_INFO sip:445600709315@sip.com,sip:999@sip.com... (2 Replies)
Discussion started by: rahulrathod
2 Replies

9. Shell Programming and Scripting

If statement falling over on a null record. Help please.

Okay i've got some code which reads a text file and loops through it and there a few if statements inside where one is failing (the one bolded). Basically the acc column contains a list of three digit access codes, some though have null records (i.e nothing in the field) so what I want to do is... (3 Replies)
Discussion started by: TonyR
3 Replies

10. UNIX for Dummies Questions & Answers

Find files which contain a null character

Hi, I would like to use grep to find files which contain NULL characters. I'm not sure how to represent the null character in the grep statement. Could anyone help please? Thanks! Helen :confused: (5 Replies)
Discussion started by: Bab00shka
5 Replies
Login or Register to Ask a Question