Visit Our UNIX and Linux User Community

Script to check for null values in a column

Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script to check for null values in a column
# 1  
Old 09-03-2013
Script to check for null values in a column

Hi Guys,

I am new to shell script.I need your help to write a shell script.
I have a csv file which has 13 columns separated by ,
I have written below script to fetch required 5 columns.
 awk -F, '(NR==1){h3=$3;h4=$4;h8=$8;h9=$9;h13=$13;next}(NF>1) \
{print h3":"$3"\n"h4":"$4"\n"h8":"$8"\n"h9":"$9"\n"h13":"$13}' $IFILE > $OFILE

Now i want to include condition in above script to check if column 13 is null/empty and if yes print ouptput "error in file".
# 2  
Old 09-03-2013
awk -F, '! $13 { print "Error in file">"/dev/null"; exit }
{print h3":"$3"\n"h4":"$4"\n"h8":"$8"\n"h9":"$9"\n"h13":"$13}' $IFILE > $OFILE

# 3  
Old 09-03-2013
Thanks for the reply.
I am tried by using the below if loop after my print stattement but not working..any idea why.
if [ $h13 == "" ]; then
echo "error in file format column " > $OFILE

also tried
if [ $h13 = "" ]; then
echo "error in file format column " > $OFILE

Last edited by vbe; 09-03-2013 at 01:56 PM..
# 4  
Old 09-03-2013
Because awk is awk, shell is shell, and they don't use each others' variables.

What are you trying to do?
# 5  
Old 09-03-2013
Try this:
if ! awk -F, '! $13 { print "Error in file">"/dev/null"; exit 1 }
        {print h3":"$3"\n"h4":"$4"\n"h8":"$8"\n"h9":"$9"\n"h13":"$13}' $IFILE > $OFILE
        echo "error"

# 6  
Old 09-04-2013
Should be $13=="" not ! $13; the latter treats a 0 like a "".
If you want to include the error in the output file, you can do that in awk
awk -F, '
($13=="") {print "Error in file"; exit 1}
(NR==1) {h3=$3;h4=$4;h8=$8;h9=$9;h13=$13; next}
{print h3":"$3"\n"h4":"$4"\n"h8":"$8"\n"h9":"$9"\n"h13":"$13}

These 2 Users Gave Thanks to MadeInGermany For This Post:

Previous Thread | Next Thread
Test Your Knowledge in Computers #172
Difficulty: Easy
Red Hat is the largest contributor to the GNOME desktop environment.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk script to append suffix to column when column has duplicated values

Please help me to get required output for both scenario 1 and scenario 2 and need separate code for both scenario 1 and scenario 2 Scenario 1 i need to do below changes only when column1 is CR and column3 has duplicates rows/values. This inputfile can contain 100 of this duplicated rows of... (1 Reply)
Discussion started by: as7951
1 Replies

2. UNIX for Beginners Questions & Answers

Check for null values in a columns. I have dozen of CSV files in a directory.

Hi Folks, I'm trying to write a simple file sanity check script. I have a directory with dozen CSV files containing id,edname,firstname,lastname,suffix,email. I like to write a awk script to check if first field contain a number and is not empty. and fields number 3,4 & 6 are not empty and... (3 Replies)
Discussion started by: dc34684
3 Replies

3. UNIX for Dummies Questions & Answers

Check for not null column in a pipe delimited file

Hi, I have a requirement where I have to check whether the mandatory columns in a pipe delimited file is null and print error message. For eg, I have to check if the 3rd,5th,6th,7th and 8th column are null and print the message "<column name> is null". The data file will have aroung 100,000... (6 Replies)
Discussion started by: reshma15193
6 Replies

4. Shell Programming and Scripting

Check increment values in column

Gents, I have a file where i would like to check the constant increment by 2 in column 2. 5450 1000 5450 1002 5450 1004 5450 1006 5465 1000 5465 1002 5465 1006 5465 1008 5550 1002 5550 1004 5550 1006 5550 1008 6830 1000 6830 1002 6830 1008 6830 1010 (6 Replies)
Discussion started by: jiam912
6 Replies

5. Shell Programming and Scripting

Check null values column

hi, I had a small question.I had a file from which i need to extract data. I have written the below script to check if the file exists and if it exists extract requierd columns from the file. IFILE=/home/home01/Report_1.csv OFILE=/home/home01/name.csv.out1 if #Checks if file exists... (1 Reply)
Discussion started by: Vivekit82
1 Replies

6. Shell Programming and Scripting

Check to identify duplicate values at first column in csv file

Hello experts, I have a requirement where I have to implement two checks on a csv file: 1. Check to see if the value in first column is duplicate, if any value is duplicate script should exit. 2. Check to verify if the value at second column is between "yes" or "no", if it is anything else... (4 Replies)
Discussion started by: avikaljain
4 Replies

7. Shell Programming and Scripting

Check for null values in columns

Hi , I have below data with fixed with of 52 bytes having three columns value data. 01930 MA GLOUCESTER 02033 02025 COHASSET 01960 MA ... (3 Replies)
Discussion started by: sonu_pal
3 Replies

8. Shell Programming and Scripting

print unique values of a column and sum up the corresponding values in next column

Hi All, I have a file which is having 3 columns as (string string integer) a b 1 x y 2 p k 5 y y 4 ..... ..... Question: I want get the unique value of column 2 in a sorted way(on column 2) and the sum of the 3rd column of the corresponding rows. e.g the above file should return the... (6 Replies)
Discussion started by: amigarus
6 Replies

9. UNIX for Dummies Questions & Answers

Check for null values in a column

Hi All, I have a file with 10 columns and get the required data for nine columns properly except 8th. In 8th column i have both NULL and NON NULL values...i.e certain records have values for all the columns including 8th column and certain records have 8th column as NULL.My requisite is,without... (20 Replies)
Discussion started by: ganesh_248
20 Replies

10. Shell Programming and Scripting

How to check Null values in a file column by column if columns are Not NULLs

Hi All, I have a table with 10 columns. Some columns(2nd,4th,5th,7th,8th and 10th) are Not Null columns. I'll get a tab-delimited file and want to check col by col and generate seperate error code for each col eg:102 if 2nd col value is NULL and 104 if 4th col value is NULL so on... I am a... (7 Replies)
Discussion started by: Mandab
7 Replies

Featured Tech Videos