Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Check for null values in a column

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Check for null values in a column
# 15  
Old 04-06-2009
Quote:
Originally Posted by radoulov
Thank you!
When I read your post I did a quick test with the bell-labs awk compiled on Linux
(I thought it was quite similar to the New awk shipped with Solaris) and, as Franklin52 pointed out,
you need to force the recompilation of the current record to get the desired result.

GNU awk and mawk behave the same.

And I confirm that the New awk (/usr/bin/nawk) shipped with Solaris and awk (New awk) on HP-UX recognize it (no recompilation of the current record is needed).

The X-Open POSIX Group awk (/usr/xpg4/bin/awk on Solaris) ignores it silently. The plain old awk (/bin/awk on Solaris) complains with an error.

Solaris:

Code:
% uname -r
5.8
% awk 'FS="," { print $2 }'<<<'a,b 
c,d'
awk: syntax error near line 1
awk: bailing out near line 1
% nawk 'FS="," { print $2 }'<<<'a,b 
c,d'
b
d
% /usr/xpg4/bin/awk 'FS="," { print $2 }'<<<'a,b 
c,d'
%

Linux:

Code:
% newawk 'FS="," { print $2 }'<<<'a,b
c,d'

d
% gawk 'FS="," { print $2 }'<<<'a,b
c,d'

d
% mawk 'FS="," { print $2 }'<<<'a,b
c,d'

d

radoulov,

Interesting, thanks for the unraveling!

Best regards
# 16  
Old 04-06-2009
Bug
Quote:
Originally Posted by Franklin52
radoulov,

Interesting, thanks for the unraveling!

Best regards
Guys the facts were very useful and interesting.. Never knew all these befpre.Thanks all for the efforts you guys put in.Smilie
Is there any way out to check whether a column contains NULL or Not without opening the file or reading it line by line.Any command or any possible way that can help me??!!!
Thanks in advance!
# 17  
Old 04-06-2009
Quote:
Originally Posted by ganesh_248
Guys the facts were very useful and interesting.. Never knew all these befpre.Thanks all for the efforts you guys put in.Smilie
Is there any way out to check whether a column contains NULL or Not without opening the file or reading it line by line.Any command or any possible way that can help me??!!!
Thanks in advance!
Without knowing your OS and shell there are too many possible ways.

You may try something like this (assuming POSIX shell and no embedded newlines in the filenames):


Code:
ifs=$IFS IFS='
'
set -- $(egrep -l '([^,]*,){7} *,' *)
for f in *; do
  for _f; do
    [ "$f" == "$_f" ] && continue 2
  done
  printf '%s\n' "$f" # do whatever you want with the file
done  

IFS=$ifs

On Solaris this script will work with /usr/xpg4/bin/egrep and possibly /usr/xpg4/bin/sh.
Last edited by radoulov; 04-06-2009 at 10:10 AM..
# 18  
Old 04-06-2009
Quote:
Originally Posted by radoulov
Without knowing your OS and shell there are too many possible ways.

You may try something like this (assuming POSIX shell and no embedded newlines in the filenames):


Code:
ifs=$IFS IFS='
'
set -- $(egrep -l '([^,]*,){7} *,' *)
for f in *; do
  for _f; do
    [ "$f" == "$_f" ] && continue 2
  done
  printf '%s\n' "$f" # do whatever you want with the file
done  

IFS=$ifs

On Solaris this script will work with /usr/xpg4/bin/egrep and possibly /usr/xpg4/bin/sh.
I am working on Kshell Radoulov.
# 19  
Old 04-06-2009
... and did you try the script?

What OS?
# 20  
Old 04-07-2009
MySQL
Quote:
Originally Posted by ganesh_248
Hi All,

I have a file with 10 columns and get the required data for nine columns properly except 8th. In 8th column i have both NULL and NON NULL values...i.e
certain records have values for all the columns including 8th column and certain records have 8th column as NULL.My requisite is,without opening or reading the file line by line i need to check if the file contains atleast one NULL value for 8th column .If NULL value exists then i need to reject the file else process the file.

for eg if my file contains records like this,

1,2,3,4,5,6,7,8,9,10
a,s,d,f,f,g,h,,k,l
aa,ss,bb,ee,ff,gg,bb,,kk,ii

how do i get my job done??
Thanks in Advance
i gave my best try for this.....
you can try any of the following three solution..
with grep
Code:
grep -l "^[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,," filenames..

this will display the filenames where 8th column is NULL
with sed
Code:
sed -n '/^[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,,/p' filenames..

with awk
Code:
 
awk '/^[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,,/{exit}' filename..

here once a NULL value found in 8th column it won't search further so it should take little less time Smilie
# 21  
Old 04-07-2009
Quote:
Originally Posted by radoulov
Without knowing your OS and shell there are too many possible ways.

You may try something like this (assuming POSIX shell and no embedded newlines in the filenames):


Code:
ifs=$IFS IFS='
'
set -- $(egrep -l '([^,]*,){7} *,' *)
for f in *; do
  for _f; do
    [ "$f" == "$_f" ] && continue 2
  done
  printf '%s\n' "$f" # do whatever you want with the file
done  

IFS=$ifs

On Solaris this script will work with /usr/xpg4/bin/egrep and possibly /usr/xpg4/bin/sh.
can you explain me your script my frined.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Check for null values in a columns. I have dozen of CSV files in a directory.

Hi Folks, I'm trying to write a simple file sanity check script. I have a directory with dozen CSV files containing id,edname,firstname,lastname,suffix,email. I like to write a awk script to check if first field contain a number and is not empty. and fields number 3,4 & 6 are not empty and... (3 Replies)
Discussion started by: dc34684
3 Replies

2. UNIX for Dummies Questions & Answers

Check for not null column in a pipe delimited file

Hi, I have a requirement where I have to check whether the mandatory columns in a pipe delimited file is null and print error message. For eg, I have to check if the 3rd,5th,6th,7th and 8th column are null and print the message "<column name> is null". The data file will have aroung 100,000... (6 Replies)
Discussion started by: reshma15193
6 Replies

3. Shell Programming and Scripting

Check increment values in column

Gents, I have a file where i would like to check the constant increment by 2 in column 2. 5450 1000 5450 1002 5450 1004 5450 1006 5465 1000 5465 1002 5465 1006 5465 1008 5550 1002 5550 1004 5550 1006 5550 1008 6830 1000 6830 1002 6830 1008 6830 1010 (6 Replies)
Discussion started by: jiam912
6 Replies

4. Shell Programming and Scripting

Script to check for null values in a column

Hi Guys, I am new to shell script.I need your help to write a shell script. I have a csv file which has 13 columns separated by , I have written below script to fetch required 5 columns. awk -F, '(NR==1){h3=$3;h4=$4;h8=$8;h9=$9;h13=$13;next}(NF>1) \ {print... (5 Replies)
Discussion started by: Vivekit82
5 Replies

5. Shell Programming and Scripting

Check null values column

hi, I had a small question.I had a file from which i need to extract data. I have written the below script to check if the file exists and if it exists extract requierd columns from the file. IFILE=/home/home01/Report_1.csv OFILE=/home/home01/name.csv.out1 if #Checks if file exists... (1 Reply)
Discussion started by: Vivekit82
1 Replies

6. Shell Programming and Scripting

Check to identify duplicate values at first column in csv file

Hello experts, I have a requirement where I have to implement two checks on a csv file: 1. Check to see if the value in first column is duplicate, if any value is duplicate script should exit. 2. Check to verify if the value at second column is between "yes" or "no", if it is anything else... (4 Replies)
Discussion started by: avikaljain
4 Replies

7. Shell Programming and Scripting

Check for null values in columns

Hi , I have below data with fixed with of 52 bytes having three columns value data. 01930 MA GLOUCESTER 02033 02025 COHASSET 01960 MA ... (3 Replies)
Discussion started by: sonu_pal
3 Replies

8. UNIX for Dummies Questions & Answers

shift values in one column as header for values in another column

Hi Gurus, I have a tab separated text file with two columns. I would like to make the first column values as headings for the second column values. Ex. >value1 subjects >value2 priorities >value3 requirements ...etc and I want to have a file >value1 subjects >value2 priorities... (4 Replies)
Discussion started by: Unilearn
4 Replies

9. Shell Programming and Scripting

print unique values of a column and sum up the corresponding values in next column

Hi All, I have a file which is having 3 columns as (string string integer) a b 1 x y 2 p k 5 y y 4 ..... ..... Question: I want get the unique value of column 2 in a sorted way(on column 2) and the sum of the 3rd column of the corresponding rows. e.g the above file should return the... (6 Replies)
Discussion started by: amigarus
6 Replies

10. Shell Programming and Scripting

How to check Null values in a file column by column if columns are Not NULLs

Hi All, I have a table with 10 columns. Some columns(2nd,4th,5th,7th,8th and 10th) are Not Null columns. I'll get a tab-delimited file and want to check col by col and generate seperate error code for each col eg:102 if 2nd col value is NULL and 104 if 4th col value is NULL so on... I am a... (7 Replies)
Discussion started by: Mandab
7 Replies
Login or Register to Ask a Question