Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Find Null values in Columns and fail execution by displaying error message Post 302940488 by tpk on Monday 6th of April 2015 12:09:08 PM
Old 04-06-2015
Linux Find Null values in Columns and fail execution by displaying error message

Hi All,

I am new to shell scripting. I have a requirement as part of my job to find out null/empty values in column 2 and column 3 from a CSV file and exit the further execution of script by displaying a simple error message.

I have developed a script to do this by reading various articles from this forum but ended up with no luck, each time I run the script it is going to else part of the loop and printing "All records are Good" and Line no 17 -F,:not found syntax error.

PFB my script and sample file which I am using for testing,

Sample file: pf_20120406.csv
Code:
D,xsn,des,1.23,3.43,34.9203,234.23423,12.345,234.987,987.342,789.345,679.0987,345.657,345.987,987.098
D,xsn,des,1.23,3.43,34.9203,234.23423,12.345,234.987,987.342,789.345,679.0987,345.657,345.987,987.098
D,,,1.23,3.43,34.9203,234.23423,12.345,234.987,987.342,789.345,679.0987,345.657,345.987,987.098
D,xsn,des,1.23,3.43,34.9203,234.23423,12.345,234.987,987.342,789.345,679.0987,345.657,345.987,987.098
D,xsn,des,1.23,3.43,34.9203,234.23423,12.345,234.987,987.342,789.345,679.0987,345.657,345.987,987.098
D,,des,1.23,3.43,34.9203,234.23423,12.345,234.987,987.342,789.345,679.0987,345.657,345.987,987.098
D,xsn,des,1.23,3.43,34.9203,234.23423,12.345,234.987,987.342,789.345,679.0987,345.657,345.987,987.098
D,xsn,,1.23,3.43,34.9203,234.23423,12.345,234.987,987.342,789.345,679.0987,345.657,345.987,987.098

pf_validations.sh
Code:
#!/usr/bin/sh
#
#filename=`ls -ltr /var/datastage/FRPDEVL/work/source/landing/dspf/pf_*.csv`
#
for fname in /var/datastage/FRPDEVL/work/source/landing/dspf/pf_*.csv;do
filename=`basename $fname`
fdate=`echo $filename|tr -dc '[:digit:]'`

  #Validation 1: Column B and C should not have a null value they are of varchar type.
  if [ $filename = 'pf_'$fdate'.csv' ]
  then
  echo "------------------------------------------------------------------------------------"
  echo "Checking Specific Validations for File: pf_$fdate.csv"
  echo "------------------------------------------------------------------------------------"
  sed '1d;$d' pf_$fdate.csv >pf_$fdate.tmp
  #grep -lE '(,,|,$)' *.csv
  value=`awk -F"," '{if ( $2 ~ /^ *$/ ) printf("102") if ( $3 ~ /^ *$/ ) printf("104") printf("\n")}' pf_$fdate.tmp`
  #while read line;do
  if [[ "$value" == "102" || "$value" == "104" ]]
  then
  echo "Null value present in file"
  echo "Hence exiting the Job"
  rm -f pf_$fdate.tmp
  exit 16  
  else
  echo "All records are good"  
  fi
  #done <pf_$fdate.tmp
  echo "------------------------------------------------------------------------------------"
  echo "Specific Validations check for File: pf_$fdate.csv completed"
  echo "------------------------------------------------------------------------------------"  
  fi
  
  rm -f pf_$fdate.tmp
 
done

Please help me how can I achieve my requirement and help me with the correct code snippet.

It would be of great help. Thanking you all in advance

With Regards,
TPK.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to check Null values in a file column by column if columns are Not NULLs

Hi All, I have a table with 10 columns. Some columns(2nd,4th,5th,7th,8th and 10th) are Not Null columns. I'll get a tab-delimited file and want to check col by col and generate seperate error code for each col eg:102 if 2nd col value is NULL and 104 if 4th col value is NULL so on... I am a... (7 Replies)
Discussion started by: Mandab
7 Replies

2. Shell Programming and Scripting

Need to return fail or pass from shell script on the basis of pl/sql code execution

Hi guys, I am quite new in shell scripting. I am tring to promote some oracle jobs into control-M. In control-M, I am calling a script which establishes a connection with database and execute some procedures. Now I want if that PL/sql Block got failed script should return failure to... (2 Replies)
Discussion started by: alok1301
2 Replies

3. Shell Programming and Scripting

find error?? find / -name "something.txt" 2>/dev/null

why is this giving me errors? i type this in: find / -name "something.txt" 2>/dev/null i get the following error messages: find: bad option 2 find: path-list predicate-list :confused: (5 Replies)
Discussion started by: magiling
5 Replies

4. Shell Programming and Scripting

How to list Matching Directories OR NULL w/o error message?

I want to be able to get all the directories in a path into a variable array, BUT if there ARE NO directories I want the Variable to be NULL and not echo any error message! If there ARE directories, this will get the list of the directories whose name begins with the string "20":... (6 Replies)
Discussion started by: pgorbas
6 Replies

5. Shell Programming and Scripting

Check for null values in columns

Hi , I have below data with fixed with of 52 bytes having three columns value data. 01930 MA GLOUCESTER 02033 02025 COHASSET 01960 MA ... (3 Replies)
Discussion started by: sonu_pal
3 Replies

6. UNIX for Dummies Questions & Answers

Find duplicated values in two columns out of three

hi! could u help in the following? I have the data (long list!) that looks like (three coumns white space separated): rs3094315 0.0665173 742429 rs12562034 0.0738998 758311 rs3934834 0.396449 995669 rs9442372 0.402693 1008567 rs3737728 0.406271 1011278 rs6687776 0.435429 1020428 rs9651273... (4 Replies)
Discussion started by: kush
4 Replies

7. UNIX for Dummies Questions & Answers

Eliminate error message (/dev/null)?

I am trying to eliminate an error message from a script. This is the error message: find: stat() error /usr/openv/netbackup/db/images/KUMAX: No such file or directory if ]; then runthiscommand=`su nxadm -c "ssh -t $new3 exec /bin/sh -s">/tmp/filew3 2>/tmp/error.txt<<EOF ... (1 Reply)
Discussion started by: newbie2010
1 Replies

8. Shell Programming and Scripting

Find for line with not null values at nth place in pipe delimited file

Hi, I am trying to find the lines in a pipe delimited file where 11th column has not null values. Any help is appreciated. Need help asap please. thanks in advance. (3 Replies)
Discussion started by: manikms
3 Replies

9. Shell Programming and Scripting

Multiple columns replace with null values.

I am trying to replace the partcular columns(Col3,col5,col20,col44,col55,co56,col59,col60,col61,col62,col74,col75,col88,col90,col91,col93,col94,col95) with empty Input file Col1,col2,col3,col4,col5------,col100 1,2,3,4,5,---------,100 3,4,5,6,7,---------,300 Output : ... (3 Replies)
Discussion started by: onesuri
3 Replies

10. UNIX for Beginners Questions & Answers

Check for null values in a columns. I have dozen of CSV files in a directory.

Hi Folks, I'm trying to write a simple file sanity check script. I have a directory with dozen CSV files containing id,edname,firstname,lastname,suffix,email. I like to write a awk script to check if first field contain a number and is not empty. and fields number 3,4 & 6 are not empty and... (3 Replies)
Discussion started by: dc34684
3 Replies
RENICE(1)                                                          User Commands                                                         RENICE(1)

NAME
renice - alter priority of running processes SYNOPSIS
renice [-n] priority [-g|-p|-u] identifier... DESCRIPTION
renice alters the scheduling priority of one or more running processes. The first argument is the priority value to be used. The other arguments are interpreted as process IDs (by default), process group IDs, user IDs, or user names. renice'ing a process group causes all processes in the process group to have their scheduling priority altered. renice'ing a user causes all processes owned by the user to have their scheduling priority altered. OPTIONS
-n, --priority priority Specify the scheduling priority to be used for the process, process group, or user. Use of the option -n or --priority is optional, but when used it must be the first argument. -g, --pgrp Interpret the succeeding arguments as process group IDs. -p, --pid Interpret the succeeding arguments as process IDs (the default). -u, --user Interpret the succeeding arguments as usernames or UIDs. -V, --version Display version information and exit. -h, --help Display help text and exit. EXAMPLES
The following command would change the priority of the processes with PIDs 987 and 32, plus all processes owned by the users daemon and root: renice +1 987 -u daemon root -p 32 NOTES
Users other than the superuser may only alter the priority of processes they own. Furthermore, an unprivileged user can only increase the ``nice value'' (i.e., choose a lower priority) and such changes are irreversible unless (since Linux 2.6.12) the user has a suitable ``nice'' resource limit (see ulimit(1) and getrlimit(2)). The superuser may alter the priority of any process and set the priority to any value in the range -20 to 19. Useful priorities are: 19 (the affected processes will run only when nothing else in the system wants to), 0 (the ``base'' scheduling priority), anything negative (to make things go very fast). FILES
/etc/passwd to map user names to user IDs SEE ALSO
nice(1), getpriority(2), setpriority(2), credentials(7), sched(7) HISTORY
The renice command appeared in 4.0BSD. AVAILABILITY
The renice command is part of the util-linux package and is available from Linux Kernel Archive <https://www.kernel.org/pub/linux/utils /util-linux/>. util-linux July 2014 RENICE(1)
All times are GMT -4. The time now is 08:09 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy