Unix/Linux Go Back    


UNIX for Beginners Questions & Answers If you're not sure where to post a Unix or Linux question, post it here. All unix and Linux beginners welcome in this forum!

Check for null values in a columns. I have dozen of CSV files in a directory.

UNIX for Beginners Questions & Answers


Tags
awk, shel

Reply    
 
Thread Tools Search this Thread Display Modes
    #1  
Old Unix and Linux 12-07-2017   -   Original Discussion by dc34684
dc34684's Unix or Linux Image
dc34684 dc34684 is offline
Registered User
 
Join Date: Dec 2017
Last Activity: 7 December 2017, 3:15 PM EST
Posts: 2
Thanks: 0
Thanked 0 Times in 0 Posts
Hammer & Screwdriver Check for null values in a columns. I have dozen of CSV files in a directory.

Hi Folks,

I'm trying to write a simple file sanity check script. I have a directory with dozen CSV files containing id,edname,firstname,lastname,suffix,email.

I like to write a awk script to check if first field contain a number and is not empty. and fields number 3,4 & 6 are not empty and that the file contains 6 fields no more no less than 6, if all of this conditions are true nothing happens but if any of these conditions failed, re-name the file to .bad. Here is what i have done so far.



Code:
for f in *.csv;
do 
  awk -F, '!(NF==6 && $1+0==$1 && $6 == ""){f=1; exit} END{exit f}' "$f" || mv "$f" "$f".bad;
done


Moderator's Comments:
Check for null values in a columns. I have dozen of CSV files in a directory.
Please wrap all code, data, files input & output/errors in CODE tags.
It makes them easier to read and preserves spacing for indentation and fixed-width data.

Last edited by rbatte1; 12-07-2017 at 11:39 AM.. Reason: Added CODE tags
Sponsored Links
    #2  
Old Unix and Linux 12-07-2017   -   Original Discussion by dc34684
RudiC's Unix or Linux Image
RudiC RudiC is offline Forum Staff  
Moderator
 
Join Date: Jul 2012
Last Activity: 19 April 2018, 4:52 PM EDT
Location: Aachen, Germany
Posts: 12,492
Thanks: 401
Thanked 3,871 Times in 3,559 Posts
So - ?
Sponsored Links
    #3  
Old Unix and Linux 12-07-2017   -   Original Discussion by dc34684
dc34684's Unix or Linux Image
dc34684 dc34684 is offline
Registered User
 
Join Date: Dec 2017
Last Activity: 7 December 2017, 3:15 PM EST
Posts: 2
Thanks: 0
Thanked 0 Times in 0 Posts
i am not sure what you mean by ?. in any case never mind.
    #4  
Old Unix and Linux 12-07-2017   -   Original Discussion by dc34684
Don Cragun's Unix or Linux Image
Don Cragun Don Cragun is offline Forum Staff  
Administrator
 
Join Date: Jul 2012
Last Activity: 19 April 2018, 3:27 PM EDT
Location: San Jose, CA, USA
Posts: 11,206
Thanks: 633
Thanked 3,903 Times in 3,341 Posts
What ? means is:
Is what you have so far working as you expect it to work? If not, in what way is it failing to work?
  • Are you getting diagnostic messages? (What are they?)
  • Are files being renamed that should not be renamed? (Under what conditions?)
  • Are files not being renamed that should be renamed? (Under what conditions?)
And, if so, where are you stuck in adding code for the remaining checks?
The Following User Says Thank You to Don Cragun For This Useful Post:
RudiC (12-08-2017)
Sponsored Links
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Multiple columns replace with null values. onesuri Shell Programming and Scripting 3 12-29-2015 08:09 AM
Replace null values in csv with zero reignangel2003 Shell Programming and Scripting 1 09-02-2015 06:32 AM
Check null values column Vivekit82 Shell Programming and Scripting 1 09-04-2013 12:54 PM
Check for null values in columns sonu_pal Shell Programming and Scripting 3 08-28-2012 04:53 AM
How to check Null values in a file column by column if columns are Not NULLs Mandab Shell Programming and Scripting 7 03-15-2008 09:57 AM



All times are GMT -4. The time now is 01:09 AM.