Need help with shell script for chekking a column in txt file - pipe delimited


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Need help with shell script for chekking a column in txt file - pipe delimited
# 1  
Old 12-23-2008
Need help with shell script for chekking a column in txt file - pipe delimited

Hi:

I have a text file date(pipe delimited) which is loaded in to the DB using sql loader(&CTL files) after some initial validation by the shell script.
Now i have a situation where the shell script needs to check a column in the text file and if it is NULL then it needs send this record/row to another new file ( later this file will be used to attach in the email to correct the null column), i do not want to delete the record from the original file but just want to check that column and if its null send the row to new file...here is data in the text file which i trimmed/formated for easy understanding:

Code:
HDR|ABCD|10-13-2008 to 10-19-2008.txt|10-19-2008|XYZ
DTL|H|5464-1|0|02-02-2008|02-03-2008||||F|||||||||
DTL|D|5464-1|1|02-02-2008|02-03-2008|1|M|JJJ
DTL|D|5464-1|2|02-02-2008|02-03-2008|1|M|JJJ
DTL|D|5464-1|3|02-02-2008|02-03-2008|1||JJJ
DTL|D|5464-1|4|02-02-2008|02-03-2008|1||JJJ
TRL|ABCD|10-13-2008 to 10-19-2008.Txt|10-19-2008|170|XYZ

I would like to check the 8th column which has 'M' for couple of records and NULL/nothing for couple others, i would like to send the ones which are NULL to new file.

Any kind of suggestion would be really appreciated.

thanks,
ravi

Last edited by vgersh99; 12-23-2008 at 02:50 PM.. Reason: code tags, PLEASE!
# 2  
Old 12-23-2008
Code:
nawk -F'|' '$8 == "" ' myPipeFile >nullFile.txt


Last edited by vgersh99; 12-23-2008 at 02:39 PM.. Reason: missed input file
# 3  
Old 12-23-2008
Thanks VGERSH99.
But could you please tell me how do i test it on the unix command prompt...i mean how do i send in the input file? so that it checks for the null column in all the records?

apreciate it.


thanks,
ravi
# 4  
Old 12-23-2008
Quote:
Originally Posted by ravi0435
Thanks VGERSH99.
But could you please tell me how do i test it on the unix command prompt...i mean how do i send in the input file? so that it checks for the null column in all the records?

apreciate it.


thanks,
ravi
my bad - missed the input file - edited the original post.
# 5  
Old 12-23-2008
Hi:

thanks again

If i execute this:

nawk -F'|' '$8 == "" ' inst.txt >null.txt
the null.txt is getting all the records dumped in to it as it is from the original file.

nawk -F'|' '$8 == " " ' inst.txt >null.txt
(with a space in " ")

the null.txt is empty.
# 6  
Old 12-23-2008
Quote:
Originally Posted by ravi0435
Hi:

thanks again

If i execute this:

nawk -F'|' '$8 == "" ' inst.txt >null.txt
the null.txt is getting all the records dumped in to it as it is from the original file.
it means that ALL your records have an 'empty' 8-th field
Quote:
Originally Posted by ravi0435
nawk -F'|' '$8 == " " ' inst.txt >null.txt
(with a space in " ")

the null.txt is empty.
it means that NONE of your records have an 8-th field containing ' ' (space).
# 7  
Old 12-23-2008
I noticed some of your records/lines have less than 8 fields - you might want to 'skip' those:
Code:
nawk -F'|' 'NF>=8 && $8 == "" ' myPipeFile >nullFile.txt

 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replace delimiter for a particular column in a pipe delimited file

I have an input file as below Emp1|FirstName|MiddleName|LastName|Address|Pincode|PhoneNumber 1234|FirstName1|MiddleName2|LastName3| Add1 || ADD2|123|000000000 Output : 1234|FirstName1|MiddleName2|LastName3| Add1 ,, ADD2|123|000000000 OR 1234,FirstName1,MiddleName2,LastName3, Add1 ||... (2 Replies)
Discussion started by: styris
2 Replies

2. UNIX for Dummies Questions & Answers

Check for not null column in a pipe delimited file

Hi, I have a requirement where I have to check whether the mandatory columns in a pipe delimited file is null and print error message. For eg, I have to check if the 3rd,5th,6th,7th and 8th column are null and print the message "<column name> is null". The data file will have aroung 100,000... (6 Replies)
Discussion started by: reshma15193
6 Replies

3. UNIX for Dummies Questions & Answers

Need to convert a pipe delimited text file to tab delimited

Hi, I have a rquirement in unix as below . I have a text file with me seperated by | symbol and i need to generate a excel file through unix commands/script so that each value will go to each column. ex: Input Text file: 1|A|apple 2|B|bottle excel file to be generated as output as... (9 Replies)
Discussion started by: raja kakitapall
9 Replies

4. Shell Programming and Scripting

Replacing a column in a pipe delimited file

Hi, I have a pipe delimited file as below and I need to replace the 2nd column of each line with null values. 1|10/15/2011|fname1|lname1 2|10/15/2012|fname2|lname2 3|10/15/2013|fname3|lname3 Output file: 1||fname1|lname1 2||fname2|lname2 3||fname3|lname3 I tried this ... (2 Replies)
Discussion started by: member2014
2 Replies

5. Shell Programming and Scripting

Replace pipe delimited column string to null

Hi All, I have a large dat file where each lines are pipe delimited values. I need to parse the file depending on the request. For example: sometimes I have told to remove all the values in the 7th column (this case remove values '3333' only from the first line and '3543' from the second line)... (4 Replies)
Discussion started by: express14
4 Replies

6. Homework & Coursework Questions

how to show particular column from pipe delimited file

hi, I have pipe delimited flat file as below 1|ab|4.5|9| 2|ac|3|12| 3|ac|4.5|8| i want to show (display) only 3rd field between pipes. please help (1 Reply)
Discussion started by: vai15517
1 Replies

7. Shell Programming and Scripting

Help with converting Pipe delimited file to Tab Delimited

I have a file which was pipe delimited, I need to make it tab delimited. I tried with sed but no use cat file | sed 's/|//t/g' The above command substituted "/t" not tab in the place of pipe. Sample file: abc|123|2012-01-30|2012-04-28|xyz have to convert to: abc 123... (6 Replies)
Discussion started by: karumudi7
6 Replies

8. Shell Programming and Scripting

How to insert a sequence number column inside a pipe delimited csv file using shell scripting?

Hi All, I need a shell script which could insert a sequence number column inside a dat file(pipe delimited). I have the dat file similar to the one as shown below.. |A|B|C||D|E |F|G|H||I|J |K|L|M||N|O |P|Q|R||S|T As shown above, the column 4 is currently blank and i need to insert sequence... (5 Replies)
Discussion started by: nithins007
5 Replies

9. Shell Programming and Scripting

count of null in pipe delimited txt file

Hi, I have a pipe delimited txt file which contains 17 fields per line/row. 16th field contains email id. I want to count the number of lines/rows that contains null in the 16th field. Plz find attached example data file. I'm looking for a command line/script which achieves this. ... (5 Replies)
Discussion started by: Sriranga
5 Replies

10. Shell Programming and Scripting

remove first column of a space delimited txt

how to remove the first column of a space delimited txt file? there are 12+ columns... what is the cleanest way? could use awk and print all but the first, but it looks kinda ugly awk '{print $2" "$3" "$4" "$5" "$6" "$7" "$8" "$9" "$10" "$11" "$12"}' file.txt whats a better way? (1 Reply)
Discussion started by: ajp7701
1 Replies
Login or Register to Ask a Question