Using awk to skip record in file


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Using awk to skip record in file
# 1  
Old 08-19-2014
Using awk to skip record in file

I need to amend the code blow such that it reads a "black list" before the "print" statement; if "substr($1,1,6)" is found in the "blacklist" it will ignore that record and continue. the code is from an awk script that is being called from shell script which passes the input values.
Code:
BEGIN { "date +%d/%m/%Y" | getline dateVal; FS = "|" }

{ print "CCA|" $1 "||" substr($1,1,6) "|" substr($1,7,10) "||A|" dateVal "|" dateVal "|Y|N|31"; }

I need something that works like this:
Code:
BEGIN { "date +%d/%m/%Y" | getline dateVal; FS = "|" }
   blist='cat /tmp/bad_values.txt'
    if !substr($1,1,6) in blist then ##here the code will read the entire blist
{ print "CCA|" $1 "||" substr($1,1,6) "|" substr($1,7,10) "||A|" dateVal "|" dateVal "|Y|N|31"; }

I know this is simple, but I really am totally new to this scripting
I am on an AIX server
The "blacklist" will be a multi-lined string file:
Code:
123456
678901
456789

Moderator's Comments:
Mod Comment Please use CODE tags for all sample input, sample output, and code segments.

Last edited by Don Cragun; 08-19-2014 at 08:37 PM.. Reason: additional information
# 2  
Old 08-19-2014
Is this a homework assignment?

Where did you find this shell/awk script that you're modifying?
# 3  
Old 08-19-2014
No Don, this is not homework assignment. This is actual job/work that I was asked to amend as the developer is off on vacation.
# 4  
Old 08-19-2014
Obviously untested (since you didn't give us a complete awk script, show us any sample input, nor desired output, but if you replace what you have shown us with the following, it should be close to what you want:
Code:
BEGIN {	FS = "|" 
	"date +%d/%m/%Y" | getline dateVal
	close("date +%d/%m/%Y")
	while((getline item < "blacklist") == 1) bl[item]
	close("blacklist")
}
!(substr($1,1,6) in bl) {
	printf("CCA|%s||%s|%s||A|%s|%s|Y|N|31\n", $1, substr($1,1,6),
		substr($1,7,10), dateVal, dateVal)
}

# 5  
Old 08-20-2014
Thanks a lot Don, this is definitely a good start. I am doing some reading in order to get a better understanding of UNIX scripting.
# 6  
Old 08-25-2014
this awk command did the trick:
Code:
BEGIN { "date +%d/%m/%Y" | getline dateVal; FS = "|" }  

NR==FNR{a[$1];next} !(substr($1,1,6) in a) {print "CCA|" $1 "||" substr($1,1,6) "|" substr($1,7,10) "||A|" dateVal "|" dateVal "|Y|N|31"; }

 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Keeping record of file 2 based on a reference file 1 in awk

I have 2 input files (tab separated): file1: make_A 1990 foo bar make_B 2010 this that make_C 2004 these those file2: make_X 1970 1995 ref_1:43 ref_2:65 make_A 1970 1995 ref_1:4 ref_2:21 ref_3:18 make_A 1980 2002 ref_1:7 ref_2:7 ref_3:0 ... (2 Replies)
Discussion started by: beca123456
2 Replies

2. Shell Programming and Scripting

Split a large file in n records and skip a particular record

Hello All, I have a large file, more than 50,000 lines, and I want to split it in even 5000 records. Which I can do using sed '1d;$d;' <filename> | awk 'NR%5000==1{x="F"++i;}{print > x}'Now I need to add one more condition that is not to break the file at 5000th record if the 5000th record... (20 Replies)
Discussion started by: ibmtech
20 Replies

3. Shell Programming and Scripting

How to compare current record,with next and previous record in awk without using array?

Hi! all can any one tell me how to compare current record of column with next and previous record in awk without using array my case is like this input.txt 0 32 1 26 2 27 3 34 4 26 5 25 6 24 9 23 0 32 1 28 2 15 3 26 4 24 (7 Replies)
Discussion started by: Dona Clara
7 Replies

4. Shell Programming and Scripting

want to skip a line in XML file using awk

HI All, I am trying to split a xml using awk. now the issue is i want to skip three lines from the xml file. first two and last one based on pattern. plz some one help. i am new to awk and struggling :wall: <?xml version="1.0"?> <notification> ..... ..... ..... ..... ........ (24 Replies)
Discussion started by: ganesan kulasek
24 Replies

5. Shell Programming and Scripting

awk-filter record by another file

I have file1 3049 3138 4672 22631 45324 112382 121240 125470 130289 186128 193996 194002 202776 228002 253221 273523 284601 284605 641858 (8 Replies)
Discussion started by: biomed
8 Replies

6. Shell Programming and Scripting

counting particular record format in a file using AWK

I am trying to count records of particular format from a file and assign it to a variable. I tried below command br_count=wc -l "inputfile.dat"| awk -F"|" '{if (NF != "14") print }' but I amnot able to get it done. Please share me some idea how to get it done. Thanks in advance (7 Replies)
Discussion started by: siteregsam
7 Replies

7. Shell Programming and Scripting

Skip first and last n records with awk

Hi, I have an awk code that reads an input file, checks the 4th column and tells if its fine. #!/bin/ksh { if ($4 == 0) print "fine" else print "some problem" }' FILENAME My problem is that, I dont want to check the first 3 and last 3 lines. This can be hard coded by using BEGIN and END... (9 Replies)
Discussion started by: gotam
9 Replies

8. Shell Programming and Scripting

AWK to skip comments in XML file

Hello, I'm trying to make a shell script to skip comments from an XML file, but with the code below only deletes comments that are in one line. Can you tell me what can be added here? nawk ' { if($0 !~/<!--/) { a=0 } if($0 ~/<!--/ && $0 ~/-->/) {a=1} if($0 ~/<!--/) {a=1} if... (1 Reply)
Discussion started by: majormark
1 Replies

9. Shell Programming and Scripting

Skip parsing the header record - Awk

Guys.... Got a scenario in which I need to skip parsing the header record while I do an awk. Does awk has the flexibility to accomplish this?. If so, how do we do this?. Thanks !!! -Anduzzi :) (2 Replies)
Discussion started by: anduzzi
2 Replies

10. Shell Programming and Scripting

select a record from one file matching from second file using awk

I need help :) I have two input files and I'd like to generate a report based on the two. filea: hostname,account1,password ,account2,password hostname,account1,password hostname,account1,password ,account1,password ,account2,password repeating hostnames are blank fileb: hosta... (7 Replies)
Discussion started by: synmag
7 Replies
Login or Register to Ask a Question