awk command


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk command
# 8  
Old 08-08-2014
Can anyone please assist me?
Condition 1:
if (substr($0,134,1) == "X") , then look for the corresponding matching record(USe Fields - substr($0,2,13),substr($0,136,16),substr($0,114,14)) that has column value 'L' in the same positon and then remove both X and L records.if there are multiple matching L records with the same values in all 3 fields then remove only one L record .
Condition 2:
Incase if there is no corresponding matching record ,then remove only X record.
Condition 3: incase if there are multiple L lines with the same values in all the fields
then we should include all those records in the output file .

Condition 4 : final output contains only L records
sample code:
Code:
awk '
{ if(substr($0,134,1) == "X") {
x[substr($0,2,13),substr($0,136,16),substr($0,114,14)] = $0
} else if(substr($0,134,1) == "L") {
l[substr($0,2,13),substr($0,136,16),substr($0,114,14)] = $0
} else # Print non-L, non-X records.
print
}
END { for(i in x)
if((i in l) || (i in x))
delete l[i]
else
print x[i]
for(i in l)
print l[i]
}' InputFile.txt

InputFile.txt
Desired Output File.txt

Last edited by Scott; 08-08-2014 at 04:54 PM.. Reason: CODE tags, not ICODE tags, please...
# 9  
Old 08-08-2014
Here is a copy of your code with indentation added to make it easier to read and line numbers added for reference in the following notes:
Code:
1 awk '
2 {	if(substr($0,134,1) == "X") {
3		x[substr($0,2,13),substr($0,136,16),substr($0,114,14)] = $0
4	} else if(substr($0,134,1) == "L") {
5		l[substr($0,2,13),substr($0,136,16),substr($0,114,14)] = $0
6	} else # Print non-L, non-X records.
7	print
8 }
9 END {	for(i in x)
10		if((i in l) || (i in x))
11			delete l[i]
12		else
13			print x[i]
14	for(i in l)
15		print l[i]
16}' InputFile.txt

Notes:
  1. Condition 1 says an X records removes one L record. Line 11 deletes all L records for any matching X record.
  2. Condition 4 says final output contains only L records; Lines 7 and 13 print non-L records.
  3. Condition 3 says that if there are multiple identical L records, all of them should be printed. Condition 1 says that if there is an X record, a matching L record is to be removed? These conditions conflict with each other? Which one takes precedence.
  4. Condition 2 is not needed. It restates a subset of the requirements stated by Condition 4.
  5. Is it really necessary to use substr($0, 134, 1) instead of $7?
  6. Is it really necessary to use substr($0, 2, 13) instead of $1?
  7. Is it really necessary to sue substr($0, 136, 16) instead of $8?
  8. Is it really necessary to sue substr($0, 114, 14) instead of $6?
  9. For your given sample data, when the key fields match, the other fields are also identical. Is this always the case? If not, how is your script supposed to decide which one will be deleted when there is an X record with a matching key?
Please clarify your requirements.
# 10  
Old 08-10-2014
Don,

sorry for the confusion .
I'm facing some issues in my code. My current code is not working as expected.
1) Condition 1 says an X records removes one L record. Line 11 deletes all L records for any matching X record.
Ans: My code deletes all matching L records. I need to delete only one L record . I'm not sure how to handle this?
2) Condition 4 says final output contains only L records; Lines 7 and 13 print non-L records.
Ans: I should print only L records . need to fix this part .
3)Condition 3 says that if there are multiple identical L records, all of them should be printed. Condition 1 says that if there is an X record, a matching L record is to be removed? These conditions conflict with each other? Which one takes precedence.
Ans: There will be cases like X records won't be there, but Multiple identical L records will be there in the file. In this case, i have to print all the L records.
4)Condition 2 is not needed. It restates a subset of the requirements stated by Condition 4.
Ans: you are right
5) question 5,6,7,8 - yes, I have to use substr function.
6)For your given sample data, when the key fields match, the other fields are also identical. Is this always the case? If not, how is your script supposed to decide which one will be deleted when there is an X record with a matching key?
Yes, there will be cases like when the key fields match, the other fields are also identical. we can delete any one of the matching record.
if the key fields are not matching, then print all records.
hope this helps . please let me know if you have any other questions.
thanks for looking into my code.
# 11  
Old 08-11-2014
This seems to do what you want:
Code:
awk '
{	if((rec_type = substr($0,134,1)) == "X") {
		# Increment count of X records for this key.
		xc[substr($0,2,13),substr($0,114,14),substr($0,136,16)]++
	} else if(rec_type == "L") {
		# Increment count of L records for this key, and save this L
		# record..
		l[substr($0,2,13),substr($0,114,14),substr($0,136,16),
			++lc[substr($0,2,13),substr($0,114,14),
				substr($0,136,16)]] = $0
	}
}
END {	# For each key, reduce the count of L records saved for that key by the
	# count of X records found for that key, and then print the earliest L
	# records from the file for the given key.
	for(i in lc) {
		# Reduce L count by X count for key i.
		lc[i] -= xc[i]
		# Print L count L records.
		for(j = 1; j <= lc[i]; j++)
			print l[i,j]
	}
}' InputFile.txt

This User Gave Thanks to Don Cragun For This Post:
# 12  
Old 08-12-2014
Don,

Thanks a lot for your help! The code is working as expected. I have tested almost all the scenarios and its perfectly working fine. I'm still testing the code and let you know incase of any issues.
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Pass awk field to a command line executed within awk

Hi, I am trying to pass awk field to a command line executed within awk (need to convert a timestamp into formatted date). All my attempts failed this far. Here's an example. It works fine with timestamp hard-codded into the command echo "1381653229 something" |awk 'BEGIN{cmd="date -d... (4 Replies)
Discussion started by: tuxer
4 Replies

2. Shell Programming and Scripting

Multiple command execution inside awk command during xml parsing

below is the output xml string from some other command and i will be parsing it using awk cat /tmp/alerts.xml <Alert id="10102" name="APP-DS-ds_ha-140018-componentFailure-S" alertDefinitionId="13982" resourceId="11427" ctime="1359453507621" fixed="false" reason="If Event/Log Level(ANY) and... (2 Replies)
Discussion started by: vivek d r
2 Replies

3. Shell Programming and Scripting

awk command to compare a file with set of files in a directory using 'awk'

Hi, I have a situation to compare one file, say file1.txt with a set of files in directory.The directory contains more than 100 files. To be more precise, the requirement is to compare the first field of file1.txt with the first field in all the files in the directory.The files in the... (10 Replies)
Discussion started by: anandek
10 Replies

4. Shell Programming and Scripting

awk command in script gives error while same awk command at prompt runs fine: Why?

Hello all, Here is what my bash script does: sums number columns, saves the tot in new column, outputs if tot >= threshold val: > cat getnon0file.sh #!/bin/bash this="getnon0file.sh" USAGE=$this" InFile="xyz.38" Min="0.05" # awk '{sum=0; for(n=2; n<=NF; n++){sum+=$n};... (4 Replies)
Discussion started by: catalys
4 Replies

5. Shell Programming and Scripting

awk command for simple join command but based on 2 columns

input1 a_a a/a 10 100 a1 a_a 20 200 b1 b_b 30 300 input2 a_a a/a xxx yyy a1 a1 lll ppp b1 b_b kkk ooo output a_a a/a 10 100 xxx yyy (2 Replies)
Discussion started by: ruby_sgp
2 Replies

6. UNIX for Dummies Questions & Answers

Basic awk question...getting awk to act on $1 of the command itself

I have a script problem that I am not able to solve due my very limited understanding of unix/awk. This is the contents of test.sh awk '{print $1}' From the prompt if I enter: ./test.sh Hello World I would expect to see "Hello" but all I get is a blank line. Only then if I enter "Hello... (2 Replies)
Discussion started by: JasonHamm
2 Replies

7. Shell Programming and Scripting

awk/sed Command : Parse parameter file / send the lines to the ksh export command

Sorry for the duplicate thread this one is similar to the one in https://www.unix.com/shell-programming-scripting/88132-awk-sed-script-read-values-parameter-files.html#post302255121 Since there were no responses on the parent thread since it got resolved partially i thought to open the new... (4 Replies)
Discussion started by: rajan_san
4 Replies

8. Shell Programming and Scripting

assign a command line argument and a unix command to awk variables

Hi , I have a piece of code ...wherein I need to assign the following ... 1) A command line argument to a variable e.g origCount=ARGV 2) A unix command to a variable e.g result=`wc -l testFile.txt` in my awk shell script When I do this : print "origCount" origCount --> I get the... (0 Replies)
Discussion started by: sweta_doshi
0 Replies
Login or Register to Ask a Question