Add rules for multiple lines processing


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Add rules for multiple lines processing
# 1  
Old 07-17-2014
Add rules for multiple lines processing

Hi ,

I want to process the below file and add rules for grouping based on key in unix shell script. For eg

Input file :

I have 3 columns and key column is group

Code:
Group  status application
G1     Complete   A1
G1     Delay        A2
G2     Complete   A3,A4
G3     Delay        A5, A6
G3     Complete   A7,A8


Output i need is a single record for each group.

Logic is that if any application in a group is delay status then status of group should be delay, else if all are complete in group then status complete

Code:
Group Status Application
G1    Delay    A2
G2    Complete A3,A4
G3    Delay     A5, A6


i am not able to get the correct solution for this , Please help me with the scripting
Moderator's Comments:
Mod Comment When sample input or output contains aligned fields, using CODE tags is crucial to show spaces and tabs in the input and output samples. Without the tags, multiple adjacent spaces and tabs are displayed as a single space. Get into the habit of always using CODE tags when showing code segments, sample input, and sample output.

Last edited by Don Cragun; 07-17-2014 at 05:11 AM.. Reason: Drop Bold tags, add CODE tags.
# 2  
Old 07-17-2014
A simple approach may be to use the last one:
Code:
awk 'NR==1{print; next}{A[$1]=$0} END{for(i in A)print A[i]}' file

The NR==1 is to print the header. If there is no header then you can use:
Code:
awk '{A[$1]=$0} END{for(i in A)print A[i]}' file

# 3  
Old 07-17-2014
thanks,

But my question is that how would i add logic to verify that if any group has status complete and delay both , then in the output it should pick delay.
2nd scenario is that if it has only complete then the status would be complete in output
# 4  
Old 07-17-2014
OK, try this:
Code:
awk 'NR==FNR{if($2=="Delay")D[$1]; next} !($2=="Complete" && $1 in D)' file file

Note: the file is specified twice (and so gets read twice)..
# 5  
Old 07-17-2014
If all lines for a given group are contiguous in your input file, the following may be better than Scrutinizer's suggestion. It only reads the file once and doesn't care if there are lines with status other than "Complete" and "Delay":
Code:
awk '
NR==1 {	print
	next
}
g != $1 {
	if(NR > 2) print o
	d = 0
	g = $1
}
!d {	o = $0
	d = ($2 == "Delay")
}
END {	print o
}' file

With your sample input, this produces:
Code:
Group  status application
G1     Delay        A2
G2     Complete   A3,A4
G3     Delay        A5, A6

which duplicates the data found in your sample input, but has spacing that is very different from the output you said you wanted. (I didn't see any obvious way to duplicate the seemingly random spacing in your sample output, and I didn't bother changing the capitalization in your header line.)

If data for some groups is not all on contiguous lines, the following still just reads your input file once:
Code:
awk '
NR==1 {	print
	next
}
D[$1] != "Delay" {
	A[$1] = $0
	D[$1] = $2
}
END {	for(i in A)
		print A[i]
}' file

but the output (other than the header) is in random order.

If the output order is important and all lines for a group are not contiguous in your input files, use Scrutinizer's code instead. But note that it won't work if there is a line in your input file with a status other than "Delay" or "Complete".
# 6  
Old 07-17-2014
Quote:
Originally Posted by Don Cragun
[..]
If the output order is important and all lines for a group are not contiguous in your input files, use Scrutinizer's code instead. But note that it won't work if there is a line in your input file with a status other than "Delay" or "Complete".
Good point. If need be that could be remedied like so:
Code:
awk 'NR==FNR{if($2==s)D[$1]; next} $2==s || !($1 in D)' s=Delay file file

# 7  
Old 07-18-2014
Hi ,
Thanks for the response

But there is a slight change and one new status has been added

now my input is like

Code:
Group  status application
G1     BAU        A1
G1     Delay        A2
G1     Delayed     A9
G2     BAU   A3 
G2     Delayed    A4
G3     Delay        A5, A6
G3     BAU   A7,A8


Expected output is

Code:
G1   Delay A2
G2   Delayed  A4
G3   Delay A5,A6

so priorties are in the order Delay(1) , Delayed(2) and BAU(3)

if all 3 status are present ( BAU , Delay & delayed) then require Delay in output
if BAU & Delay - Delay
BAU & Delayed - Delayed
Delay and Delayed - Delay

Last edited by Don Cragun; 07-18-2014 at 04:54 AM.. Reason: Add CODE tags.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing multiple lines from input file, if multiple lines match a pattern.

GM, I have an issue at work, which requires a simple solution. But, after multiple attempts, I have not been able to hit on the code needed. I am assuming that sed, awk or even perl could do what I need. I have an application that adds extra blank page feeds, for multiple reports, when... (7 Replies)
Discussion started by: jxfish2
7 Replies

2. Shell Programming and Scripting

Processing multiple files

Hello I have a program cfxfrwb which is designed to remove headers from reports files. The cfxfrwb is located in the following directory /u01/efin/v40/live/bin I run the program against a single report file in the temp directory and it does it's job../cfxfrwb... (2 Replies)
Discussion started by: captainrhodes
2 Replies

3. Shell Programming and Scripting

File lines starts with # not processed or exclude that lines from processing

I have a file like below #Fields section bald 1234 2345 456 222 abcs dddd dddd ssss mmmm mmm mmm i need do not process a files stating with # I was written code below while read -r line do if then echo ${line} >> elif then ... (3 Replies)
Discussion started by: Chenchireddy
3 Replies

4. Shell Programming and Scripting

Add unique header to multiple lines

I have a file of lines with the following format: AACCCGTAGATCCGAACTTGTG ACCCGTAGATCCGAACTTGTG CCGTAGATCCGAACTTGTG CGTAGATCCGAACTTGT I want to give a header to each line, using awk, where the header is equal to the line that follows, like this: >AACCCGTAGATCCGAACTTGTG ... (2 Replies)
Discussion started by: Palgrave
2 Replies

5. Shell Programming and Scripting

How to add the multiple lines of xml tags before a particular xml tag in a file

Hi All, I'm stuck with adding multiple lines(irrespective of line number) to a file before a particular xml tag. Please help me. <A>testing_Location</A> <value>LA</value> <zone>US</zone> <B>Region</B> <value>Russia</value> <zone>Washington</zone> <C>Country</C>... (0 Replies)
Discussion started by: mjavalkar
0 Replies

6. Shell Programming and Scripting

search and replace, when found, delete multiple lines, add new set of lines?

hey guys, I tried searching but most 'search and replace' questions are related to one liners. Say I have a file to be replaced that has the following: $ cat testing.txt TESTING AAA BBB CCC DDD EEE FFF GGG HHH ENDTESTING This is the input file: (3 Replies)
Discussion started by: DeuceLee
3 Replies

7. Shell Programming and Scripting

Processing Multiple Files

Hello Everyone, I am new to scripting and confused with how to do this efficiently. I am trying to use AWK to do this. I have a lot of files in a folder which has the data of my throughput measurements in two columns i.e. Serial # and Throughput. like this 177.994 847.9 178.996 ... (1 Reply)
Discussion started by: hakim
1 Replies

8. Shell Programming and Scripting

PERL: add string to multiple lines

Dear all, I am stuck while trying to add a string to multiple lines. Let me try to explain using an example: Input: -------- myExample_send ("MasterSends", n, "Data Type", MPI_INT); correct Output:... (4 Replies)
Discussion started by: bonny
4 Replies

9. Shell Programming and Scripting

Add Multiple Lines in an existing file

Hi Unix Experts, I like to add the following multiple lines in an existing file: CHANNELS: MaxChannels=600 MaxActiveChannels=600 MaxInitiators=22 I would be highly appreciated if somebody can shed some light on it. Thank you in advance!!! Khan (5 Replies)
Discussion started by: hkhan12
5 Replies

10. UNIX for Dummies Questions & Answers

multiple file processing

I need to process a directory which will have a different amount of files in it from time to time. This is an error directory, I need to process each file indvidually in to one log file, then display the file to the user. So I would like to display the file name as well in the log file. example... (4 Replies)
Discussion started by: jagannatha
4 Replies
Login or Register to Ask a Question