Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Help with file processing using awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help with file processing using awk
# 1  
Old 03-05-2016
Error Help with file processing using awk
hello All, I'm new to AWK programming and learned myself few things to process a file and deal with duplicate lines, but I got into a scenario which makes me clueless to handle. Here is the scenario..

Input file:

Code:
user  role
-----  ----
AAA  add
AAA  delete
BBB  delete
CCC  delete
DDD  add
BBB  add

Expected output:
Code:
user role
----  -----
AAA  add
BBB  add
CCC  delete
DDD  add

As per my expected output, if the same user has 2 roles (add & delete) only "add" should be preferred for that user and to be printed ignoring the delete row, but if a user has just one role (add / delete) it can be printed as such. note: line sequence can be in any order.

Thanks for guiding me the rite way to approach!!
Last edited by Scrutinizer; 03-05-2016 at 03:08 AM.. Reason: detailed explanation
# 2  
Old 03-05-2016
Hi, try:
Code:
awk '!($1 in A) || $2=="add" {A[$1]=$2} END{for(i in A) print i, A[i]}' file

or if
Code:
user  role
-----  ----

is part of the file:
Code:
awk 'FNR<3{print; next} !($1 in A) || $2=="add" {A[$1]=$2} END{for(i in A) print i, A[i]}' file

Last edited by Scrutinizer; 03-05-2016 at 03:23 AM..
These 2 Users Gave Thanks to Scrutinizer For This Post:
julearn looney
# 3  
Old 03-05-2016
Thank you!, appreciate... it works like a charm.
But can you let me know how to print two more columns accordingly,
cos when I tried with printing $3 and $4 it always printed the final column value

<<ignore the header>>
Code:
user role  location eid
-------------------------
AAA  add  UK  1
AAA  delete  US  4
BBB  delete  CA  5
CCC  delete  AU  2
DDD  add  FR  3
BBB  add  IN  4

expected output :
Code:
user role  location  eid
-------------------------
AAA  add  UK  1
BBB  add  IN  4
CCC  delete  AU  2
DDD  add  FR  3

Last edited by Don Cragun; 03-05-2016 at 05:09 PM.. Reason: Add CODE tags.
# 4  
Old 03-05-2016
Store the entire line, $0, in the array. Because the key $1 is part of the $0, one does not need to print it at the END.
Code:
awk '!($1 in A) || $2=="add" {A[$1]=$0} END{for(i in A) print A[i]}' file

This User Gave Thanks to MadeInGermany For This Post:
julearn
# 5  
Old 03-05-2016
Does it have to be entirely in awk? Try
Code:
{ read A; read B; printf "%s\n" "$A" "$B";  sort; } < file | awk '!T[$1]++'
user role location eid
-------------------------
AAA add UK 1
BBB add IN 4
CCC delete AU 2
DDD add FR 3

This User Gave Thanks to RudiC For This Post:
looney
# 6  
Old 03-05-2016
sort is a nice idea - add comes before delete when sorted.
If the key field is precisely specified , even sort -u should work
Code:
sort -k 1,1 -u file

Cannot test it right now...might need the second key field to be added...
# 7  
Old 03-05-2016
Code:
{ read A; read B; printf "%s\n" "$A" "$B";  sort; } < file | awk '!T[$1]++'

Hi RudiC,

Could you please explain the code and the flow.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Processing a formatted file with awk

Hi - I want to interrogate information about my poker hands, sessions are all recorded in a text file in a particular format. Each hand starts with the string <PokerStars> followed by a unique hand reference and other data like date/time. There is then all the information about each hand. My first... (5 Replies)
Discussion started by: rbeech23
5 Replies

2. UNIX for Dummies Questions & Answers

awk - Rename output file, after processing, same as input file

I have one input file ABC.txt and one output DEF.txt. After the ABC is processed and created output, I want to rename ABC.txt to ABC.orig and DEF to ABC.txt. Currently when I am doing this, it does not process the input file as it cannot read and write to the same file. How can I achieve this? ... (12 Replies)
Discussion started by: High-T
12 Replies

3. Programming

awk processing / Shell Script Processing to remove columns text file

Hello, I extracted a list of files in a directory with the command ls . However this is not my computer, so the ls functionality has been revamped so that it gives the filesizes in front like this : This is the output of ls command : I stored the output in a file filelist 1.1M... (5 Replies)
Discussion started by: ajayram
5 Replies

4. Shell Programming and Scripting

Help with File Processing (AWK)

Input File: 1234, 2345,abc 1,24141,gw 222,rff,sds 2232145,sdsd,121 Output file to be generated: 000001234,2345,abc 000000001,24141,gw 000000222,rff,sds 002232145,sdsd,121 i.e; the first column is padded to get 9 digits. I tried with following: (3 Replies)
Discussion started by: karumudi7
3 Replies

5. Shell Programming and Scripting

Help with File Processing (AWK)

Input File: 1234, 2345,abc 1,24141,gw 222,rff,sds 2232145,sdsd,121 Output file to be generated: 000001234,2345,abc 000000001,24141,gw 000000222,rff,sds 002232145,sdsd,121 i.e; the first column is padded to get 9 digits. I tried with following: (1 Reply)
Discussion started by: karumudi7
1 Replies

6. Programming

AWK processing of a three-column file

I have a 3-column data file, for which I wish to print certain parts of $3 PHI PSI A(x) -177.5 -177.5 1.0625 -177.5 -172.5 0.55 -177.5 -167.5 0.0478125 -177.5 -162.5 0 -177.5 -157.5 0.284375 -177.5 -152.5 0.187188 -177.5 -147.5 0.236875 -177.5 -142.5 0.383438 -177.5 ... (3 Replies)
Discussion started by: chrisjorg
3 Replies

7. Shell Programming and Scripting

awk help in processing file.

I am trying to process file which has following data #23456789012345 ACNASPSA13N0N0 ACNAPCPA05N0N0 ACNAFATS11N0N0 I want to take out each line from the file and what to put in the file by name which if part of the line starting from offset 10 to 15. It means I want to create three file... (3 Replies)
Discussion started by: ekb
3 Replies

8. UNIX for Dummies Questions & Answers

poor performance processing file with awk

Hello, I'm running a script on AIX to process lines in a file. I need to enclose the second column in quotation marks and write each line to a new file. I've come up with the following: #!/bin/ksh filename=$1 exec >> $filename.new cat $filename | while read LINE do echo $LINE | awk... (2 Replies)
Discussion started by: scooter53080
2 Replies

9. Shell Programming and Scripting

how to change the current file processing to some other random file in awk ?

Hello, say suppose i am processing an file emp.dat the field of which are deptno empno empname etc now say suppose i want to change the file to emp.lst then how can i do it? Here i what i attempted but in vain BEGIN{ system("sort emp.dat > emp.lst") FILENAME="emp.lst" } { print... (2 Replies)
Discussion started by: salman4u
2 Replies

10. Shell Programming and Scripting

processing a file with sed and awk

Hello, I have what is probably a simple task in text manipulation, but I just can't wrap my brain around it. I have a text file that looks something like the following. Note that some have middle initials in the first field and some don't. john.r.smith:john.smith@yahoo.com... (4 Replies)
Discussion started by: manouche
4 Replies
Login or Register to Ask a Question