Sponsored Content
Top Forums Shell Programming and Scripting Nawk script to compare records of a file based on a particular column. Post 302550215 by usha rao on Thursday 25th of August 2011 03:51:37 AM
Old 08-25-2011
Power Nawk script to compare records of a file based on a particular column.

Hi Gurus,

I am struggling with nawk command where i am processing a file based on columns.

Here is the sample data file.

Code:
 
UM113570248|24-AUG-11|4|man1|RR211 Alert: Master Process failure |24-AUG-11
UM113570624|24-AUG-11|4|man1| Alert: Pattern 'E_DCLeDAOException' found |24-AUG-11
UM113570248|24-AUG-11|4|man1|RR231:  Alert: Master Process failure |
UM113569599|24-AUG-11|4|man1|RRWEB413 ChartsDCC RoadRunner App|OLTX_Iote6|24-AUG-11


In this file, the first column is the primary key.But there are some duplicate entries in first column.

in this case :

Code:
 
UM113570248|24-AUG-11|4|man1|RR211 Alert: Master Process failure |24-AUG-11
UM113570248|24-AUG-11|4|man1|RR231:  Alert: Master Process failure |

Now my requirement is..
I want to search the file according to the first column and if there are any duplicate entries in first column then i will search the 6th column and print that row in which the 6th column is not empty.

My desired output is as below:

Code:
 
UM113570248|24-AUG-11|4|man1|RR211 Alert: Master Process failure |24-AUG-11
UM113570624|24-AUG-11|4|man1| Alert: Pattern 'E_DCLeDAOException' found |24-AUG-11
UM113569599|24-AUG-11|4|man1|RRWEB413 ChartsDCC RoadRunner App|OLTX_Iote6|24-AUG-11

I have till now tried this script but somehow its not giving me the desired output.

Code:
 
cat testingdata.txt | nawk -F"|" '{ if (!a[$1]) {a[$1]; if ( $6!= "" ) print $0 > "out1" ; else exit }}'

Can anyone help me understand what i am doing wrong in my awk script.

Thanks in Advance
Usha
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Select records based on search criteria on first column

Hi All, I need to select only those records having a non zero record in the first column of a comma delimited file. Suppose my input file is having data like: "0","01/08/2005 07:11:15",1,1,"Created",,"01/08/2005" "0","01/08/2005 07:12:40",1,1,"Created",,"01/08/2005"... (2 Replies)
Discussion started by: shashi_kiran_v
2 Replies

2. UNIX for Dummies Questions & Answers

Filtering records of a file based on a value of a column

Hi all, I would like to extract records of a file based on a condition. The file contains 47 fields, and I would like to extract only those records that match a certain value in one of the columns, e.g. COL1 COL2 COL3 ............... COL47 1 XX 45 ... (4 Replies)
Discussion started by: risk_sly
4 Replies

3. Shell Programming and Scripting

Compare files column to column based on keys

Here is my situation. I need to compare two tab separated files (diff is not useful since there could be known difference between files). I have found similar posts , but not fully matching.I was thinking of writing a shell script using cut and grep and while loop but after going thru posts it... (2 Replies)
Discussion started by: blackjack101
2 Replies

4. Shell Programming and Scripting

Removing duplicate records in a file based on single column

Hi, I want to remove duplicate records including the first line based on column1. For example inputfile(filer.txt): ------------- 1,3000,5000 1,4000,6000 2,4000,600 2,5000,700 3,60000,4000 4,7000,7777 5,999,8888 expected output: ---------------- 3,60000,4000 4,7000,7777... (5 Replies)
Discussion started by: G.K.K
5 Replies

5. Emergency UNIX and Linux Support

[Solved] Extract records based on a repeated column value

Hi guys, I need help in making a command to find some data. I have multiple files in which multiple records are present.. Each record is separated with a carriage return and in each record there are multiple fields with each field separated by "|" what i want is that I want to extract... (1 Reply)
Discussion started by: m_usmanayub
1 Replies

6. Shell Programming and Scripting

Removing duplicate records in a file based on single column explanation

I was reading this thread. It looks like a simpler way to say this is to only keep uniq lines based on field or column 1. https://www.unix.com/shell-programming-scripting/165717-removing-duplicate-records-file-based-single-column.html Can someone explain this command please? How are there no... (5 Replies)
Discussion started by: cokedude
5 Replies

7. Shell Programming and Scripting

Compare based on column value

Hi Experts, I want to compare 2 text files based on their column values text1 is like prd-1234 yes no yes yes prd-2345 no no no yes prd-6475 yes yes yes no and test 2 is prd-1234 no no no yes prd-2345 yes no no no desired out put as follows prd-1234 1 3 prd-235 1 4 basically it shows... (5 Replies)
Discussion started by: tijomonmathew
5 Replies

8. Shell Programming and Scripting

Compare two files based on column

Hi, I have two files roughly 1200 fields in length for each row, sorted on the 2nd field. I need to compare based on that 2nd column between file1 and file2 and print lines that exist in both files into separate files (I can't guarantee that every line in file1 is in file2). Example: File1: ... (1 Reply)
Discussion started by: origon
1 Replies

9. Shell Programming and Scripting

Compare two csv's with column based

Hi, I am having below two CSV's col_1,col_2,col_3 1,2,4 1,3,6 col_1,col_3,col2,col_5,col_6 1,2,3,4,5 1,6,3,,, I need to compare based on the columns where the mismatch is expected output col_1,col_2,col_3 1,2,4 (3 Replies)
Discussion started by: rohit_shinez
3 Replies

10. UNIX for Beginners Questions & Answers

Filtering records of a csv file based on a value of a column

Hi, I tried filtering the records in a csv file using "awk" command listed below. awk -F"~" '$4 ~ /Active/{print }' inputfile > outputfile The output always has all the entries. The same command worked for different users from one of the forum links. content of file I was... (3 Replies)
Discussion started by: sunilmudikonda
3 Replies
ucblinks(1B)					     SunOS/BSD Compatibility Package Commands					      ucblinks(1B)

NAME
ucblinks - adds /dev entries to give SunOS 4.x compatible names to SunOS 5.x devices SYNOPSIS
/usr/ucb/ucblinks [-e rulebase] [-r rootdir] DESCRIPTION
ucblinks creates symbolic links under the /dev directory for devices whose SunOS 5.x names differ from their SunOS 4.x names. Where possi- ble, these symbolic links point to the device's SunOS 5.x name rather than to the actual /devices entry. ucblinks does not remove unneeded compatibility links; these must be removed by hand. ucblinks should be called each time the system is reconfiguration-booted, after any new SunOS 5.x links that are needed have been created, since the reconfiguration may have resulted in more compatibility names being needed. In releases prior to SunOS 5.4, ucblinks used a nawk rule-base to construct the SunOS 4.x compatible names. ucblinks no longer uses nawk for the default operation, although nawk rule-bases can still be specifed with the -e option. The nawk rule-base equivalent to the SunOS 5.4 default operation can be found in /usr/ucblib/ucblinks.awk. OPTIONS
-e rulebase Specify rulebase as the file containing nawk(1) pattern-action statements. -r rootdir Specify rootdir as the directory under which dev and devices will be found, rather than the standard root directory /. FILES
/usr/ucblib/ucblinks.awk sample rule-base for compatibility links ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWscpu | +-----------------------------+-----------------------------+ SEE ALSO
devlinks(1M), disks(1M), ports(1M), tapes(1M), attributes(5) SunOS 5.11 13 Apr 1994 ucblinks(1B)
All times are GMT -4. The time now is 12:44 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy