Sponsored Content
Top Forums Shell Programming and Scripting Nawk script to compare records of a file based on a particular column. Post 302550215 by usha rao on Thursday 25th of August 2011 03:51:37 AM
Old 08-25-2011
Power Nawk script to compare records of a file based on a particular column.

Hi Gurus,

I am struggling with nawk command where i am processing a file based on columns.

Here is the sample data file.

Code:
 
UM113570248|24-AUG-11|4|man1|RR211 Alert: Master Process failure |24-AUG-11
UM113570624|24-AUG-11|4|man1| Alert: Pattern 'E_DCLeDAOException' found |24-AUG-11
UM113570248|24-AUG-11|4|man1|RR231:  Alert: Master Process failure |
UM113569599|24-AUG-11|4|man1|RRWEB413 ChartsDCC RoadRunner App|OLTX_Iote6|24-AUG-11


In this file, the first column is the primary key.But there are some duplicate entries in first column.

in this case :

Code:
 
UM113570248|24-AUG-11|4|man1|RR211 Alert: Master Process failure |24-AUG-11
UM113570248|24-AUG-11|4|man1|RR231:  Alert: Master Process failure |

Now my requirement is..
I want to search the file according to the first column and if there are any duplicate entries in first column then i will search the 6th column and print that row in which the 6th column is not empty.

My desired output is as below:

Code:
 
UM113570248|24-AUG-11|4|man1|RR211 Alert: Master Process failure |24-AUG-11
UM113570624|24-AUG-11|4|man1| Alert: Pattern 'E_DCLeDAOException' found |24-AUG-11
UM113569599|24-AUG-11|4|man1|RRWEB413 ChartsDCC RoadRunner App|OLTX_Iote6|24-AUG-11

I have till now tried this script but somehow its not giving me the desired output.

Code:
 
cat testingdata.txt | nawk -F"|" '{ if (!a[$1]) {a[$1]; if ( $6!= "" ) print $0 > "out1" ; else exit }}'

Can anyone help me understand what i am doing wrong in my awk script.

Thanks in Advance
Usha
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Select records based on search criteria on first column

Hi All, I need to select only those records having a non zero record in the first column of a comma delimited file. Suppose my input file is having data like: "0","01/08/2005 07:11:15",1,1,"Created",,"01/08/2005" "0","01/08/2005 07:12:40",1,1,"Created",,"01/08/2005"... (2 Replies)
Discussion started by: shashi_kiran_v
2 Replies

2. UNIX for Dummies Questions & Answers

Filtering records of a file based on a value of a column

Hi all, I would like to extract records of a file based on a condition. The file contains 47 fields, and I would like to extract only those records that match a certain value in one of the columns, e.g. COL1 COL2 COL3 ............... COL47 1 XX 45 ... (4 Replies)
Discussion started by: risk_sly
4 Replies

3. Shell Programming and Scripting

Compare files column to column based on keys

Here is my situation. I need to compare two tab separated files (diff is not useful since there could be known difference between files). I have found similar posts , but not fully matching.I was thinking of writing a shell script using cut and grep and while loop but after going thru posts it... (2 Replies)
Discussion started by: blackjack101
2 Replies

4. Shell Programming and Scripting

Removing duplicate records in a file based on single column

Hi, I want to remove duplicate records including the first line based on column1. For example inputfile(filer.txt): ------------- 1,3000,5000 1,4000,6000 2,4000,600 2,5000,700 3,60000,4000 4,7000,7777 5,999,8888 expected output: ---------------- 3,60000,4000 4,7000,7777... (5 Replies)
Discussion started by: G.K.K
5 Replies

5. Emergency UNIX and Linux Support

[Solved] Extract records based on a repeated column value

Hi guys, I need help in making a command to find some data. I have multiple files in which multiple records are present.. Each record is separated with a carriage return and in each record there are multiple fields with each field separated by "|" what i want is that I want to extract... (1 Reply)
Discussion started by: m_usmanayub
1 Replies

6. Shell Programming and Scripting

Removing duplicate records in a file based on single column explanation

I was reading this thread. It looks like a simpler way to say this is to only keep uniq lines based on field or column 1. https://www.unix.com/shell-programming-scripting/165717-removing-duplicate-records-file-based-single-column.html Can someone explain this command please? How are there no... (5 Replies)
Discussion started by: cokedude
5 Replies

7. Shell Programming and Scripting

Compare based on column value

Hi Experts, I want to compare 2 text files based on their column values text1 is like prd-1234 yes no yes yes prd-2345 no no no yes prd-6475 yes yes yes no and test 2 is prd-1234 no no no yes prd-2345 yes no no no desired out put as follows prd-1234 1 3 prd-235 1 4 basically it shows... (5 Replies)
Discussion started by: tijomonmathew
5 Replies

8. Shell Programming and Scripting

Compare two files based on column

Hi, I have two files roughly 1200 fields in length for each row, sorted on the 2nd field. I need to compare based on that 2nd column between file1 and file2 and print lines that exist in both files into separate files (I can't guarantee that every line in file1 is in file2). Example: File1: ... (1 Reply)
Discussion started by: origon
1 Replies

9. Shell Programming and Scripting

Compare two csv's with column based

Hi, I am having below two CSV's col_1,col_2,col_3 1,2,4 1,3,6 col_1,col_3,col2,col_5,col_6 1,2,3,4,5 1,6,3,,, I need to compare based on the columns where the mismatch is expected output col_1,col_2,col_3 1,2,4 (3 Replies)
Discussion started by: rohit_shinez
3 Replies

10. UNIX for Beginners Questions & Answers

Filtering records of a csv file based on a value of a column

Hi, I tried filtering the records in a csv file using "awk" command listed below. awk -F"~" '$4 ~ /Active/{print }' inputfile > outputfile The output always has all the entries. The same command worked for different users from one of the forum links. content of file I was... (3 Replies)
Discussion started by: sunilmudikonda
3 Replies
DH_LINK(1)							     Debhelper								DH_LINK(1)

NAME
dh_link - create symlinks in package build directories SYNOPSIS
dh_link [debhelperoptions] [-A] [-Xitem] [sourcedestination...] DESCRIPTION
dh_link is a debhelper program that creates symlinks in package build directories. dh_link accepts a list of pairs of source and destination files. The source files are the already existing files that will be symlinked from (called target by ln(1)). The destination files are the symlinks that will be created (called link name by ln(1)). There must be an equal number of source and destination files specified. Be sure you do specify the absolute path to both the source and destination files (unlike you would do if you were using something like ln(1)). Please note that the leading slash is optional. dh_link will generate symlinks that comply with Debian policy - absolute when policy says they should be absolute, and relative links with as short a path as possible. It will also create any subdirectories it needs to put the symlinks in. Any pre-existing destination files will be replaced with symlinks. dh_link also scans the package build tree for existing symlinks which do not conform to Debian policy, and corrects them (v4 or later). FILES
debian/package.links Lists pairs of source and destination files to be symlinked. Each pair should be put on its own line, with the source and destination separated by whitespace. In each pair the source file (called target by ln(1)) comes first and is followed by the destination file (called link name by ln(1)). Thus the pairs of source and destination files in each line are give in the same order as they would be given to ln(1). In contrast to ln(1), source and destination paths must be absolute (the leading slash is optional). OPTIONS
-A, --all Create any links specified by command line parameters in ALL packages acted on, not just the first. -Xitem, --exclude=item Exclude symlinks that contain item anywhere in their filename from being corrected to comply with Debian policy. source destination ... Create a file named destination as a link to a file named source. Do this in the package build directory of the first package acted on. (Or in all packages if -A is specified.) EXAMPLES
dh_link usr/share/man/man1/foo.1 usr/share/man/man1/bar.1 Make bar.1 be a symlink to foo.1 dh_link var/lib/foo usr/lib/foo usr/share/man/man1/foo.1 usr/share/man/man1/bar.1 Make /usr/lib/foo/ be a link to /var/lib/foo/, and bar.1 be a symlink to the foo.1 var/lib/foo usr/lib/foo usr/share/man/man1/foo.1 usr/share/man/man1/bar.1 Same as above but as content for a debian/package.links file. SEE ALSO
debhelper(7) This program is a part of debhelper. AUTHOR
Joey Hess <joeyh@debian.org> 11.1.6ubuntu2 2018-05-10 DH_LINK(1)
All times are GMT -4. The time now is 08:06 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy