Visit Our UNIX and Linux User Community


Search for string dublicates in column


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Search for string dublicates in column
# 8  
Old 02-19-2008
Quote:
Originally Posted by Lukeadams
Wow.... So the output is any line that appears more than once - but only printed once.

Can you explain what's going on here?
Two things first:

- in Awk uninitialized variables have value zero (or null, depending on the context)
- the '++' operator is used for adding one, it can be used to increment a variable either before or after taking its value.

Consider this:

Code:
$ print 'one
two
two
three
three
three'|awk '{printf "$0 is %s, first x[$0] is %s ,",$0,x[$0]++}{print "then x[$0] is",x[$0]}'
$0 is one, first x[$0] is 0 ,then x[$0] is 1
$0 is two, first x[$0] is 0 ,then x[$0] is 1
$0 is two, first x[$0] is 1 ,then x[$0] is 2
$0 is three, first x[$0] is 0 ,then x[$0] is 1
$0 is three, first x[$0] is 1 ,then x[$0] is 2
$0 is three, first x[$0] is 2 ,then x[$0] is 3

So now it should be easier to understand, this:

Code:
$ print 'one
two
two
three
three
three'|awk 'x[$0]++==1'
two
three

... and this:

Code:
$ print 'one
two
two
three
three
three'|awk '++x[$0]==2'
two
three


Previous Thread | Next Thread
Test Your Knowledge in Computers #30
Difficulty: Medium
Microsoft was founded in 1971.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to find string based on pattern and search for its corresponding rows in column

Experts, Need your support for this awk script. we have only one input file, all these column 1 and column 2 are in same file and have to do lookup for values in one file(column1 and column2) but output we need in another file Need to grep row whose string contains 9K from column 1. When found... (6 Replies)
Discussion started by: as7951
6 Replies

2. Shell Programming and Scripting

Search for string in column using variable: awk

I'm interested to match column pattern through awk using an external variable for data: -9 1:751343:T:A -9 0 T A 0.726 -5.408837e-03 9.576603e-03 7.967536e-01 5.722312e-01 -9 1:751756:T:C -9 0 T C 0.727 -5.360458e-03 9.579447e-03 7.966977e-01 5.757858e-01... (7 Replies)
Discussion started by: genome
7 Replies

3. Shell Programming and Scripting

Search string in multiple files and display column wise

I have 3 files. Each of those files have the same number of records, however certain records have different values. I would like to grep the field in ALL 3 files and display the output with only the differences in column wise and if possible line number File1 Name = Joe Age = 33... (3 Replies)
Discussion started by: sidnow
3 Replies

4. Shell Programming and Scripting

How to search and replace string from nth column from a file?

I wanted to search for a string and replace it with other string from nth column of a file which is comma seperated which I am able to do with below # For Comma seperated file without quotes awk 'BEGIN{OFS=FS=","}$"'"$ColumnNo"'"=="'"$PPK"'"{$"'"$ColumnNo"'"="'"$NPK"'"}{print}' ${FileName} ... (5 Replies)
Discussion started by: Amit Joshi
5 Replies

5. Shell Programming and Scripting

How to search and replace string in column in file with command sed?

how to search and replace string in column in file with command sed or other search "INC0000003.in" and replace column 4 = "W" $ cat file.txt INC0000001.in|20150120|Y|N|N INC0000002.in|20150120|Y|N|N INC0000003.in|20150120|Y|N|N INC0000004.in|20150120|Y|N|Noutput... (4 Replies)
Discussion started by: ppmanja3
4 Replies

6. UNIX for Advanced & Expert Users

Recursively search the string from a column in no. of files

i have a file named keyword.csv(contains around 8k records) which contains a no. of columns. The 5th column contains all the keywords. I want to recursively search these keywords in all .pl files(around 1k) and display the filename....Afterthat i will use the filename and some of the column from... (3 Replies)
Discussion started by: millan
3 Replies

7. Shell Programming and Scripting

Search several string and convert into a single line for each search string using awk command AIX?.

I need to search the file using strings "Request Type" , " Request Method" , "Response Type" and by using result set find the xml tags and convert into a single line?. below are the scenarios. Cat test Nov 10, 2012 5:17:53 AM INFO: Request Type Line 1.... (5 Replies)
Discussion started by: laknar
5 Replies

8. UNIX for Dummies Questions & Answers

Search and replace string only in a particular column in a delimited file

I have file with multiple columns. Column values for a record may be same. Now i have to replace a column value(this can be same for the other columns) with new value. File.txt A,B,C,D,A,B,C,D,A,B,C,D A,B,C,D,A,B,C,D,A,B,C,D A,B,C,D,A,B,C,D,A,B,C,D A,B,C,D,A,B,C,D,A,B,C,D... (1 Reply)
Discussion started by: ksailesh
1 Replies

9. Shell Programming and Scripting

Search in a column by a string

Hi All, My file looks like : hsdhj dsajhf jshdfajkh jksdhfj jkdhsfj shfjhd shdf hdsfjkh jsdfhj hdshf sdjh dhs foot dsjhfj jdshf dasfh jdsh dsjfh jdfshj david Now, I want to search entire column by a string... (10 Replies)
Discussion started by: naw_deepak
10 Replies

10. Shell Programming and Scripting

String search and return value from column

Dear All I had below mention file as my input file. 87980457 Jan 12 2008 2:00AM 1 60 BSC1 81164713 Jan 12 2008 3:00AM 1 60 BSC2 78084521 Jan 12 2008 4:00AM 1 60 BSC3 68385193... (3 Replies)
Discussion started by: jaydeep_sadaria
3 Replies

Featured Tech Videos