Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Find duplicate words in first column between "10" repetiotions

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Find duplicate words in first column between "10" repetiotions
# 1  
Old 09-23-2014
Find duplicate words in first column between "10" repetiotions
hello

I have a file of the form:
Code:
 
nameA  number number ... 
nameB  number number ...
nameA  number number ...
nameC  number number ...
nameD  number number ...
                  10
nameA  number number ...
nameC  number number ...
nameB  number number ...
nameC  number number ...
nameD  number number ...
                  10
nameA  number number
...

and I want to detect only the duplicates of the first column in between the "10" and print the line.
the output should be
Code:
nameA  number number ...
nameC  number number ...

("namex" is a random name and "numberx" random number)
Thank you in advance
Last edited by phaethon; 09-23-2014 at 12:54 PM..
# 2  
Old 09-23-2014
something along these lines: awk -f pha.awk myFile where pha.awk is:
Code:
function pDup (){
   for (i in d)
     if (d[i]>1) print l[i]
   split("",d)
   split("",l)
}

$1==10 {pDup();next}
{ d[$1]++; l[$1]=$0}
END { pDup() }

Last edited by vgersh99; 09-23-2014 at 01:36 PM..
This User Gave Thanks to vgersh99 For This Post:
phaethon
# 3  
Old 09-23-2014
Thank you very much ! I have no words! it worked!
# 4  
Old 09-23-2014
Another awk approach:-
Code:
awk '$1==10{split("",A);next}++A[$1]>1' file

This User Gave Thanks to Yoda For This Post:
phaethon
# 5  
Old 09-24-2014
Quote:
Originally Posted by Yoda
Another awk approach:-
Code:
awk '$1==10{split("",A);next}++A[$1]>1' file

slight modification to Yoda's approach to not print duplicates for each set in the output
Code:
awk '$1==10{split("",A);next}++A[$1]==2' file

This User Gave Thanks to SriniShoo For This Post:
phaethon
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

find . -path "*_nobackup*" -prune -iname "*.PDF" \( ! -name "*_nobackup.*" \)

These three finds worked as expected: $ find . -iname "*.PDF" $ find . -iname "*.PDF" \( ! -name "*_nobackup.*" \) $ find . -path "*_nobackup*" -prune -iname "*.PDF" They all returned the match: ./folder/file.pdf :b: This find returned no matches: $ find . -path "*_nobackup*" -prune... (3 Replies)
Discussion started by: wolfv
3 Replies

2. UNIX for Dummies Questions & Answers

Replacing "." with "GG" in a certain column of a file that has heading

Hi, all, I have a file that looks like: ## XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX ## YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY #AA AB AC AD AE AF AG AH AI AJ AK AL 20 60039 60039 ... (5 Replies)
Discussion started by: kush
5 Replies

3. Shell Programming and Scripting

Find lines with "A" then change "E" to "X" same line

I have a bunch of random character lines like ABCEDFG. I want to find all lines with "A" and then change any "E" to "X" in the same line. ALL lines with "A" will have an "X" somewhere in it. I have tried sed awk and vi editor. I get close, not quite there. I know someone has already solved this... (10 Replies)
Discussion started by: nightwatchrenba
10 Replies

4. Shell Programming and Scripting

Substituting comma "," for dot "." in a specific column when comma"," is a delimiter

Hi, I'm dealing with an issue and losing a lot of hours figuring out how i would solve this. I have an input file which looks like this: ('BLABLA +200-GRS','Serviço ','TarifaçãoServiço','wap.bla.us.0000000121',2985,0,55,' de conversão em escada','Dia','Domingos') ('BLABLA +200-GRR','Serviço... (6 Replies)
Discussion started by: poliver
6 Replies

5. Shell Programming and Scripting

Using sed to find text between a "string " and character ","

Hello everyone Sorry I have to add another sed question. I am searching a log file and need only the first 2 occurances of text which comes after (note the space) "string " and before a ",". I have tried sed -n 's/.*string \(*\),.*/\1/p' filewith some, but limited success. This gives out all... (10 Replies)
Discussion started by: haggismn
10 Replies

6. AIX

"Frames" and "Words" in fcstat output

What are "Frames" and "Words" in the fcstat output? vio1:/home/padmin:# fcstat fcs0 <snip> Transmit Statistics Receive Statistics ------------------- ------------------ Frames: 122844229 363445456 Words: 50940091904 171210861568 <snip> The... (1 Reply)
Discussion started by: kah00na
1 Replies

7. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone... (7 Replies)
Discussion started by: shis100
7 Replies

8. Shell Programming and Scripting

"Join" or "Merge" more than 2 files into single output based on common key (column)

Hi All, I have working (Perl) code to combine 2 input files into a single output file using the join function that works to a point, but has the following limitations: 1. I am restrained to 2 input files only. 2. Only the "matched" fields are written out to the "matched" output file and... (1 Reply)
Discussion started by: Katabatic
1 Replies

9. Shell Programming and Scripting

"find command" to find the files in the current directories but not in the "subdir"

Dear friends, please tell me how to find the files which are existing in the current directory, but it sholud not search in the sub directories.. it is like this, current directory contains file1, file2, file3, dir1, dir2 and dir1 conatins file4, file5 and dir2 contains file6,... (9 Replies)
Discussion started by: swamymns
9 Replies

10. Shell Programming and Scripting

grep to find content in between curly braces, "{" and "},"

problem String ~~~~~~~~~~~~~~~~~~ icecream= { smart peopleLink "good" LC "happy" , smartpeopleLink "dull" LC "sad" } aend = {smart vc4 eatr kalu} output needed ~~~~~~~~~~~~~~~~~~ smart peopleLink "good" LC "happy" , smartpeopleLink "dull" LC "sad" smart vc4... (4 Replies)
Discussion started by: keshav_rk
4 Replies
Login or Register to Ask a Question