Sponsored Content
Top Forums Shell Programming and Scripting Remove very first pair of duplicate words Post 302612199 by ctsgnb on Sunday 25th of March 2012 06:46:43 AM
Old 03-25-2012
Also try

Code:
awk '{y=x;x=w;w=$1}(y==x||y==w){next}{print y}END{print x!=w?x RS w:w }'  infile

Just be aware that this code will not display the line in red because the same line appear in line n+2
Code:
                                                                                 [...]                                                                                      MMIT                                                                                  VAR_1D_DATA_TYPE                                                                                  15-03-2012                                                                                                                                                               														 MMIT                                                                                  VAR_10D_DATA_TYPE                                                                                      MMIT                                                                                  VAR_10D_DATA_TYPE
                                                                                 [...]


or
Code:
awk '{z=y;y=x;x=w;w=$1}(z==y||t){t=0;next}(z==x&&y==w){t=1;next}{print z}END{print (y!=x?y RS:z) (x!=w?x RS w:w)}' infile


Last edited by ctsgnb; 03-25-2012 at 11:34 AM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Duplicate words

Hi, can anyone help me with this small problem. Say i've got duplicate words in a list e.g blue orange green green pink blue red How can I delete all reoccurences of the same word? Thanks (1 Reply)
Discussion started by: zulander
1 Replies

2. Shell Programming and Scripting

remove duplicate

i have a text its contain many record, but its written in one line, i want to remove from that line the duplicate record, not record have fixed width ex: width = 4 inputfile test.txt =abc cdf abc abc cdf fgh fgh abc abc i want the outputfile =abc cdf fgh only those records can any one help... (4 Replies)
Discussion started by: kazanoova2
4 Replies

3. UNIX for Dummies Questions & Answers

Identify duplicate words in a line using command

Hi, Let me explain the problem clearly: Let the entries in my file be: lion,tiger,bear apple,mango,orange,apple,grape unix,windows,solaris,windows,linux red,blue,green,yellow orange,maroon,pink,violet,orange,pink Can we detect the lines in which one of the words(separated by field... (8 Replies)
Discussion started by: srinivasan_85
8 Replies

4. Shell Programming and Scripting

remove duplicate words in a line

Hi, Please help! I have a file having duplicate words in some line and I want to remove the duplicate words. The order of the words in the output file doesn't matter. INPUT_FILE pink_kite red_pen ball pink_kite ball yellow_flower white no white no cloud nine_pen pink cloud pink nine_pen... (6 Replies)
Discussion started by: sam_2921
6 Replies

5. Shell Programming and Scripting

Need to remove the words

Hi folks, I have file with the below 1245633505 +manual mroennfeldt@news.com.au 1245633506 +manual sal@bynews.com.au 1245633506 +manual whson@btimes.com 1245633507 +manual karla.marsden@tnews.com.au 1245633508 +manual king@netn.com.au Now, I need the output of the files only with... (4 Replies)
Discussion started by: gsiva
4 Replies

6. Shell Programming and Scripting

remove words

All, I have a file with below entries. /java/usr/abc/123 /java/usr/xyz/123_21 /java/usr/ab12/345/234 ......... ......... And I need entry as /java/usr/abc/config /java/usr/xyz/config /java/usr/ab12/config ......... ......... Actually, I need to remove all other entries... (2 Replies)
Discussion started by: anshu ranjan
2 Replies

7. Shell Programming and Scripting

Remove duplicate

Hi , I have a pipe seperated file repo.psv where i need to remove duplicates based on the 1st column only. Can anyone help with a Unix script ? Input: 15277105||Common Stick|ESHR||Common Stock|CYRO AB 15277105||Common Stick|ESHR||Common Stock|CYRO AB 16111278||Common Stick|ESHR||Common... (12 Replies)
Discussion started by: samrat dutta
12 Replies

8. UNIX for Advanced & Expert Users

Find duplicate words using sed

I have following statement and I want to find duplicate word using sed command. How is it possible? "detect string and remove the duplicate string" There could be many statements in a file and each line may have duplicate word. Thanks! (1 Reply)
Discussion started by: jnrohit2k
1 Replies

9. Shell Programming and Scripting

[All variants] remove first pair of parentheses

How to remove first pair of parentheses and content in them from the beginning of the line? Here's the list: (ok)-test (ok)-test-(ing) (some)-test-(ing)-test test-(ing) Desired result: test test-(ing) test-(ing)-test test-(ing) Here's what I already tried with GNU sed: sed -e... (6 Replies)
Discussion started by: useretail
6 Replies

10. UNIX for Dummies Questions & Answers

Remove duplicate words from column 1

Tried using sed and uniq but it's removing the entire line. Can't seem to figure a way to just remove the word. Any help is appreciated. I have a file: dog, text1, text2, text3 dog, text1, text2, text3 dog, text1, text2, text3 cat, text1, text2, text3 Trying to remove all duplicate instances... (6 Replies)
Discussion started by: jimmyf
6 Replies
Devel::Refcount(3pm)					User Contributed Perl Documentation				      Devel::Refcount(3pm)

NAME
"Devel::Refcount" - obtain the REFCNT value of a referent SYNOPSIS
use Devel::Refcount qw( refcount ); my $anon = []; print "Anon ARRAY $anon has " . refcount($anon) . " reference "; my $otherref = $anon; print "Anon ARRAY $anon now has " . refcount($anon) . " references "; DESCRIPTION
This module provides a single function which obtains the reference count of the object being pointed to by the passed reference value. FUNCTIONS
$count = refcount($ref) Returns the reference count of the object being pointed to by $ref. COMPARISON WITH SvREFCNT This function differs from "Devel::Peek::SvREFCNT" in that SvREFCNT() gives the reference count of the SV object itself that it is passed, whereas refcount() gives the count of the object being pointed to. This allows it to give the count of any referent (i.e. ARRAY, HASH, CODE, GLOB and Regexp types) as well. Consider the following example program: use Devel::Peek qw( SvREFCNT ); use Devel::Refcount qw( refcount ); sub printcount { my $name = shift; printf "%30s has SvREFCNT=%d, refcount=%d ", $name, SvREFCNT($_[0]), refcount($_[0]); } my $var = []; printcount 'Initially, $var', $var; my $othervar = $var; printcount 'Before CODE ref, $var', $var; printcount '$othervar', $othervar; my $code = sub { undef $var }; printcount 'After CODE ref, $var', $var; printcount '$othervar', $othervar; This produces the output Initially, $var has SvREFCNT=1, refcount=1 Before CODE ref, $var has SvREFCNT=1, refcount=2 $othervar has SvREFCNT=1, refcount=2 After CODE ref, $var has SvREFCNT=2, refcount=2 $othervar has SvREFCNT=1, refcount=2 Here, we see that SvREFCNT() counts the number of references to the SV object passed in as the scalar value - the $var or $othervar respectively, whereas refcount() counts the number of reference values that point to the referent object - the anonymous ARRAY in this case. Before the CODE reference is constructed, both $var and $othervar have SvREFCNT() of 1, as they exist only in the current lexical pad. The anonymous ARRAY has a refcount() of 2, because both $var and $othervar store a reference to it. After the CODE reference is constructed, the $var variable now has an SvREFCNT() of 2, because it also appears in the lexical pad for the new anonymous CODE block. PURE-PERL FALLBACK An XS implementation of this function is provided, and is used by default. If the XS library cannot be loaded, a fallback implementation in pure perl using the "B" module is used instead. This will behave identically, but is much slower. Rate pp xs pp 225985/s -- -66% xs 669570/s 196% -- SEE ALSO
o Test::Refcount - assert reference counts on objects AUTHOR
Paul Evans <leonerd@leonerd.org.uk> perl v5.14.2 2011-11-15 Devel::Refcount(3pm)
All times are GMT -4. The time now is 12:55 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy