Sponsored Content
Top Forums UNIX for Dummies Questions & Answers removing duplicates of a pattern from a file Post 302232223 by dennis.jacob on Thursday 4th of September 2008 06:25:32 AM
Old 09-04-2008
Quote:
Originally Posted by era
Code:
perl -pe 'BEGIN { $i = 1 } ++$i while (s/PATTERN/NONESVCH_$i/); s/NONESVCH/PATTERN/g' file

The indexing will be zero-based if you take out the BEGIN part.
the same with awk:

Code:
awk 'BEGIN {cnt=1} { while($0 ~ /PATTERN/) sub(/PATTERN/,"xxxxx"cnt++, $0); gsub(/xxxxx/,"PATTERN_",$0); print; } ' filename

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

removing duplicates from a file

i have a file with some 1000 entries it will contain entries like 1000,ram 2000,pankaj 1001,rahim 1000,ram 2532,govind 2000,pankaj 3000,venkat 2532,govind what i want is i want to extract only the distinct rows from this file so my output should contain only 1000,ram... (2 Replies)
Discussion started by: trichyselva
2 Replies

2. Shell Programming and Scripting

Removing duplicates in a sorted file by field.

I have data like this: It's sorted by the 2nd field (TID). envoy,90000000000000634600010001,04/11/2008,23:19:27,RB00266,0015,DETAIL,ERROR, envoy,90000000000000634600010001,04/12/2008,04:23:45,RB00266,0015,DETAIL,ERROR,... (1 Reply)
Discussion started by: kinksville
1 Replies

3. Shell Programming and Scripting

Removing duplicates from log file?

I have a log file with posts looking like this: -- Messages can be delivered by different systems at different times. The id number is used to sort out duplicate messages. What I need is to strip the arrival time from each post, sort posts by id number, and reattach arrival time to respective... (2 Replies)
Discussion started by: Ilja
2 Replies

4. Shell Programming and Scripting

Removing Duplicates from file

Hi Experts, Please check the following new requirement. I got data like the following in a file. FILE_HEADER 01cbbfde7898410| 3477945| home| 1 01cbc275d2c122| 3478234| WORK| 1 01cbbe4362743da| 3496386| Rich Spare| 1 01cbc275d2c122| 3478234| WORK| 1 This is pipe separated file with... (3 Replies)
Discussion started by: tinufarid
3 Replies

5. Shell Programming and Scripting

formatting a file and removing duplicates

Hi, I have a file that I want to change the format of. It is a large file in rows but I want it to be comma separated (comma then a space). The current file looks like this: HI, Joe, Bob, Jack, Jack After I would want to remove any duplicates so it would look like this: HI, Joe,... (2 Replies)
Discussion started by: kylle345
2 Replies

6. Shell Programming and Scripting

Removing duplicates depending on file size

Hi all, I am working with a huge amount of files in a Linux environment and I was trying to filter my data. Here's what my data looks like Name............................Size OLUSDN.gf.gif-1.JPEG.......5 kb LKJFDA01.gf.gif-1.JPEG.....3 kb LKJFDA01.gf.gif-2.JPEG.....1 kb... (7 Replies)
Discussion started by: Error404
7 Replies

7. UNIX for Dummies Questions & Answers

Removing duplicates from a file

Hi All, I am merging files coming from 2 different systems ,while doing that I am getting duplicates entries in the merged file I,01,000131,764,2,4.00 I,01,000131,765,2,4.00 I,01,000131,772,2,4.00 I,01,000131,773,2,4.00 I,01,000168,762,2,2.00 I,01,000168,763,2,2.00... (5 Replies)
Discussion started by: Sri3001
5 Replies

8. UNIX for Dummies Questions & Answers

Grep from pattern file without removing duplicates?

I have been using grep to output whole lines using a pattern file with identifiers (fileA): fig|562.2322.peg.1 fig|562.2322.peg.3 fig|562.2322.peg.3 fig|562.2322.peg.3 fig|562.2322.peg.7 From fileB with corresponding identifiers in the second column: NODE_0 fig|562.2322.peg.1 peg ... (2 Replies)
Discussion started by: Mauve
2 Replies

9. Shell Programming and Scripting

Removing duplicates from new file

i hav two files like i want to remove/delete all the duplicate lines in file2 which are viz unix,unix2,unix3 (2 Replies)
Discussion started by: sagar_1986
2 Replies

10. Shell Programming and Scripting

Removing duplicates from new file

i hav two files like i want to remove/delete all the duplicate lines in file2 which are viz unix,unix2,unix3.I have tried previous post also,but in that complete line must be similar.In this case i have to verify first column only regardless what is the content in succeeding columns. (3 Replies)
Discussion started by: sagar_1986
3 Replies
RAKE(1) 						 Ruby Programmers Reference Guide						   RAKE(1)

NAME
rake -- Ruby Make SYNOPSIS
rake [--f Rakefile] [--version] [-CGNPgnqstv] [-D [PATTERN]] [-E CODE] [-I LIBDIR] [-R RAKELIBDIR] [-T [PATTERN]] [-e CODE] [-p CODE] [-r MODULE] [--rules] [variable=value] target ... DESCRIPTION
Rake is a simple ruby(1) build program with capabilities similar to the regular make(1) command. Rake has the following features: o Rakefiles (Rake's version of Makefiles) are completely defined in standard Ruby syntax. No XML files to edit. No quirky Makefile syntax to worry about (is that a tab or a space?). o Users can specify tasks with prerequisites. o Rake supports rule patterns to synthesize implicit tasks. o Flexible FileLists that act like arrays but know about manipulating file names and paths. o A library of prepackaged tasks to make building rakefiles easier. OPTIONS
--version Display the program version. -C --classic-namespace Put Task and FileTask in the top level namespace -D [PATTERN] --describe [PATTERN] Describe the tasks (matching optional PATTERN), then exit. -E CODE --execute-continue CODE Execute some Ruby code, then continue with normal task processing. -G --no-system --nosystem Use standard project Rakefile search paths, ignore system wide rakefiles. -I LIBDIR --libdir LIBDIR Include LIBDIR in the search path for required modules. -N --no-search --nosearch Do not search parent directories for the Rakefile. -P --prereqs Display the tasks and dependencies, then exit. -R RAKELIBDIR --rakelib RAKELIBDIR --rakelibdir RAKELIBDIR Auto-import any .rake files in RAKELIBDIR. (default is rakelib ) -T [PATTERN] --tasks [PATTERN] Display the tasks (matching optional PATTERN) with descriptions, then exit. -e CODE --execute CODE Execute some Ruby code and exit. -f FILE --rakefile FILE Use FILE as the rakefile. -h --help Prints a summary of options. -g --system Using system wide (global) rakefiles (usually ~/.rake/*.rake ). -n --dry-run Do a dry run without executing actions. -p CODE --execute-print CODE Execute some Ruby code, print the result, then exit. -q --quiet Do not log messages to standard output. -r MODULE --require MODULE Require MODULE before executing rakefile. -s --silent Like --quiet, but also suppresses the 'in directory' announcement. -t --trace Turn on invoke/execute tracing, enable full backtrace. -v --verbose Log message to standard output (default). --rules Trace the rules resolution. SEE ALSO
ruby(1) make(1) http://rake.rubyforge.org/ REPORTING BUGS
Bugs, features requests and other issues can be logged at <http://onestepback.org/redmine/projects/show/rake>. You will need an account to before you can post issues. Register at <http://onestepback.org/redmine/account/register>. Or you can send an email to the author. AUTHOR
Rake is written by Jim Weirich <jim@weirichhouse.org> UNIX
November 7, 2012 UNIX
All times are GMT -4. The time now is 05:50 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy