Sponsored Content
Full Discussion: Remove duplicate ???
Top Forums Shell Programming and Scripting Remove duplicate ??? Post 101678 by futurelet on Friday 10th of March 2006 05:56:43 PM
Old 03-10-2006
Ruby:
Code:
gets(nil).scan( /^(.*?)\|(.*?)\|(.*\n(?:\1.*\n)*)/ ){|b|
  b.last.gsub!( /^.*(?=\|)/,"" ).gsub!( /\n/,"" )
  puts b.join(",")
}

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

remove duplicate

i have a text its contain many record, but its written in one line, i want to remove from that line the duplicate record, not record have fixed width ex: width = 4 inputfile test.txt =abc cdf abc abc cdf fgh fgh abc abc i want the outputfile =abc cdf fgh only those records can any one help... (4 Replies)
Discussion started by: kazanoova2
4 Replies

2. Shell Programming and Scripting

Remove duplicate

Hi all, I have a text file fileA.txt DXRV|02/28/2006 11:36:49.049|SAC||||CDxAcct=2420991350 DXRV|02/28/2006 11:37:06.404|SAC||||CDxAcct=6070970034 DXRV|02/28/2006 11:37:25.740|SAC||||CDxAcct=2420991350 DXRV|02/28/2006 11:38:32.633|SAC||||CDxAcct=6070970034 DXRV|02/28/2006... (2 Replies)
Discussion started by: sabercats
2 Replies

3. UNIX for Dummies Questions & Answers

Remove duplicate in array

Hi, I have a list of numbers stored in an array as below. 5 7 10 30 30 40 50 Please advise how could I remove the duplicate value in the array ? Thanks in advance. (5 Replies)
Discussion started by: Rock
5 Replies

4. Shell Programming and Scripting

remove duplicate

Hi, I am tryung to use shell or perl to remove duplicate characters for example , if I have " I love google" it will become I love ggle" or even "I loveggle" if removing duplicate white space Thanks CC (6 Replies)
Discussion started by: ccp
6 Replies

5. Shell Programming and Scripting

Help with remove duplicate content

Input file data_1 10 US data_1 2 US data_1 5 UK data_2 20 ENGLAND data_2 12 KOREA data_3 4 CHINA . . data_60 123 US data_60 23 UK data_60 45 US Desired output file data_1 10 US data_1 5 UK data_2 20 ENGLAND data_2 12 KOREA (2 Replies)
Discussion started by: perl_beginner
2 Replies

6. Shell Programming and Scripting

How to remove duplicate ID's?

HI I have file contains 1000'f of duplicate id's with (upper and lower first character) as below i/p: a411532A411532a508661A508661c411532C411532 Requirement: But i need to ignore lowercase id's and need only below id's o/p: A411532 A508661 C411532 (9 Replies)
Discussion started by: buzzme
9 Replies

7. Shell Programming and Scripting

Remove duplicate

Hi , I have a pipe seperated file repo.psv where i need to remove duplicates based on the 1st column only. Can anyone help with a Unix script ? Input: 15277105||Common Stick|ESHR||Common Stock|CYRO AB 15277105||Common Stick|ESHR||Common Stock|CYRO AB 16111278||Common Stick|ESHR||Common... (12 Replies)
Discussion started by: samrat dutta
12 Replies

8. UNIX for Dummies Questions & Answers

Remove duplicate

Hi, How can I replace || with space and then remove duplicate from following text? T111||T222||T444||T222||T555 Thanks in advance (10 Replies)
Discussion started by: tinku981
10 Replies

9. UNIX for Dummies Questions & Answers

Remove Duplicate Lines

Hi I need this output. Thanks. Input: TAZ YET FOO FOO VAK TAZ BAR Output: YET VAK BAR (10 Replies)
Discussion started by: tara123
10 Replies

10. Shell Programming and Scripting

How to remove duplicate lines?

Hi All, I am storing the result in the variable result_text using the below code. result_text=$(printf "$result_text\t\n$name") The result_text is having the below text. Which is having duplicate lines. file and time for the interval 03:30 - 03:45 file and time for the interval 03:30 - 03:45 ... (4 Replies)
Discussion started by: nalu
4 Replies
RAKE(1) 						 Ruby Programmers Reference Guide						   RAKE(1)

NAME
rake -- Ruby Make SYNOPSIS
rake [--f Rakefile] [--version] [-CGNPgnqstv] [-D [PATTERN]] [-E CODE] [-I LIBDIR] [-R RAKELIBDIR] [-T [PATTERN]] [-e CODE] [-p CODE] [-r MODULE] [--rules] [variable=value] target ... DESCRIPTION
Rake is a simple ruby(1) build program with capabilities similar to the regular make(1) command. Rake has the following features: o Rakefiles (Rake's version of Makefiles) are completely defined in standard Ruby syntax. No XML files to edit. No quirky Makefile syntax to worry about (is that a tab or a space?). o Users can specify tasks with prerequisites. o Rake supports rule patterns to synthesize implicit tasks. o Flexible FileLists that act like arrays but know about manipulating file names and paths. o A library of prepackaged tasks to make building rakefiles easier. OPTIONS
--version Display the program version. -C --classic-namespace Put Task and FileTask in the top level namespace -D [PATTERN] --describe [PATTERN] Describe the tasks (matching optional PATTERN), then exit. -E CODE --execute-continue CODE Execute some Ruby code, then continue with normal task processing. -G --no-system --nosystem Use standard project Rakefile search paths, ignore system wide rakefiles. -I LIBDIR --libdir LIBDIR Include LIBDIR in the search path for required modules. -N --no-search --nosearch Do not search parent directories for the Rakefile. -P --prereqs Display the tasks and dependencies, then exit. -R RAKELIBDIR --rakelib RAKELIBDIR --rakelibdir RAKELIBDIR Auto-import any .rake files in RAKELIBDIR. (default is rakelib ) -T [PATTERN] --tasks [PATTERN] Display the tasks (matching optional PATTERN) with descriptions, then exit. -e CODE --execute CODE Execute some Ruby code and exit. -f FILE --rakefile FILE Use FILE as the rakefile. -h --help Prints a summary of options. -g --system Using system wide (global) rakefiles (usually ~/.rake/*.rake ). -n --dry-run Do a dry run without executing actions. -p CODE --execute-print CODE Execute some Ruby code, print the result, then exit. -q --quiet Do not log messages to standard output. -r MODULE --require MODULE Require MODULE before executing rakefile. -s --silent Like --quiet, but also suppresses the 'in directory' announcement. -t --trace Turn on invoke/execute tracing, enable full backtrace. -v --verbose Log message to standard output (default). --rules Trace the rules resolution. SEE ALSO
ruby(1) make(1) http://rake.rubyforge.org/ REPORTING BUGS
Bugs, features requests and other issues can be logged at <http://onestepback.org/redmine/projects/show/rake>. You will need an account to before you can post issues. Register at <http://onestepback.org/redmine/account/register>. Or you can send an email to the author. AUTHOR
Rake is written by Jim Weirich <jim@weirichhouse.org> UNIX
November 7, 2012 UNIX
All times are GMT -4. The time now is 07:06 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy