Sponsored Content
Top Forums Shell Programming and Scripting Remove repeated line using Perl Post 302452308 by rdcwayx on Friday 10th of September 2010 12:54:12 AM
Old 09-10-2010
Code:
sort -u

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

how to remove line from /etc/vfstab using shell / perl

Hi, could someone help me on this i want to remove line from /etc/vfstab in the system how to do that it is rite now like this /dev/vx/dsk/appdg1/mytestvol /dev/vx/rdsk/appdg1/mytestvol /mytest vxfs 3 no largefiles /dev/vx/dsk/appdg1/mytestvol1 ... (2 Replies)
Discussion started by: tarunn.dubeyy
2 Replies

2. Shell Programming and Scripting

remove anything after repeated string pattern found

HI, Can anyone help me with a script. i/p calc 1 2 3 4 5 6 7 8 calc 4 5 6 calc 7 8 9 o/p calc 1 2 3 4 5 6 7 8 calc 4 5 6 i.e remove anything after where two times the string calc is found. thanks (3 Replies)
Discussion started by: Indra2011
3 Replies

3. Shell Programming and Scripting

remove brackets and put it in a column and remove repeated entry

Hi all, I want to remove the remove bracket sign ( ) and put in the separate column I also want to remove the repeated entry like in first row in below input (PA156) is repeated ESR1 (PA156) leflunomide (PA450192) (PA156) leflunomide (PA450192) CHST3 (PA26503) docetaxel... (2 Replies)
Discussion started by: manigrover
2 Replies

4. Shell Programming and Scripting

Perl regex to remove a segment in a line

Hello, ksh on Sun5.8 here. I have a pipe-delimited, variable length record file with sub-segments identified with a tilda that we receive from a source outside of our control. The records are huge, and Perl seems to be the only shell that can handle the huge lines. I am new to Perl, and am... (8 Replies)
Discussion started by: gary_w
8 Replies

5. Shell Programming and Scripting

ksh Remove and replace repeated in file

Hi, i need to read a line from a file and count the number of times it appear in, then continuous to the second line with the same. So when i count a line i have to remove all duplicates in the file to not count it another time. while read line do n=$(grep -c $line File) print "$line... (5 Replies)
Discussion started by: ToniX
5 Replies

6. Shell Programming and Scripting

Remove regularly repeated lines

How can i delete some regular repeated lines in a file? example: in_file EDGE 1 2 12 EDGE 2 3 23 EDGE 3 4 34 EDGE 5 6 56 EDGE 6 7 67 EDGE 7 8 78 EDGE 9 10 910 EDGE 10 11 1011 EDGE 11 12 1112 EDGE 13 14 1314 EDGE 14 15 1415 EDGE 15 16 1516 EDGE 17 18 1718 EDGE 18 19 1819 EDGE 19... (8 Replies)
Discussion started by: saeed.soltani
8 Replies

7. Shell Programming and Scripting

Need to remove first 6 lines and last line in a array ---- perl scripting

Hi I have stored a command output in an array like below @a = `xyz`; actually xyz comnad will give the output like this tracker date xxxxxxx xxxxxxx --------------------- 1 a 2 b ---------------------- i have stored the "xyz" output to an... (3 Replies)
Discussion started by: siva kumar
3 Replies

8. Shell Programming and Scripting

Find repeated word and take sum of the second field to it ,for all the repeated words in awk

Hi below is the input file, i need to find repeated words and sum up the values of it which is second field from the repeated work.Im trying but getting no where close to it.Kindly give me a hint on how to go about it Input fruits,apple,20,fruits,mango,20,veg,carrot,12,veg,raddish,30... (11 Replies)
Discussion started by: 100bees
11 Replies

9. Shell Programming and Scripting

Remove repeated letter words

Hi, I have this text file with these words and I need help with removing words with repeated letter from these lines. 1 ama 5 bib 29 bob 2 bub 5 civic 2 dad 10 deed 1 denned 335 did 1 eeee 1 eeeee 2 eke 8... (4 Replies)
Discussion started by: crepe6
4 Replies

10. Shell Programming and Scripting

Remove duplicate lines which has been repeated 4 times

Remove duplicate lines which has been repeated 4 times attached test.txt below command tried and not getting expect output. for i in `cat test.txt | uniq` do num=`cat test.txt | grep $i | wc -l` echo $i $num done test.txt ... (17 Replies)
Discussion started by: Kalia
17 Replies
uniq(1) 						      General Commands Manual							   uniq(1)

NAME
uniq - report repeated lines in a file SYNOPSIS
fields] chars] [input_file [output_file]] DESCRIPTION
reads the input text file input_file, comparing adjacent lines, and copies the result to output_file. If input_file is not specified, the standard input and standard output are used. If input_file is specified, but output_file is not, results are printed to standard output. input_file and output_file must not be the same file. Line-Comparison Options recognizes the following options when comparing adjacent lines: Print those lines that are repeated in the original file. Print copy only of each repeated line in the input file. Generate an output report in default style except that each line is preceded by a count of the number of times it occurred. If this option is specified, the and options are ignored if either or both are also present. If none of the options or are present, prints the results of the union of the and options, producing a copy of the original input file with the second and succeeding copies of any repeated lines removed. (Note that repeated lines must be adjacent in order to be found -- see sort(1)). Field-Skip Options Two options are provided for skipping an initial portion of each line when making comparisons: Ignore the first fields fields, together with any blanks before each. fields is a positive decimal integer. A field is defined as a string of non-space, non-tab characters separated by tabs and/or spaces from its neighbors. Ignore the first chars characters. chars is a positive decimal integer. Each line in the input is assumed to be terminated with a new line character for purposes of comparison. Fields are skipped before characters. EXTERNAL INFLUENCES
Environment Variables must be equal to the value it had when the input files were sorted. determines the interpretation of text within files as single- and/or multi-byte characters, and defines a space character when the or option is used. determines the language in which messages are displayed. If or is not specified in the environment or is set to the empty string, the value of is used as a default for each unspecified or empty variable. If is not specified or is set to the empty string, a default of "C" (see lang(5)) is used instead of If any internationalization variable contains an invalid setting, behaves as if all internationalization variables are set to "C". See environ(5). International Code Set Support Single- and multi-byte character code sets are supported. RETURN VALUE
Exit values are: 0 Successful completion. >0 Error condition occurred. AUTHOR
was developed by OSF and HP. SEE ALSO
comm(1), sort(1). STANDARDS CONFORMANCE
uniq(1)
All times are GMT -4. The time now is 05:50 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy