Sponsored Content
Top Forums Shell Programming and Scripting Get a none duplicate list file Post 23000 by usfrog on Friday 14th of June 2002 11:24:05 AM
Old 06-14-2002
sort will do (look at the man page for more options):

cat myFile | sort -u > myFileNoDup # create 2nd file
mv myFileNoDup myFile # replace first file
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing duplicate files from list with different path

I have a list which contains all the jar files shipped with the product I am involved with. Now, in this list I have some jar files which appear again and again. But these jar files are present in different folders. My input file looks like this /path/1/to a.jar /path/2/to a.jar /path/1/to... (10 Replies)
Discussion started by: vino
10 Replies

2. Shell Programming and Scripting

List Duplicate

Hi All This is not class assignment . I would like to know awk script how to list all the duplicate name from a file ,have a look below Sl No Name Dt of birth Location 1 aaa 1/01/1975 delhi 2 bbb 2/03/1977 mumbai 3 aaa 1/01/1976 mumbai 4 ... (22 Replies)
Discussion started by: vakharia Mahesh
22 Replies

3. UNIX for Dummies Questions & Answers

CSV file:Find duplicates, save original and duplicate records in a new file

Hi Unix gurus, Maybe it is too much to ask for but please take a moment and help me out. A very humble request to you gurus. I'm new to Unix and I have started learning Unix. I have this project which is way to advanced for me. File format: CSV file File has four columns with no header... (8 Replies)
Discussion started by: arvindosu
8 Replies

4. Shell Programming and Scripting

Remove duplicate lines from first file comparing second file

Hi, I have two files with below data:: file1:- 123|aaa|ppp 445|fff|yyy 999|ttt|jjj 555|hhh|hhh file2:- 445|fff|yyy 555|hhh|hhh The records present in file1, not present in file 2 should be writtent to the out put file. output:- 123|aaa|ppp 999|ttt|jjj Is there any one line... (3 Replies)
Discussion started by: gani_85
3 Replies

5. Shell Programming and Scripting

Find duplicate based on 'n' fields and mark the duplicate as 'D'

Hi, In a file, I have to mark duplicate records as 'D' and the latest record alone as 'C'. In the below file, I have to identify if duplicate records are there or not based on Man_ID, Man_DT, Ship_ID and I have to mark the record with latest Ship_DT as "C" and other as "D" (I have to create... (7 Replies)
Discussion started by: machomaddy
7 Replies

6. UNIX for Dummies Questions & Answers

Duplicate lines in a file

I have a file with following data A B C I would like to print like this n times(For eg:5 times) A B C A B C A B C A B C A (7 Replies)
Discussion started by: nsuresh316
7 Replies

7. Shell Programming and Scripting

Duplicate files and output list

Gents, I have a file like this. 1 1 1 2 2 3 2 4 2 5 3 6 3 7 4 8 5 9 I would like to get something like it 1 1 2 2 3 4 5 3 6 7 Thanks in advance for your support :b: (8 Replies)
Discussion started by: jiam912
8 Replies

8. Shell Programming and Scripting

Find and remove duplicate record and print list

Gents, I needs to delete duplicate values and only get uniq values based in columns 2-27 Always we should keep the last record found... I need to store one clean file and other with the duplicate values removed. Input : S3033.0 7305.01 0 420123.8... (18 Replies)
Discussion started by: jiam912
18 Replies

9. Shell Programming and Scripting

List duplicate files based on Name and size

Hello, I have a huge directory (with millions of files) and need to find out duplicates based on BOTH file name and File size. I know fdupes but it calculates MD5 which is very time-consuming and especially it takes forever as I have millions of files. Can anyone please suggest a script or... (7 Replies)
Discussion started by: prvnrk
7 Replies

10. UNIX for Beginners Questions & Answers

Iterate through a list - checking for a duplicate then report it ot

I have a job that produces a file of barcodes that gets added to every time the job runs I want to check the list to see if the barcode is already in the list and report it out if it is. (3 Replies)
Discussion started by: worky
3 Replies
SUBSTR_REPLACE(3)							 1							 SUBSTR_REPLACE(3)

substr_replace - Replace text within a portion of a string

SYNOPSIS
mixed substr_replace (mixed $string, mixed $replacement, mixed $start, [mixed $length]) DESCRIPTION
substr_replace(3) replaces a copy of $string delimited by the $start and (optionally) $length parameters with the string given in $replace- ment. PARAMETERS
o $string - The input string. An array of strings can be provided, in which case the replacements will occur on each string in turn. In this case, the $replacement, $start and $length parameters may be provided either as scalar values to be applied to each input string in turn, or as arrays, in which case the corresponding array element will be used for each input string. o $replacement - The replacement string. o $start - If $start is positive, the replacing will begin at the $start'th offset into $string. If $start is negative, the replacing will begin at the $start'th character from the end of $string. o $length - If given and is positive, it represents the length of the portion of $string which is to be replaced. If it is negative, it rep- resents the number of characters from the end of $string at which to stop replacing. If it is not given, then it will default to strlen( $string ); i.e. end the replacing at the end of $string. Of course, if $length is zero then this function will have the effect of inserting $replacement into $string at the given $start offset. RETURN VALUES
The result string is returned. If $string is an array then array is returned. EXAMPLES
Example #1 Simple substr_replace(3) examples <?php $var = 'ABCDEFGH:/MNRPQR/'; echo "Original: $var<hr /> "; /* These two examples replace all of $var with 'bob'. */ echo substr_replace($var, 'bob', 0) . "<br /> "; echo substr_replace($var, 'bob', 0, strlen($var)) . "<br /> "; /* Insert 'bob' right at the beginning of $var. */ echo substr_replace($var, 'bob', 0, 0) . "<br /> "; /* These next two replace 'MNRPQR' in $var with 'bob'. */ echo substr_replace($var, 'bob', 10, -1) . "<br /> "; echo substr_replace($var, 'bob', -7, -1) . "<br /> "; /* Delete 'MNRPQR' from $var. */ echo substr_replace($var, '', 10, -1) . "<br /> "; ?> Example #2 Using substr_replace(3) to replace multiple strings at once <?php $input = array('A: XXX', 'B: XXX', 'C: XXX'); // A simple case: replace XXX in each string with YYY. echo implode('; ', substr_replace($input, 'YYY', 3, 3))." "; // A more complicated case where each replacement is different. $replace = array('AAA', 'BBB', 'CCC'); echo implode('; ', substr_replace($input, $replace, 3, 3))." "; // Replace a different number of characters each time. $length = array(1, 2, 3); echo implode('; ', substr_replace($input, $replace, 3, $length))." "; ?> The above example will output: A: YYY; B: YYY; C: YYY A: AAA; B: BBB; C: CCC A: AAAXX; B: BBBX; C: CCC NOTES
Note This function is binary-safe. SEE ALSO
str_replace(3), substr(3), String access and modification by character. PHP Documentation Group SUBSTR_REPLACE(3)
All times are GMT -4. The time now is 01:26 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy