03-07-2014
Find duplicate words using sed
I have following statement and I want to find duplicate word using sed command. How is it possible?
"detect string and remove the duplicate string"
There could be many statements in a file and each line may have duplicate word.
Thanks!
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi, can anyone help me with this small problem.
Say i've got duplicate words in a list e.g
blue
orange
green
green
pink
blue
red
How can I delete all reoccurences of the same word?
Thanks (1 Reply)
Discussion started by: zulander
1 Replies
2. Shell Programming and Scripting
Hi,
Please help!
I have a file having duplicate words in some line and I want to remove the duplicate words.
The order of the words in the output file doesn't matter.
INPUT_FILE
pink_kite red_pen ball pink_kite ball
yellow_flower white no white no
cloud nine_pen pink cloud pink nine_pen... (6 Replies)
Discussion started by: sam_2921
6 Replies
3. Shell Programming and Scripting
hello,
i 'd like your help about a bash script which:
1. finds inside the html file (it is attached with my post) the code number of the Latest Stable Kernel,
2.finds the link which leads to the download location of the Latest Stable Kernel version,
(the right link should lead to the file... (3 Replies)
Discussion started by: alex83
3 Replies
4. Shell Programming and Scripting
Hi,
In a file, I have to mark duplicate records as 'D' and the latest record alone as 'C'.
In the below file, I have to identify if duplicate records are there or not based on Man_ID, Man_DT, Ship_ID and I have to mark the record with latest Ship_DT as "C" and other as "D" (I have to create... (7 Replies)
Discussion started by: machomaddy
7 Replies
5. Shell Programming and Scripting
<Update>
I have the solution:
sed 's/\{3\}/&;&;---;4/'
The thread can be marked as solved!
</Update>
Hi There,
I'm working on a script processing some data from a website into cvs format. There is only one final problem left I can't find a solution.
I've processed my file... (0 Replies)
Discussion started by: lolworlds
0 Replies
6. Shell Programming and Scripting
I have file which is almost look like below
MMIT
MMIT
... (2 Replies)
Discussion started by: manas_ranjan
2 Replies
7. Shell Programming and Scripting
hello
I have a file of the form:
nameA number number ...
nameB number number ...
nameA number number ...
nameC number number ...
nameD number number ...
10
nameA number number ...
nameC number number ...
nameB number number ...
nameC number number ...... (4 Replies)
Discussion started by: phaethon
4 Replies
8. UNIX for Dummies Questions & Answers
Tried using sed and uniq but it's removing the entire line. Can't seem to figure a way to just remove the word. Any help is appreciated. I have a file:
dog, text1, text2, text3
dog, text1, text2, text3
dog, text1, text2, text3
cat, text1, text2, text3 Trying to remove all duplicate instances... (6 Replies)
Discussion started by: jimmyf
6 Replies
9. Shell Programming and Scripting
I am looking for a sed in which I can recognize all of the text in between two indicators and then replace it with a place holder.
For instance, the 1st indicator is a list of words "no|noone|havent" and the 2nd indicator is a list of punctuation ".|,|!".From a sentence such as
"noone... (3 Replies)
Discussion started by: owwow14
3 Replies
10. Shell Programming and Scripting
Hi dears
i have text file like this:
INPUT.txt
001_1_173 j nuh ]az
001_1_174 j ]esma. nuh ]/.xori
.
.
. and have another text
like this
TABLE.txt
j j
nuh word1... (6 Replies)
Discussion started by: alii
6 Replies