Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Delete duplicate in certain number of string

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Delete duplicate in certain number of string
# 1  
Old 02-11-2011
Delete duplicate in certain number of string
Hi,
do you have awk or sed sommand taht will delete duplicate lines like.

sample:
Code:
server1-log1-14
server1-log2-14
superserver-time-2
superserver-log-2

output:
Code:
server-log1-14
superserver-time-2



thansk
Last edited by Franklin52; 02-11-2011 at 03:24 AM.. Reason: Please use code tags
# 2  
Old 02-11-2011
Code:
awk -F \- '!a[$1]++' infile

This User Gave Thanks to rdcwayx For This Post:
kenshinhimura
# 3  
Old 02-11-2011
thanks dude..awesome..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Find duplicate values in specific column and delete all the duplicate values

Dear folks I have a map file of around 54K lines and some of the values in the second column have the same value and I want to find them and delete all of the same values. I looked over duplicate commands but my case is not to keep one of the duplicate values. I want to remove all of the same... (4 Replies)
Discussion started by: sajmar
4 Replies

2. Shell Programming and Scripting

Delete duplicate directories?

Is there a way via some bash script or just cmd to find duplicate directories? i have main folders: TEST1 TEST2 In folder TEST1 is some amount of same folders as in folder TEST2 can be this done? i tried fdupe but it only search for dupe files not whle dirs thx! (8 Replies)
Discussion started by: ZerO13
8 Replies

3. Shell Programming and Scripting

Delete duplicate row

Hi all, how can delete duplicate files in file form, e.g. $cat file1 aaa 123 234 345 456 bbb 345 345 657 568 ccc 345 768 897 456 aaa 123 234 345 456 ddd 786 784 234 263 ccc 345 768 897 456 aaa 123 234 345 456 ccc 345 768 897 456 then i need ouput file1 some, (4 Replies)
Discussion started by: aav1307
4 Replies

4. Shell Programming and Scripting

Remove not only the duplicate string but also the keyword of the string in Perl

Hi Perl users, I have another problem with text processing in Perl. I have a file below: Linux Unix Linux Windows SUN MACOS SUN SUN HP-AUX I want the result below: Unix Windows SUN MACOS HP-AUX so the duplicate string will be removed and also the keyword of the string on... (2 Replies)
Discussion started by: askari
2 Replies

5. Shell Programming and Scripting

Find number in string and delete it AWK

Hi all, i have some logs on my linux server that looks like this: CDR.2012-04-30:30-04-2012 14:09:36;123456456654;A;Greetings! Your amount is 42.24 dollars (without VAT) until 30/04/2012 11:00. CDR.2012-04-30:30-04-2012 14:09:36;12154878454212;A;Greetings! Your amount is 4203.2 dollars... (2 Replies)
Discussion started by: arrals_vl
2 Replies

6. Shell Programming and Scripting

Delete duplicate rows

Hi, This is a followup to my earlier post him mno klm 20 76 . + . klm_mango unix_00000001; alp fdc klm 123 456 . + . klm_mango unix_0000103; her tkr klm 415 439 . + . klm_mango unix_00001043; abc tvr klm 20 76 . + . klm_mango unix_00000001; abc def klm 83 84 . + . klm_mango... (5 Replies)
Discussion started by: jacobs.smith
5 Replies

7. Shell Programming and Scripting

Delete Duplicate line (not really) from the file

I need help in figuring out hoe to delete lines in a data file. The data file is huge. I am currently using "vi" to search and delete the lines - which is cumbersome since it takes lots of time to save that file (due to its huge size). Here is the issue. I have a data file with the following... (4 Replies)
Discussion started by: GosarJunk
4 Replies

8. UNIX for Dummies Questions & Answers

Delete duplicate second line

Hi ALL I need a help I need to retain only the first line of 035 if I have two line before =040 , if only one then need to take that Eg: Input =035 (ABC)12324141241 =035 (XYZPQR)704124 =040 AB$QS$WEWR =035 (ABC)08080880809 =035 (XYZPQR)9809314 =040 ... (4 Replies)
Discussion started by: umapearl
4 Replies

9. Shell Programming and Scripting

filtering out duplicate substrings, regex string from a string

My input contains a single word lines. From each line data.txt prjtestBlaBlatestBlaBla prjthisBlaBlathisBlaBla prjthatBlaBladpthatBlaBla prjgoodBlaBladpgoodBlaBla prjgood1BlaBla123dpgood1BlaBla123 Desired output --> data_out.txt prjtestBlaBla prjthisBlaBla... (8 Replies)
Discussion started by: kchinnam
8 Replies

10. UNIX for Advanced & Expert Users

Delete Duplicate Cronjobs

I set up same cronjobs in two different users to generate messages at 5:30 AM Not Its generating duplicate messages. I want to delete the cron entries set up in the first user, but I am unable to view the entries in that user. I tried to find the process Id, but its not showing any id Could... (2 Replies)
Discussion started by: nskworld
2 Replies
Login or Register to Ask a Question