Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

deleting words in list with more than 2 identical adjacent characters

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers deleting words in list with more than 2 identical adjacent characters
# 1  
Old 06-20-2011
deleting words in list with more than 2 identical adjacent characters
Morning Guys & Gals,


I am trying to figure out a way to remove lines from a file that have more than
2 identical characters in sequence..


So if for instance the list would look like ;

Quote:
abcdefghij
aabcdefghi
aaabcdefgh
aaaabbcdef
suuuperman
suupermaan
alpha00112
alpha00011
the output would be ;
Quote:
abcdefghij
aabcdefghi
suupermaan
alpha00112
I can't seem to get my head around perl (among many other things..) so if
possible would appreciate feedback on using sed or awk as well on the above.

Any help greatly appreciated !
# 2  
Old 06-20-2011
Hi


Code:
$ cat a
abcdefghij
aabcdefghi
aaabcdefgh
aaaabbcdef
suuuperman
suupermaan
alpha00112
alpha00011

$ sed  '/\([a-z0-9]\)\1\{2,\}/d' a
abcdefghij
aabcdefghi
suupermaan
alpha00112


Guru.
# 3  
Old 06-20-2011
Thanks for the quick response Guru ! Looks to be working great Smilie Could you also perhaps advise what would an option if there were special characters in the list as well ? Is that possible with sed ?
# 4  
Old 06-20-2011
try this
Code:
# cat file
212!!\|||
@@@listx1
@@xlist21
suppperrr
supperrx3
^&&&563ht
&&5336asf
--+++_(**
:[:::][]]
[[::]][][
 
abcdefghij
aabcdefghi
aaabcdefgh
aaaabbcdef
suuuperman
suupermaan
alpha00112
alpha00011

Code:
# sed '/\([^A-Za-z0-9_]\|[A-Za-z0-9]\)\1\{2,\}/d' file
@@xlist21
supperrx3
&&5336asf
[[::]][][
 
abcdefghij
aabcdefghi
suupermaan
alpha00112

regards
ygemici
# 5  
Old 06-20-2011
Thanks alot for your great response ! You guys here rock Smilie
# 6  
Old 06-20-2011
Thats great guys, thanks a lot ! Just another quick query, could I ask you to give quick explanation on what the;
\1\{2,\}
is enabling so I can work on it and if necessary change to alternative possibilities ?

It looks like it is allowing 1 through to 2 adjacent characters allowed, am trying to learn
this sed, however it is proving a challenge !
# 7  
Old 06-20-2011
Quote:
Originally Posted by TAPE
Thats great guys, thanks a lot ! Just another quick query, could I ask you to give quick explanation on what the;
\1\{2,\}
is enabling so I can work on it and if necessary change to alternative possibilities ?

It looks like it is allowing 1 through to 2 adjacent characters allowed, am trying to learn
this sed, however it is proving a challenge !
\1\{2,\} is that matchs string for pattern is repetitions 2 or more than its in defined `\(....\) `

regards
ygemici
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Deleting a list of words from a text file

Hello, I have a list of words separated by spaces I am trying to delete from a text file, and I could not figure out what is the best way to do this. what I tried (does not work) : delete="password key number verify" arr=($delete) for i in arr { sed "s/\<${arr}\>]*//g" in.txt } >... (5 Replies)
Discussion started by: Hawk4520
5 Replies

2. UNIX for Dummies Questions & Answers

Deleting words between every appearance of two words

Hi there, newbie there. I've been browsing the forums hoping to find a solution that answers a problem similar to what I need, but haven't had much luck. Any help would be greatly appreciated. Thanks! I need to delete a bunch of text between every appearance of two words in a really large file... (3 Replies)
Discussion started by: lendl
3 Replies

3. UNIX for Dummies Questions & Answers

Deleting words and sorting

I have a file that looks some like this: I need to delete most of the information and sort the rest in such way that I get the following output file Any help will be greatly appreciated (3 Replies)
Discussion started by: Xterra
3 Replies

4. Shell Programming and Scripting

finding and removing 2 identical consecutive words in a text

i want to write a shell script that correct a text file.for example if i have the input file: "john has has 2 apples anne has 3 oranges oranges" i want that the output file be like this: "john has 2 apples anne has 3 oranges" i've tried to read line by line from input text file into array... (11 Replies)
Discussion started by: cocostaec
11 Replies

5. Shell Programming and Scripting

Get characters between two words

Guys, Here is the txt file... SLIC N0SLU704034789 rŒ° EJ00 ó<NL DMRG>11 100 4B 2 SLIC N0SLU704034789 rŒ° TJ10 <4000><NL> 2 SLIC N0SLU704034789 ... (2 Replies)
Discussion started by: gowrishankar05
2 Replies

6. Shell Programming and Scripting

Removing identical words in column

I have a file that needs to be cleaned up. Here is the file: Project Project John Project Gary Project Sean Project2 Project2 Lisa Project2 Tyler Project2 Sam Project3 Project3 Mike Project3 Bran I need the o/p to be: Project John Gary Sean Project2 (7 Replies)
Discussion started by: leepet01
7 Replies

7. Shell Programming and Scripting

Deleting words between tags

Hi !!! I need to write a script(ksh) that deletes any character outside <start> tag and </start> from a file. For eg: $cat file.txt <start> ad bd </start> as</start> <start> d e f mb<start>mu g h i (7 Replies)
Discussion started by: PRKS
7 Replies

8. Shell Programming and Scripting

deleting blank line and row containing certain words in single sed command

Hi Is it possible to do the following in a single command /usr/xpg4/bin/sed -e '/rows selected/d' /aemu/CALLAUTO/callauto.txt > /aemu/CALLAUTO/callautonew.txt /usr/xpg4/bin/sed -e '/^$/d' /aemu/CALLAUTO/callautonew.txt > /aemu/CALLAUTO/callauto_new.txt exit (1 Reply)
Discussion started by: aemunathan
1 Replies

9. Shell Programming and Scripting

deleting symbols and characters between two words

Hi Please tell me how could i delete symbols, whitespaces, characters, words everything between two words in a line. Let my file is aaa BB ccc ddd eee FF kkk xxx 123456 BB 44^& iop FF 999 xxx uuu rrr BB hhh nnn FF 000 I want to delete everything comes in between BB and FF( deletion... (3 Replies)
Discussion started by: rish_max
3 Replies

10. Shell Programming and Scripting

list only identical filename

Hi Friends, We have four filenames with first few digits are identical.From those files, i need to pick up 2 files based on my parameter. The following example will give more information to understand my question. Files in the directory: small_customer_aa.csv small_customer_ab.csv... (6 Replies)
Discussion started by: HAA
6 Replies
Login or Register to Ask a Question