10-27-2008
removing items with repeated first 3 character
AWK help:
I have a file with following format. I need to remove any entries which are repeated based on first 3 characters. So from the following files I need to remove any entries start with "mas".
mas01bct
mas02bct
mas03bct
mas01bct
mas01bct
mas01bct
mas11bct
mas01bct
mas01bct
mas01bct
mas01bct
mas01bct
mas01bct
mas01bct
mas01bct
pas00abc
mrk01abc
lbc02mis
So the output file should contain:
pas00abc
mrk01abc
lbc02mis
Thanks and appreciate your help.
9 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
looking for a bit of help with sed.
I have a file that looks a bit like this:
sdfghhjk
asdfdfghgj
asdfhgghj
werdfvtfh
edftbgh
1211211221
sdffgfm
dfghnhjm
dfvfsgbgh
adsfv bdhgn
1111111dffg
dfv1122
dsgvbghn111111
fffffffgbdghn
fffffff
sfgh3333gs vdf (5 Replies)
Discussion started by: robsonde
5 Replies
2. Shell Programming and Scripting
Please assist with awk scirpts:
I need to remove items from a file in a batch:
The file that I will remove from has the following format:
abc00tef:10.81.12.3
abc01tef:10.81.12.3
abc02tef:10.81.12.3
abc03tef:10.81.12.3
abc04tef:10.81.12.3
abc05tef:10.81.12.3
I have a file which... (5 Replies)
Discussion started by: amir07
5 Replies
3. Shell Programming and Scripting
OS=HP-UX ksh
The following works, except I want to include the <start> and <end> in the output.
awk -F '<start>' 'BEGIN{RS="<end>"; OFS="\n"; ORS=""} {print $2} somefile.log'
The following work in bash but not in ksh
sed -n '/^<start>/,/^<end>/{/LABEL$/!p}' somefile.log (4 Replies)
Discussion started by: Ikon
4 Replies
4. Programming
Hi everyone! So I have a listView on my Form named "officeView" I already have the code to add and update info into it, but Im having troubles deleting items out of it. :/
Now I know how to delete an Item from the listView, but I want the item before the deleted item to become automatically... (0 Replies)
Discussion started by: romeo5577
0 Replies
5. Shell Programming and Scripting
I have a file like this:
DDD_ABCDE2AB2_1104081408.104480
I need to remove the 1 after the . in the file name so that it reads:
DDD_ABCDE2AB2_1104081408.04480
Having some difficulty getting the command to work. I tried using
cut -d 26
but that just doesn't work. (3 Replies)
Discussion started by: bbbngowc
3 Replies
6. UNIX for Dummies Questions & Answers
I need to remove square brackets from output of script.
Output is:
and I need to remove the square brackets so I am lett with
121 Is sed the only means to do this and if so what are the options?
...ok so far I have managed to get rid of ] by using /usr/bin/sed 's/]//' but that... (5 Replies)
Discussion started by: rob171171
5 Replies
7. UNIX for Dummies Questions & Answers
I need help removing the last character of every line if it is a certain character. For example I need to get rid of a % character if it is in the last position.
Input:
aaa%
%bbb
ccc
d%dd%
Output should be:
aaa
%bbb
ccc
d%dd
I tried this but it gets rid of all of the % characters.... (5 Replies)
Discussion started by: raptor25
5 Replies
8. Shell Programming and Scripting
Hi below is the input file, i need to find repeated words and sum up the values of it which is second field from the repeated work.Im trying but getting no where close to it.Kindly give me a hint on how to go about it
Input
fruits,apple,20,fruits,mango,20,veg,carrot,12,veg,raddish,30... (11 Replies)
Discussion started by: 100bees
11 Replies
9. Shell Programming and Scripting
Hello Experts,
I am trying to write a shell script to find duplicate items in an array, this is what i have tried :
#!/bin/bash
echo "This is another sample Progg to remove duplicate items from an array"
echo "How many number do you want to insert"
read n
for (( i=0; i<$n; i++ ))
do
... (5 Replies)
Discussion started by: mukulverma2408
5 Replies
uniq(1) General Commands Manual uniq(1)
Name
uniq - report repeated lines in a file
Syntax
uniq [-udc[+n][-n]] [input[output]]
Description
The command reads the input file comparing adjacent lines. In the normal case, the second and succeeding copies of repeated lines are
removed; the remainder is written on the output file. Note that repeated lines must be adjacent in order to be found. For further infor-
mation, see
Options
The n arguments specify skipping an initial portion of each line in the comparison:
-n Skips specified number of fields. A field is defined as a string of non-space, non-tab characters separated by tabs and spaces from its
neighbors.
+n Skips specified number of characters in addition to fields. Fields are skipped before characters.
-c Displays number of repetitions, if any, for each line.
-d Displays only lines that were repeated.
-u Displays only unique (nonrepeated) lines.
If the -u flag is used, just the lines that are not repeated in the original file are output. The -d option specifies that one copy of
just the repeated lines is to be written. The normal mode output is the union of the -u and -d mode outputs.
The -c option supersedes -u and -d and generates an output report in default style but with each line preceded by a count of the number of
times it occurred.
See Also
comm(1), sort(1)
uniq(1)