Shell Programming and Scripting

AWK help:

I have a file with following format. I need to remove any entries which are repeated based on first 3 characters. So from the following files I need to remove any entries start with "mas".


mas01bct
mas02bct
mas03bct
mas01bct
mas01bct
mas01bct
mas11bct
mas01bct
mas01bct
mas01bct
mas01bct
mas01bct
mas01bct
mas01bct
mas01bct
pas00abc
mrk01abc
lbc02mis


So the output file should contain:

pas00abc
mrk01abc
lbc02mis

Thanks and appreciate your help.

> cut -c1-3 file01 | sort | uniq -c | awk '$1<2 {print $2}' >file01k
> egrep -f file01k file01
pas00abc
mrk01abc
lbc02mis

Use nawk or /usr/xpg4/bin/awk on Solaris:

Code:

awk 'END { 
  while (++i <= c) 
    if (f[substr(s[i],1,3)] < 2) 
      print s[i] 
      }
{ 
  f[substr($0,1,3)]++ 
  s[++c]=$0 
  }' infile

Another one:

Code:

awk '
NR==FNR{a[substr($0,1,3)]++;next}
a[substr($0,1,3)]<2' file file

Use nawk or /usr/xpg4/bin/awk on Solaris.

Regards

Thanks, but I am interested to remove only desired repeated item e.g. "mas" only or any other what I would like to remove not all. So I need to hardcode "mas" .

Thanks.

Code:

awk 'END { 
  while (++i <= c)
    if (t && s[i] !~ "^"p)
      print s[i] 
      }
f["^"p]++ { t = 1 } 
{ s[++c] = $0 }' p=mas infile

Thanks, but I am getting error:

$ ./remove_device_type.mas
awk: syntax error near line 3
awk: illegal statement near line 3
awk: syntax error near line 6
awk: bailing out near line 6