need to remove invariant characters

Tags

shell scripts

Top Forums Shell Programming and Scripting need to remove invariant characters

08-22-2012

Moderator

3,791, 1,452

Join Date: Oct 2010

Last Activity: 1 August 2020, 1:38 AM EDT

Posts: 3,791

Thanks Given: 183

Thanked 1,452 Times in 1,302 Posts

Try this updated solution:

Code:

gawk '
{
   key[NR]=$1;
   for(i=1;i<length($2);i+=2) {
       site[i,NR]=substr($2,i,2)
       if(i>maxi)maxi=i
   }
}
END {
    c=0
    for(i=1;i<=maxi;i+=2) {
           v=""
           mismatch=0
           for(r=1;r<=NR;r++) {
              if(v=="" && !(site[i,r]~"-"))v=site[i,r];
              else if(length(v)&&site[i,r]!=v) {
                 if(++mismatch>1) {
                     keep[++c]=i;
                     r=NR
                 }
              }
           }
        }
        for(r=1;r<=NR;r++) {
           printf "%s ", key[r]
           for(i=1;i<=c;i++) printf "%s", site[keep[i],r]
           printf "\n"
        }
}' infile

Chubler_XL

View Public Profile for Chubler_XL

Find all posts by Chubler_XL

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove first 2 characters and last two characters of each line

here's what im trying to do. i have a file containing lines similar to this: data.txt: 1hsRmRsbHRiSFZNTTA1dlEyMWFkbU5wUW5CSlIyeDFTVU5SYjJOSFRuWmpia0ZuWXpKV2FHTnRU 1lKUnpWMldrZFZaMG95V25oYQpSelEyWTBka2QyRklhSHBrUjA1b1kwUkJkd3BOVXpWM1lVaG5k...

2. UNIX for Dummies Questions & Answers

How do I remove ^M characters with VI

I have a file with all kinds of ^M at the end of each line. How the heck can these be removed? I tried a global search and replace, but it doesn't seem to work. Thanks!

3. Shell Programming and Scripting

Remove whitespaces in the n first characters?

I assume removing whitespaces in the n first characters of a string would be an easy task for sed? If so, how?

4. UNIX for Dummies Questions & Answers

How to Remove Special Characters

Dear Members, We have a file which contains some special characters. I need to replace these special character by a new line character(\n). The Special character is \x85. I am not sure what this character means and how we can remove it. Any inputs are greatly appreciated. Thanks...

5. Shell Programming and Scripting

Remove characters from file name

Here is my code. for file in *1.3.html ; do mv "$file" `echo $file | tr '.1.3' ''` ; done For some reason I am getting an error. mv: file.idlesince.1.3.html and file.idlesince.1.3.html are identical Could this be done a different way?

6. UNIX for Advanced & Expert Users

remove characters

hi i have a file with these strings: 123_abc_X1116990 how to get rid of 123_abc_ and keep only X1116990? I have columns of these: 123_abc_X1134640 123_dfg_X1100237 123_tyu_X1103112 123_tyui_X1116990 thx

7. UNIX for Dummies Questions & Answers

How to remove Characters before '~'

8. UNIX for Dummies Questions & Answers

Remove control characters

Hi, When I do a man and save it into a file, I end up getting a lot of control characters. How can I remove them?? I tried this: /1,$ s/^H//g But I get an error saying "no previous regular expression". Can someone help me with this. Thanks, Aravind

LEARN ABOUT DEBIAN

update-metainit

UPDATE-METAINIT(1)					User Contributed Perl Documentation					UPDATE-METAINIT(1)

NAME

       update-metainit - Generates init scripts

SYNOPSIS

       update-metainit [--remove initname [--purge] ]

DESCRIPTION

       Metainit solves the problem of writing good init scripts. Instead of manually creating these important files, they are derived from a
       declaritive description in the metainit files in /etc/metainit. These files can be shipped with packages or created by the local adminis-
       trator.

       If update-metainit called without argument, it will regenerate init scripts for all the files in /etc/metainit. The generated files contain
       a large warning in form of a comment that they will be overridden. Modifications are preferably done in the files in /etc/metainit and made
       effective by running update-metainit. If needed, the administrator can prevent modified init files by removing the warning comment.

OPTIONS

       --remove initname
	   This command will remove any generated and non-modified scripts that were created by the metainit file with the name initname.

       --purge
	   Only usable with --remove. Will remove the generated files even if modified.

SEE ALSO

       dh_metainit(1)

AUTHOR

       Joachim Breitner <nomeata@debian.org>

perl v5.8.8							    2007-07-30							UPDATE-METAINIT(1)

Shell Programming and Scripting