Hello,
I have a nexus alignment file that looks like this:


bar101_min2covg_binarynex 11001-100111
bar102_min2covg_binarynex 110010010011
bar103_min2covg_binarynex 11101010--11
etc.

There are 41 rows and 28014 characters in each, with 0, 1, and missing data (-) as the three possibilities. Probably 80% of all the sites are invariant, and I would like to remove them from the alignment. So, I'm looking for a way to scan through this alignment file and remove all sites where all rows' values match, or where only 1 row differs, ignoring missing datapoints to make this determination (i.e. if several rows have missing data at a site but all the others match, it gets chopped). A slight complication is that the data come in pairs, so I need to evaluate sites 1/2, 3/4, 5/6, 7/8, etc. etc. in pairs and eliminate them only if both sites are invariant across all rows. I'm kind of stumped at how to approach this, and fairly new to this kind of data manipulation. Does anyone have suggestions for how I might approach this?

The ideal output from the example would be:

bar101_min2covg_binarynex 001001
bar102_min2covg_binarynex 000100
bar103_min2covg_binarynex 1010--

Thanks for the help!