UNIX for Dummies Questions & Answers

I don't really know much about UNIX commands, so if someone could help me understand how to do this, I'd really appreciate it.

I have a text file with data that looks like this (filename: numbers.txt):
1 1 1 1 1 1 1 1 1 2 1 1_2 2_1
1 1 1 1 1 1 1 1 2 1 2 1_2 2_1
1 1 1 1 1 1 1 1 2 1 2 1_2 2_1
1 1 1 1 1 1 1 1 3 1 1 1_3 3_1
1 1 1 1 1 1 1 2 1 2 1 1_2 2_1
1 1 1 1 1 1 1 2 2 1 2 1_2 2_1
1 1 1 1 1 1 1 3 1 3 1 1_3 3_1
1 1 1 1 1 1 1 3 1 1 3 1_3 3_1
1 1 1 1 1 1 1 4 1 1 1 1_4 4_1
1 1 1 1 1 1 2 1 2 1 1 1_2 2_1

11 integers followed by an unspecified number of entries in the form "x_y".

What I want to do is this:
1) look at ONLY the x_y portions of each line, determining which lines are unique AFTER the 11 integers. In the above text, only the second-to-last line meets those criteria (1_4 4_1 doesn't appear anywhere else on the list).
2) Take those (partially) unique lines and write them to a new text file called new_numbers.txt.

(In my above example, new_numbers.txt would have only one line of text: 1 1 1 1 1 1 1 4 1 1 1 1_4 4_1)

If anyone can help me understand how to do this, I'd be very grateful! Thank you so much for your time and help!

---------- Post updated at 05:02 PM ---------- Previous update was at 04:59 PM ----------

If it's helpful, I should mention that the file (numbers.txt) is a file I've created myself, so if it would be easier to complete my task if the text were formatted differently, I can do that easily. (Like, if it would be better to have some sort of special character between the 11 integers and the x_y numbers, or if the x_y numbers should come at the beginning of the line, etc)

Thanks!

Code:

sort -ozac100.out -k12 -k11 -u zac100.in

cat zac100.out
1 1 1 1 1 1 1 1 1 2 1 1_2 2_1
1 1 1 1 1 1 1 1 2 1 2 1_2 2_1
1 1 1 1 1 1 2 1 2 1 1 1_2 2_1 
1 1 1 1 1 1 1 1 3 1 1 1_3 3_1
1 1 1 1 1 1 1 3 1 1 3 1_3 3_1
1 1 1 1 1 1 1 4 1 1 1 1_4 4_1

is that it?

Unfortunately not. The only line that should be in the output file is the one that ends in 1_4 4_1. I want it to interpret all the lines ending in 1_2 2_1 as duplicates (even though literally they're only partial duplicates).

Thanks for the effort, though! Any other ideas?

Code:

> cat sorttest
1 1 1 1 1 1 1 1 1 2 1 1_2 2_1
1 1 1 1 1 1 1 1 2 1 2 1_2 2_1
1 1 1 1 1 1 1 1 2 1 2 1_2 2_1
1 1 1 1 1 1 1 1 3 1 1 1_3 3_1
1 1 1 1 1 1 1 2 1 2 1 1_2 2_1
1 1 1 1 1 1 1 2 2 1 2 1_2 2_1
1 1 1 1 1 1 1 3 1 3 1 1_3 3_1
1 1 1 1 1 1 1 3 1 1 3 1_3 3_1
1 1 1 1 1 1 1 4 1 1 1 1_4 4_1
1 1 1 1 1 1 2 1 2 1 1 1_2 2_1

Code:

sort -k12 sorttest | uniq -c -f11 | perl -nle 'print $2 if /^(\s*1 )(.+)/'

Code:

> sort -k12 sorttest | uniq -c -f11 | perl -nle 'print $2 if /^(\s*1 )(.+)/'
1 1 1 1 1 1 1 4 1 1 1 1_4 4_1

the code you can use:
sort -k 12 numbers.txt|uniq -f 11 -c|awk -F " " '$1==1{print}'|cut -f 2- >new_file






if you want to know about this above how it performs then just ask

regards,
Sanjay

Have you tried your solution?

yes vgres it is working fine.
and giving the correct output
1 1 1 1 1 1 1 4 1 1 1 1_4 4_1