UNIX for Dummies Questions & Answers

Sorry to bother you guys again. I have a file1 with multiple columns like this:

Code:

gga_miR_100	gga_miR_300	gga_miR_3500	gga_miR_4600	gga_miR_5600	gga_miR_30	gga_miR_500
kj	rwg	ghhh	jy	jyu	we	vf
5g	5hg	h6	56h	i8	45t	44r4
4bg	4r546	9lgtr	(fer)	4fr	f433	
	3feev	f4	bf4	35g		
		vfr		ge		
		2rr

and I also have a file2 have multiple lines like this:

Code:

gga_miR_3500
gga_miR_4600
gga_miR_500
gga_miR_30

How can I extract the columns in file1 if the first line of each column in file1 exactly match the lines in file2, so I am expecting the output file is like this:

Code:

gga_miR_3500	gga_miR_4600	gga_miR_30	gga_miR_500
ghhh	jy	we	vf
h6	56h	45t	44r4
9lgtr	(fer)	f433	
f4	bf4		
vfr			
2rr

No attempts from your side? Pity...

Try

Code:

awk     'FNR==NR        {H[NR]=$1; MX=NR; next}
         FNR==1         {for (j=1; j<=MX; j++) for (i=1; i<=NF; i++) if (H[j]==$i) C[j]=i}
                        {for (i=1; i<=MX; i++) printf "%s\t", $C[i]
                         printf "\n"}
        ' file2 FS="\t" file1
gga_miR_3500    gga_miR_4600    gga_miR_500     gga_miR_30
ghhh    jy      vf      we
h6      56h     44r4    45t
9lgtr   (fer)           f433
f4      bf4
vfr
2rr

Please note that your columns are NOT in the sequence in which they appear in file2!

Thank you very much RudiC. Actually I am an embryology biologist and I don't have any computer science background and I am a totally rookie. I have to deal with some very large data set from the experimental work recently and I don't know how to cope with them. I learned that some unix command can help me to do that, then I tried to use some very simple command like such as sed or awk to do some basic work and it is really amazing for me. However, I don't have ability to deal with the more complicated problem even these problems seems very naive for you computer guys. So that's why I post many questions on this forum recently. I wish I can figure out some problems but I really don't have the ability right now. Actually I am studying some programming skills by myself now and I hope I can also help some other people like me in the future. I raelly appreciate anyone helped me.

A nice brain teaser!
The following keeps the order of columns in file1:

Code:

awk '
BEGIN {FS="\t"}
FNR==NR {Ctext[$1]; next}
FNR==1 {
  for (i=NF;i>=1;i--) if ($i in Ctext) {C[i]}
}
{
  for (i=1;i<=NF;i++) if (i in C) {printf "%s", sep$i; sep=FS}
  print sep=""
}
' file2 file1

While the following remaps the order according to file2, like Rudi's solution:

Code:

awk '
BEGIN {FS="\t"}
FNR==NR {Ctext[$1]=NR; next}
FNR==1 {
  for (i=NF;i>=1;i--) if ($i in Ctext) {C[Ctext[$i]]=i}
}
{
  for (i=1;(i in C);i++) {printf "%s", sep$C[i]; sep=FS}
  print sep=""
}
' file2 file1

This is all well and we try to help as much as we can. The reason why most of us insist that a genuine effort in solving the problem by oneself is made is because we try to help you to help yourself. Most of us, me included, are professionals and we get paid to do for customers what we help each other to achieve here. The big dfference between writing here and doing work for a customer is: i do not expect my customer to spend any effort - i expect him to pay me for doing his work instead. Here it is the other way round. I do not expect to get paid - not in the narrower sense, obviously, but not even in a virtual sense, like "being paid by recognition" or using what i do here to advertise my skills, but on the other hand i expect my vis-a-vis to spend some effort to digest what i try explain to him and to try to learn what i explain.

Otherwise i have to ask myself why i charge my customer some money for my work when i do it for free here.

But there is - or at least should be - another reason for you to invest effort in solving your problems: whenever you work using a certain tool there is a dialectical connection between the work and the tool: the tools is shaped by the work you expect it to do but also your work is shaped by the tools you use. You sure know the saying that if you only have a hammer everything starts to look like a nail. This the tool shaping the work and the way the work is done. So, when you do your work using a computer as a tool you will not only be able to do it better when you are more competent with it but chances are you will be changing the way you work and you may find new applications to the tool or ways of applying it which you are not aware of right now.

After so much philosophical contemplation here is a practical suggestion for you: most of your problems seem to involve "data massaging" of one sort or the other. Most (perhaps all) of what you do can be done using awk and/or sed. There is a phantastic book about exactly these two tools: it is written by Dale Dougherty and i can wholeheartedly suggest it. It is a good and diverting read while being the best and most complete single source about the topic i ever read.

bakunin

Hi.

Perhaps this will be useful:
More at Beginning Perl for Bioinformatics - O'Reilly Media , however, it looks a bit old me ... cheers, drl