Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Extract specific content from data and rename its header problem asking

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extract specific content from data and rename its header problem asking
# 1  
Old 03-23-2010
Extract specific content from data and rename its header problem asking
Input file 1:
Code:
>pattern_5
GAATTCGTTCATGTAGGTTGASDASFGDSGRTYRYGHDGSDFGSDGGDSGSDGSDFGSDF
ATTTAATTATGATTCATACGTCATATGTTATTATTCAATCGTATAAAATTATGTGACCTT
SDFSDGSDFKSDAFLKJASLFJASKLFSJAKJFHASJKFHASJKFHASJKFHSJAKFHAW
>pattern_1
AAGTCTTAAGATATCACCGTCGATTAGGTTTATACAGCTTTTGTGTTATTTAAATTTGAC
ASDASRFSARFASFEDEGSDGHSDHWDYTQATWQRQOPTPEOTPWRIYRHIRGOPEIWRA
.
.

Input file 2:
Code:
pattern_5    5   15
pattern_5    18  25 
pattern_1    10  19
pattern_1    22  27
.
.

Desired output:
Code:
>pattern_5_0.01
TCGTTCATGTA
>pattern_5_0.02
TTGASDAS
>pattern_1_0.01
GATATCACCG
>pattern_1_0.02
GATTAG
.
.

I got a long list of input file 1 and input file 2. Input file 1 is the raw data while input file 2 is the range of input file 1 data that I'm interested to extract and generate the output result file. The column 2 and column 3 of input file 2 is the position that I interested to extract from the data of input file 1. The output file I will rename with the header like "pattern_*_0.0*"
It seems like awk or perl scripts able to archive these goal.
Thanks a lot for any advice.
Last edited by patrick87; 03-23-2010 at 06:31 AM.. Reason: further explaining of my question
# 2  
Old 03-23-2010
Can you explain how you get the below line in red from your desired output..

Code:
>pattern_5_0.01
TCGTTCATGTA
>pattern_5_0.02
TTGASDAS
>pattern_1_0.01
GATATCACCG
>pattern_1_0.02
GATTAG

# 3  
Old 03-24-2010
Hi malcomex999,
I just edited my question and explain the usage of column 2 and column 3 inside input file 2.
Thanks for your advice.

---------- Post updated 03-24-10 at 01:53 AM ---------- Previous update was 03-23-10 at 04:32 AM ----------

Hi malcomex999,
do you got any idea to archive the desired goal?
Thanks a lot.
# 4  
Old 03-24-2010
Hi, patrick87:
Code:
awk 'FNR==NR {if (/^>/) p=substr($0,2); else a[p]=a[p] $0; next} {printf(">%s_0.%02u\n%s\n", $1, ++i[$1], substr(a[$1], $2, $3-$2+1))}' f1 f2

While processing the first file (FNR==NR), if a line begins with ">", grab everything that follows it and store it in p, the pattern name. If a line does not begin with a ">", then it is data for the current pattern, p; append the line to a[p], that pattern's entry in array a. Repeat until done with the first file.

For the second file, we use the pattern name in the first field and the index values in the second and third fields to extract the required substring from a[$1], while incrementing a counter for each pattern name seen, in the i array, i[$1].

Cheers,
Alister
Last edited by alister; 03-24-2010 at 05:31 AM..
# 5  
Old 03-24-2010
Thanks alister,
I'm trying apply your awk code to my case now Smilie
Besides that, thanks a lot for your further explanation of your awk code too.
I very appreciate and thanks for your help and advice.
Thanks again ^^

---------- Post updated at 04:19 AM ---------- Previous update was at 03:43 AM ----------

Hi Alister,
Your awk code worked perfectly in my case. Thanks a lot.
Can I ask you if my input file 2 change like this:
Code:
pattern_5    15  5
pattern_5    18  25 
pattern_1    10  19
pattern_1    22  27
.
.

How I can edit the awk code that you suggested to give the same output result as above?
Is it I need to add the "if" condition in the awk code for this problem?
Thanks again for your advice.
# 6  
Old 03-24-2010
You can modify alister code like this...
Code:
awk 'FNR==NR {if (/^>/) p=substr($0,2); 
else a[p]=a[p] $0; next} 
{printf(">%s_0.%02u\n%s\n", $1, ++i[$1], substr(a[$1], $2, ($2>=$3?$3:$3-$2+1)))}' f1 f2

# 7  
Old 03-24-2010
Quote:
Originally Posted by malcomex999
You can modify alister code like this...
Code:
awk 'FNR==NR {if (/^>/) p=substr($0,2); 
else a[p]=a[p] $0; next} 
{printf(">%s_0.%02u\n%s\n", $1, ++i[$1], substr(a[$1], $2, ($2>=$3?$3:$3-$2+1)))}' f1 f2

Hi, malcomeex999:

That tweak is incorrect, if I understand the modification to f2 correctly. If the second field is greater than the third, then it instead of being treated as the beginning index of the substring, it should be considered the end index (and the interpretation of the third field should be complementarily swapped). The correct solution requires that the second argument to substr() be modified as well, since in the case of $2 > $3, it should be $3 not $2.

By the way, malcomeex999 and rdcwayx, thank you very much for your bit awards. It's appreciated Smilie


Hi, patrick87:

One solution to handle both cases (even if they appear within the same file2):
Code:
awk 'FNR==NR {if (/^>/) p=substr($0,2); else a[p]=a[p] $0; next}
     {if ($2>$3) {t=$2; $2=$3; $3=t}; printf(">%s_0.%02u\n%s\n", $1, ++i[$1], substr(a[$1], $2, $3-$2+1))}' f1 f2

It works identically to my earlier solution except that it tests the second and third fields in f2. If the first index is greater than the second, their values are swapped before the substr() call.

Regards,
Alister
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with rename data content

Input file: data21_result1 data23_result1 data43_result1 data43_result2 data43_result3 data3_result1 . . data9_result1 Desired output data1_result1 data2_result1 data3_result1 data3_result2 data3_result3 data4_result1 (3 Replies)
Discussion started by: perl_beginner
3 Replies

2. Shell Programming and Scripting

extract specific string and rename file

Hi all, I am working on a small prog.. i have a file.txt which contains random data... K LINES V4 ADD CODE `COMPANY` ADD CODE `DISTRIBUTOR` SEQ NAME^K LINES V5 SEQ NAME^K LINES V6 ADD `PACK-LDATE` SEQ NAME^K^KCOMMAND END^KHEADINFO... (1 Reply)
Discussion started by: mukeshguliao
1 Replies

3. Shell Programming and Scripting

Help with rename header content based on reference file problem

I got long list of reference file >data_tmp_number_22 >data_tmp_number_12 >data_tmp_number_20 . . Input file: >sample_data_1 Math, 5, USA, tmp SDFEWRWERWERWRWER FSFDSFSDFSDGSDGSD >sample_data_2 Math, 15, UK, tmp FDSFSDFF >sample_data_3 Math, 50, USA, tmp ARQERREQR . . Desired... (7 Replies)
Discussion started by: perl_beginner
7 Replies

4. Shell Programming and Scripting

Extract all content that match exactly only specific word

Input: 21 templeta parent 35718 36554 . - . ID=parent_cluster_50.21.11; Name=Partial%20parent%20for%20training%20set; 21 templeta kids 35718 36554 . - . ID=_52; Parent=parent_cluster_5085.21.11; 21 templeta ... (7 Replies)
Discussion started by: patrick87
7 Replies

5. Shell Programming and Scripting

mailx requirement - email body header in bold and data content in normal text

Dear all- I have a requirement to send an email via email with body content which looks something below- Email body contents -------------------- RequestType: Update DateAcctOpened: 1/5/2010 Note that header information and data content should be normal text.. Please advice on... (5 Replies)
Discussion started by: sureshg_sampat
5 Replies

6. Shell Programming and Scripting

Way to extract detail and its content above specific value problem asking

Input file: >position_10 sample:68711 coords:5453-8666 number:3 type:complete len:344 MSINQYSSDFHYHSLMWQQQQQQQQHQNDVVEEKEALFEKPLTPSDVGKLNRLVIPKQHA ERYFPLAAAAADAVEKGLLLCFEDEEGKPWRFRYSYWNSSQSYVLTKGWSRYVKEKHLDA NRTS* >position_4 sample:68711 coords:553-866 number:4 type:partial len:483... (7 Replies)
Discussion started by: patrick87
7 Replies

7. Shell Programming and Scripting

Remove specific pattern header and its content problem facing

Input file: >TRACK: Position: 1 TYPE: 1 Pos: SVAVPQRHHPGGTVFREPIIIPAIPRLVPGWNKPIIIGRHAFGDQYRATDRVIPGPGKLE LVYTPVNGEPETVKVYDFQGGGIAQTQYNTDESIRGFAHASFQMALLKGLPLYMSTKNTI LKRYDGRFKDIFQEIYESTYQKDFEAKNLWYEHRLIDDMVAQMIKSEGGFVMALKNYDGD >TRACK: Position: 1 TYPE: 2 Pos: FAHASFQMALLKGLPLYMS... (8 Replies)
Discussion started by: patrick87
8 Replies

8. Shell Programming and Scripting

Extract specific data content from a long list of data

My input: Data name: ABC001 Data length: 1000 Detail info Data Direction Start_time End_time Length 1 forward 10 100 90 1 forward 15 200 185 2 reverse 50 500 450 Data name: XFG110 Data length: 100 Detail info Data Direction Start_time End_time Length 1 forward 50 100 50 ... (11 Replies)
Discussion started by: patrick87
11 Replies

9. Shell Programming and Scripting

Extract all the content after a specific data

My input: >seq_1 DSASSTRRARRRRTPRTPSLRSRRSDVTCS >seq_3 RMRLRRWRKSCSERS*RRSN >seq_8 RTTGLSERPRLPTTASRSISSRWTR >seq_10 NELPLEKGSLDSISIE >seq_9 PNQGDAREPQAHLPRRQGPRDRPLQAYA+ QVQHRRHDHSRTQH*LCRRRQREDCDRLHR >seq_4 DRGKGQAGCRRPQEGEALVRRCS>seq_6 FA*GLAAQDGEA*SGRG My output: Extract all... (22 Replies)
Discussion started by: patrick87
22 Replies

10. Shell Programming and Scripting

Extract specific content from a file

My input file: >sequence_1 ASSSSSSSSSSSDDDDDDDDDDDCCCCCCC ASDSFDFFDFDFFWERERERERFSDFESFSFD >sequence_2 ASDFDFDFFDDFFDFDSFDSFDFSDFSDFDSFASDSADSADASD ASDFFDFDFASFASFASFAFSFFSDASFASFASFAFS >sequence_3 VEDFGSDGSDGSDGSDGSDGSDGSDG dDFSDFSDFSDFSDFSDFSDFSDFSDF SDGFDGSFDGSGSDGSDGSDGSDGSDG My... (22 Replies)
Discussion started by: patrick87
22 Replies
Login or Register to Ask a Question