Help with duplicate common data content


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help with duplicate common data content
# 1  
Old 09-06-2012
Help with duplicate common data content

Input file:
Code:
#data_131 0 >content..._* 1 >content..._at_+/97.20%     
#data_137 0 >content..._* 1 >content..._at_+/97.20% 2 >seq..._* 3 >content..._at_+/97.20% 4 >content..._at_+/97.20%
#data_141 0 >content..._*       
#data_150 0 >content..._* 1 >content..._at_+/97.20% 2 >seq..._* 3 >content..._at_+/97.20%

Desired output:
Code:
#data_131 0 >content..._* 
#data_131 1 >content..._at_+/97.20%     
#data_137 0 >content..._* 
#data_137 1 >content..._at_+/97.20% 
#data_137 2 >seq..._* 
#data_137 3 >content..._at_+/97.20% 
#data_137 4 >content..._at_+/97.20%
#data_141 0 >content..._*       
#data_150 0 >content..._*
#data_150 1 >content..._at_+/97.20%
#data_150 2 >seq..._*
#data_150 3 >content..._at_+/97.20%

Start from column 2, 2 column of data will form 1 group, eg. column 2 and 3 of each line is one group of data, column 3 and 4 of each line is another one group of data. All of them share the common data which is column 1.
Thus I plan to duplicate the column 1 content for every set of group.
# 2  
Old 09-06-2012
Assuming lines are always this "good" (column 1 followed by at least 1 group of 2 columns):
Code:
awk '{for(i=2;i<=(NF-1);i+=2) print $1,$i,$(i+1)}' file

This User Gave Thanks to elixir_sinari For This Post:
# 3  
Old 09-06-2012
Try something like this..

Code:
awk '{for(i=1;i<=NF;i++) {if ( $i ~ /^>/ ) { print $1,$(i-1), $i} }}' file


Last edited by pamu; 09-06-2012 at 07:05 AM.. Reason: corrected after lookin through elixir's sollution...
This User Gave Thanks to pamu For This Post:
# 4  
Old 09-06-2012
Code:
awk -F">" '{for(i=1;i<=NF;i++){split($i,b," ");if(i==1){x=b[1];y=b[2];continue}; print x" "y" "FS b[1];y=b[2]}}' filename

This User Gave Thanks to raj_saini20 For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove the duplicate content in a file

Here is the contents of test.txt Dependencies Resolved Changes in packages about to be updated: ChangeLog for: 1:perl-Archive-Extract-0.38-131.el6_4.x86_64, - Resolves: #915692 - CVE-2013-1667 (DoS in rehashing code) Dependencies Resolved Changes in packages about to be updated: ... (5 Replies)
Discussion started by: ashokvpp
5 Replies

2. Shell Programming and Scripting

Common Issue: Table to be displayed in mail content and not as an attachment

The requirement is to send a report from an UNIX server. I have a list of data to be shown on the report, say from database. I am using sendexchange to send mail and it always send in plain text fromat and the data displayed are not aligned properly. I have seperated all the columns with fixed... (14 Replies)
Discussion started by: PikK45
14 Replies

3. UNIX for Dummies Questions & Answers

Extract common data out of multiple files

I am trying to extract common list of Organisms from different files For example I took 3 files and showed expected result. In real I have more than 1000 files. I am aware about the useful use of awk and grep but unaware in depth so need guidance regarding it. I want to use awk/ grep/ cut/... (7 Replies)
Discussion started by: macmath
7 Replies

4. Shell Programming and Scripting

Help with duplicate data content problem asking

Input file: A_69510335_ASD>aw 1199470 USA A_119571157_C>awe,QWEQE 113932840 USA C_34646666_qwe>TAWTT,G,TT 112736796 UK C_69510335_QW>T 1199470 USA D_70520237_WR>QEE,G 34459863 UK D_71380003_QWR>T 145418226 IK . Desired output: A_69510335_ASD>aw 1199470 USA... (1 Reply)
Discussion started by: perl_beginner
1 Replies

5. Shell Programming and Scripting

find common data

Hey guys, I have two files. file1 and file2. file1: a,1 b,2 c,343 d,343 e,4343 f,4544 file 2: a, d, e, Now i need to find the common data between these files from file1. i.e a,1 (8 Replies)
Discussion started by: jaituteja
8 Replies

6. Shell Programming and Scripting

Help with replace duplicate content

Input file: CCNI data564_input1 264 CORO1A data564_input2 155 ABC-B data17_input1 3466 ABC-B data17_input2 1133 ABC-B data17_input3 2162 ABC-B data17_input4 2019 HNRNPA2B1 data95_input1 101 HNRNPA2B1 data95_input2 340 IFITM1 data105_input2 291 IFITM2 data105_input1 505... (3 Replies)
Discussion started by: cpp_beginner
3 Replies

7. Shell Programming and Scripting

Help with remove duplicate content

Input file data_1 10 US data_1 2 US data_1 5 UK data_2 20 ENGLAND data_2 12 KOREA data_3 4 CHINA . . data_60 123 US data_60 23 UK data_60 45 US Desired output file data_1 10 US data_1 5 UK data_2 20 ENGLAND data_2 12 KOREA (2 Replies)
Discussion started by: perl_beginner
2 Replies

8. Shell Programming and Scripting

Delete duplicate data and pertain the latest month data.

Hi I have a file with following records It contains three months of data, some data is duplicated,i need to access the latest data from the duplicate ones. for e.g; i have foll data "200","0","","11722","-63","","","","11722","JUL","09" "200","0","","11722","-63","","","","11722","JUL","09"... (10 Replies)
Discussion started by: vee_789
10 Replies

9. Shell Programming and Scripting

Help with remove duplicate content and only keep the first content detail

Input data_10 SSA data_2 TYUE data_3 PEOCV data_6 SSAT data_21 SSA data_19 TYUEC data_14 TYUE data_15 SSA data_32 PEOCV . . Desired Output data_10 SSA data_2 TYUE data_3 PEOCV data_6 SSAT data_19 TYUEC (9 Replies)
Discussion started by: patrick87
9 Replies

10. Shell Programming and Scripting

Extract specific data content from a long list of data

My input: Data name: ABC001 Data length: 1000 Detail info Data Direction Start_time End_time Length 1 forward 10 100 90 1 forward 15 200 185 2 reverse 50 500 450 Data name: XFG110 Data length: 100 Detail info Data Direction Start_time End_time Length 1 forward 50 100 50 ... (11 Replies)
Discussion started by: patrick87
11 Replies
Login or Register to Ask a Question