Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

merged 10 files with column extraction into one

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers merged 10 files with column extraction into one
# 1  
Old 09-20-2008
merged 10 files with column extraction into one
Hi, I have 600 text files. In each txt file, I have 3 columns, e.g:

File 1
a 0.21 0.003
b 0.34 0.004
c 0.72 0.002

File 2
a 0.25 0.0083
b 0.38 0.0047
c 0.79 0.00234

File 3
a 0.45 0.0063
b 0.88 0.0027
c 0.29 0.00204
...

my filename as "sc2408_0_5278.txt sc2408_0_5279.txt ..."

I would like to input each 10 files into one file as

a 0.21 0.25 0.45 ..
b 0.34 0.38 0.88 ..
c 0.72 0.79 0.29 ..

with information of column 1 and column 2 only

from File 5278--> 5287, 5288-->5297, and etc; then average all the values in that file except the first column, which results in

a e
b f
c g
...

where
e= (0.21+0.25+0.45+..)/10
f = (0.34 +0.38+0.88)/10
g = (0.72+0.79+0.29)/10

what's the way to do it?




Thank you in advance!
# 2  
Old 09-20-2008
Quote:
awk '/^a/{print $1" "$2}' sc2408_0_52[78-88]|xargs|sed 's/ a/ /g'
till this it will print
Quote:
a 0.21 0.25 0.45 ..
Quote:
awk '/^a/{print $1" "$2}' sc2408_0_52[78-88]|xargs|sed 's/ a/ /g'|awk '{print $1" "(($2+$3+$4+$5+$6+$7+$8+$9+$10+$11)/10)}'
this will print a and avg value..
modify as you want and use it...
# 3  
Old 09-21-2008
Thank you for your brilliant advice.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

UNIX - 2 tab delimited files, conditional column extraction

Please know that I am very new to unix and trying to learn 'on the job'. I'm only manipulating large tab-delimited files (millions of rows), but I'm stuck and don't know how to proceed with the following. Hoping for some friendly advice :) I have 2 tab-delimited files - with differing column &... (10 Replies)
Discussion started by: GTed
10 Replies

2. UNIX for Advanced & Expert Users

Column Extraction for a particular match

Hi, PFB the input: unix/java/perl/random.txt unix1/java1/random1.txt unix2/java2/perl2/random2.txt unix3/java3/random3.txt unix4/random4.txt i want the following output: random.txt random1.txt random2.txt random3.txt random4.txt the patterns can change but i need the .txt file... (5 Replies)
Discussion started by: arindam guha
5 Replies

3. Shell Programming and Scripting

Column extraction from multiple files to multiple files

I have roughly ~30 .txt files in a directory which all have unique names. These files all contain text arranged in columns separated by whitespace (example file: [#YY MM DD hh mm WDIR WSPD GST WVHT DPD APD MWD PRES ATMP WTMP DEWP VIS TIDE #yr mo dy hr mn degT m/s m/s m sec ... (5 Replies)
Discussion started by: aozgaa
5 Replies

4. UNIX for Dummies Questions & Answers

Need help for data extraction if files

Hello all, I want to extract some particular data from a files and than add all the values . but i m not able to cut the particular word(USU-INOCT and USU-OUTOCT) as it is coming not in column. and than able to add values coming in it . can anyone help me Please cat <file name> ... (7 Replies)
Discussion started by: anamdev
7 Replies

5. Shell Programming and Scripting

Print merged rows from two files by applying if condition

Hi all, I have list of two kind of files and I want to compare the rows and print the merged data by applying if condition. First kind of file looks like: and second kind of file looks like : I want to print the rows present in second file followed by 3 more columns from first... (6 Replies)
Discussion started by: CAch
6 Replies

6. Shell Programming and Scripting

Multiple lines in a single column to be merged as a single line for a record

Hi, I have a requirement with, No~Dt~Notes 1~2011/08/1~"aaa bbb ccc ddd eee fff ggg hhh" Single column alone got splitted into multiple lines. I require the output as No~Dt~Notes 1~2011/08/1~"aaa<>bbb<>ccc<>ddd<>eee<>fff<>ggg<>hhh" mean to say those new lines to be... (1 Reply)
Discussion started by: Bhuvaneswari
1 Replies

7. Shell Programming and Scripting

Files extraction - any help ?

Hi Friends, i am new to unix,i have a big doubt/help. I have files in folders SER1 and SER2 with naming convention as below file_2010-03-19.txt and so on the file naming format is file_<date>.txt. I would like to copy the files to directory "Landing" I have entries in a log file log.txt... (5 Replies)
Discussion started by: Gopal_Engg
5 Replies

8. UNIX for Dummies Questions & Answers

Column Extraction

Hello all, I have attached a txt doc with data sorted into columns (it is best to open with "word pad" to maintain the correct format. In the data there are two columns E/N and Ko. I wanted to know how to extract the data and form an excel sheet. Or at least just extract the data. The data is... (4 Replies)
Discussion started by: gingburg
4 Replies

9. Shell Programming and Scripting

Partial Column extraction/Process/Repasting changed Columns back to Source file

I have the following requirement. file1.txt (this could contain 5 million rows) ABC 1234 XYZ .... (3000 bytes) QRD 4612 GHT .... (3000 bytes) I need to create file2.txt 1234 4612 I have a EAI process to change file2.txt into file3.txt 4555 3743 Then I would have to use... (0 Replies)
Discussion started by: jostul
0 Replies
Login or Register to Ask a Question