Adding files of numerical data


 
Thread Tools Search this Thread
Top Forums Programming Adding files of numerical data
# 8  
Old 02-05-2006
Thanks very much for that. Unfortunately i can't really understand it. Sorry, my knowledge of C is terrible. Where do i put the names of the two files i'm adding together. And what does nawk do? Thanks again for all your help.
# 9  
Old 02-07-2006
Is that piece of code not just adding all the second column together? I need to add row1,column2 of file1 to row1,column2 of fille2;
row2,column2 of file1 to row2,column2 of fille2;
..3, ... 2 to .. 3, ... 2 ;
and so on, and put these new values into the second column of a new array, with the first column remaining the same in all the files.
Please if you can help me at all, i have to get this done real soon.
Thanks.
# 10  
Old 02-07-2006
ok, here is what i am trying to understand
input data will come form 2 files, each having 4 columns, so if i merge it columnwise in a single file, I will have 8 columns, and the result should be a file containing 3*col2+2*col6
is that correct
to start with what you can do is merge the two files using the paste command and redirect the output to an intermediate third file
paste data1 data2 > data3

now data3 has 8 columns, the second column of first file data1 is in column position 2 of data3, and 2nd column of data2 is in column position 6 of data3

now what you need is 3*column2+2*column6. Is that right? if so then you use the command awk as below

awk '{printf("%e\n", (3*$2+2*$6))}' data3 > data4

now data4 is the final file
# 11  
Old 02-07-2006
He does not want more columns. He wants to add the values in the columns across files. The only thing wrong with vger99's approach is that it doesn't ignore comments.

Code:
nawk '$1 = "%" { arr[$1] += $2 }
END {
  for ( i in arr )
     print i, arr[i]
}' myFile

On linux you probably use awk instead of nawk. $2 is the 2nd column. $1 is the first column. Look at your second data line... it would do "arr[5] += 3E-6". other data lines with a 5 in the first column get added to the same array element. At the end, all defined elements are displayed. Want to do two file? replace "myfile" with "myfile1 myfile2". It will loop over all input files. To do your ratios, mention one input file twice and the other three times.
# 12  
Old 02-07-2006
to ignore 'comments' - assuming comments start with a '%' in the first character position:
Code:
nawk '/^[^%]/ { arr[$1] += $2 }
END {
  for ( i in arr )
     print i, arr[i]
}' myFile

# 13  
Old 02-07-2006
About ignoring comments... I fixed mine. I am assuming $1 is a "%". (I had $! instead of $1). But I like vger99's better anyway.
# 14  
Old 02-07-2006
Guys, you're absolute legends. I won't get to try it out til the morning, but if ye reckon it'll work thats good enough for me. One question though; The values in the first column range from 1E-2 to about 4E+4. Does your program just increment as it goes, or does it read in each $1 value itself. Either way will this program work with such small/large values? Thanks again.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Divide a numerical data column by a variable

Hello, I have two files, f1 and f2. f1 has 5 columns like so: a b c d 154 e f g h 365 ..... f2 has two columns, the first column contains the name of the above file and second column contains a constant which is to be used for division. e.g. file1 56 I want to divide the 5th... (2 Replies)
Discussion started by: Gussifinknottle
2 Replies

2. Shell Programming and Scripting

how to extract data from numbered files using linux in the numerical order-

Hi experts, I have a list of files containing forces as the only number as follows. Force1.txt Force2.txt Force3.txt Force4.txt Force5.txt . . . . . . . . . Force100.txt I want to put all the data(only a number ) in these forces files in the file with the same order like 1,2,3 ..100 .... (2 Replies)
Discussion started by: hamnsan
2 Replies

3. Shell Programming and Scripting

Deleting particular files with a numerical suffix

Hello I have a directory with a list of files which have a particular numerical suffix. E.g filename_0 filename_1 filename_18500 filename_10000 I want to delete all files from this directory which have a filename which have a numerical suffix greater than 10540. So any files... (5 Replies)
Discussion started by: kamal_p_99
5 Replies

4. UNIX for Dummies Questions & Answers

List files according to the numerical value

Hi, I have a large number of files which are named as follows. VF_50, VF_100, VF_150, VF_250, VF_300, VF_350, VF_400, VF_450, VF_500. When I do an 'ls' it arranges the files in the following way VF_100, VF_150, VF_250, VF_300, VF_350, VF_400, VF_450, VF_50, VF_500. Is there a way to... (2 Replies)
Discussion started by: lost.identity
2 Replies

5. UNIX for Dummies Questions & Answers

moving/copying files in a numerical order

Hi I am newbie to unix scripting, but i have enough knowledge to understand. I have a specific questions like, I use to collect like 3500 files per experiment, each one named like data_001.img.. data_002.img data_003.img .... data_3500.img I would like to move every 12 files in the 3500... (3 Replies)
Discussion started by: wpat
3 Replies

6. Shell Programming and Scripting

script to sort a string of numerical data in set fields

So, I will be working with someone and basically we are trying to build a form that is submitted most likely via the web and the data is just a string of numbers. like: 19383882872201110929282821818182827349190102837364718191001932873711 Now, each number is part of a numerical value of... (4 Replies)
Discussion started by: tlarkin
4 Replies

7. Shell Programming and Scripting

compare files by numerical value

Hi everyone, I would love to have a script that does the following: I have one file that looks like that: ATOM 1 BB SER 1 1 -31.958 -25.125 -11.061 1.00 0.00 ATOM 3 BB GLY 1 2 -32.079 -26.085 -14.466 1.00 0.00 ATOM 4 BB VAL 1 3 ... (1 Reply)
Discussion started by: s-layer
1 Replies

8. Shell Programming and Scripting

Reading Numerical Binary Data using KSH

Hi, I've searched and couldn't find anyone else with this problem. Is there anyway (preferably using ksh - but other script languages would do) that I can read in binary float data into a text file. The data (arrays from various stages of radar processing) comes in various formats, but mainly... (3 Replies)
Discussion started by: Jonny2Vests
3 Replies

9. Shell Programming and Scripting

How to strip non numerical data out of file?

Hi, How can I remove all non numerical data from line, so I don't want to delete the line but to have only the numbers. e.g.: ######### 123 aaa124 125bbb 126 127 ######### So I want all the leading and trailing non numerical stuff(letters/white space/tabs anything else except... (10 Replies)
Discussion started by: Juha
10 Replies

10. Shell Programming and Scripting

Listing files in numerical order

Hi, I'm trying to write a ksh script to copy a specified number of files from one directory to another. The files are named in the convention <switchname>_log.<num> and the numbers are sequential single digit onwards. I figured I could find some parameter for ls which would list the files in... (3 Replies)
Discussion started by: Steve_H
3 Replies
Login or Register to Ask a Question