10-07-2010
awk script processing data from 2 files
Hi!
I have 2 files containing data that I need to process at the same time, I have problems in reading a different number of lines from the different files.
Here is an explanation of what I need to do (possibly with an awk script).
File "samples.txt" contains data in the format:
time_instant measure
HTML Code:
903.0 -
906.43 18.4
912.7 17.5
918.05 11.2
File "time.txt" contains data in the format
time_instant position
HTML Code:
900 out
901 out
902 out
903 in
904 in
905 in
906 in
907 in
908 out
909 out
910 out
911 in
912 in
913 in
I need to compute temporal averages of the measures in samples.txt, but the temporal averages must be related only to the time instants in which the position in time.txt is "in".
So, I need at the end to have:
a/b
where a=sum(measure_i*duration_measure_i)
b=total_measurement_duration
- measure_i is the second field in every line of samples.txt
- duration_measure_i is the difference between the 1st fields in two consecutive lines in samples.txt and eventually the duration of the period in which the position in time.txt was "out"
So, for example I would do for the first measurement (that is on the second line in samples.txt, since the 1st line only tells me when measurements start):
a=18.4*(906.43-903.0-0)
b=(906.43-903.0-0)
Then for the second:
a+=17.5*(912.7-906.43-2)
b+=(912.7-906.43-2)
I am quite new to awk and don't know how to read two lines at a time from samples.txt and search in time.txt the lines that are between the time instants written from samples.txt
Any suggestion? Thank you very much!!
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Helllo UNIX Forum :)
Since I am posting on this board, yes, I am new to UNIX!
I read a copy of "UNIX made easy" from 1990, which felt like a making a "computer-science time jump" backwards ;)
So, basically I have some sort of understanding what the basic concept is.
Problem Description:... (6 Replies)
Discussion started by: Kasimir
6 Replies
2. Shell Programming and Scripting
I have a file, first 5 columns are very normal, like "1107",106027,71400,"Y","BIOLOGY",,
however, the 6th columns, the user can put comments, anything, just any characters, like new line, double quote, single quote, whatever from the keyboard, like"Please load my previous SOM597G course content in... (3 Replies)
Discussion started by: freelong
3 Replies
3. Shell Programming and Scripting
Hi
I have files in our UNIX directory like the below
-rw-r--r-- 1 devinfo devsupp 872 Sep 14 02:09 IMGBTREE27309_12272_11_1_0_FK.idx0
-rw-r--r-- 1 devinfo devsupp 872 Sep 14 02:09 IMGBTREE27309_12272_11_0_0_PK.idx0
-rw-r--r-- 1 devinfo devsupp 432 Sep 14... (7 Replies)
Discussion started by: rbmuruga
7 Replies
4. Shell Programming and Scripting
I have following result.log file (always has 2 lines) which I need to process,
cat result.log
name.cmd.method,"result","abc","xyz";
name="hello,mine.12345,"&"tree"&" xyz "&" tree "&" xyz",
data="way,"&" 1"&"rate-me"&"1"&"rate-me",str="",ret="";
now I need to extract the strings/data as... (4 Replies)
Discussion started by: perlDiva
4 Replies
5. Shell Programming and Scripting
Hello,
I have some bitrate data in a csv which is in an odd format and is difficult to process in Excel when I have thousands of rows. Therefore, I was thinking of doing this in bash and using awk as the primary application except that due to its complication, I'm a little stuck.
... (24 Replies)
Discussion started by: shadyuk
24 Replies
6. Programming
Hello,
I extracted a list of files in a directory with the command ls . However this is not my computer, so the ls functionality has been revamped so that it gives the filesizes in front like this :
This is the output of ls command : I stored the output in a file filelist
1.1M... (5 Replies)
Discussion started by: ajayram
5 Replies
7. Shell Programming and Scripting
Hy!
I need to post-process some data files which have variable (and periodic) number of fields. For example, I need to square (data -> data*data) the folowing data file: -5.34281E-28 -3.69822E-29 8.19128E-29 9.55444E-29 8.16494E-29 6.23125E-29
4.42106E-29 2.94592E-29 1.84841E-29 ... (5 Replies)
Discussion started by: radudownload
5 Replies
8. Shell Programming and Scripting
Dear Friends,
I am in urgent need for awk/sed/sh script for converting a specific data format (.txt) to .xls.
The input is as follows:
>gi|1234|ref|
Query = 1 - 65, Target = 1677 - 1733
Score = 8.38, E = 0.6529, P = 0.0001513, GC = 46
fd sdfsdfsdfsdf
fsdfdsfdfdfdfdfdf... (6 Replies)
Discussion started by: Amit1
6 Replies
9. Shell Programming and Scripting
Hi,
I'm using awk command in bash script. I'm able to pass multiple files to awk for processing.The code i can use is as below(sample code)
#!/bin/bash
awk -F "," 'BEGIN {
...
...
...
}' file1 file2 file3
In the above code i'm passing the file names manually and it is fine till my... (7 Replies)
Discussion started by: shree11
7 Replies
10. UNIX for Beginners Questions & Answers
Hi guys, I receive a list from pipe (with fixed number of lines) like this:
name1
name2
name3
And in my ./ folder I have three files:
01-oldname.test
02-someoldname.test
03-evenoldername.test
How to rename files one by one using while read?
Desired result:
01-name1.test
02-name2.test... (3 Replies)
Discussion started by: useretail
3 Replies
LEARN ABOUT OPENSOLARIS
pdbtxt2html
pdbtxt2html(1) General Commands Manual pdbtxt2html(1)
NAME
pdbtxt2html - Doc Text to HTML converter for Palm Pilots
SYNOPSIS
pdbtxt2html [ -t ] file.txt [ file.html ]
pdbtxt2html -v
DESCRIPTION
pdbtxt2html converts text converted from a Doc(4) file via txt2pdbdoc(1) to HTML. If no HTML filename is given, the generated HTML is sent
to standard output.
Document Title
The first line of the file is used for the HTML document title.
Bookmarks
The last line of the file is examined and, if it contains a string enclosed between < and >, that is taken to be the bookmark marker. The
entire file is then scanned looking for lines beginning with it (ignoring leading whitespace). These lines are converted to HTML headings.
The number of whitespace characters after the first bookmark marker is used for heading level 1. The level of subsequent headings is set
to the number of whitespace characters between the bookmark marker and the bookmark text minus the number for the first bookmark plus one.
Embedded URLs
Valid URLs (according to RFC 1630) embedded in the text are turned into hyperlinks. The ftp, gopher, http, https, mailto, news, telnet,
and wais URLs are recognized.
OPTIONS
-t Compile a table of contents and insert it between the first heading and the body.
-v Print the version number to standard output and exit.
EXAMPLE
To convert a Doc file to HTML:
txt2pdbdoc alice.pdb alice.txt
pdbtxt2html alice.txt alice.html
SEE ALSO
html2pdbtxt(1), txt2pdbdoc(1), doc(4), pdb(4)
Tim Berners Lee. Universal Resource Identifiers in WWW, Network Working Group of the Internet Engineering Task Force, June 1994.
http://info.internet.isi.edu/in-notes/rfc/files/rfc1630.txt
AUTHOR
Paul J. Lucas <pauljlucas@mac.com>
txt2pdbdoc January 21, 2005 pdbtxt2html(1)