Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: awk solution for taking bins
Top Forums UNIX for Dummies Questions & Answers awk solution for taking bins Post 302773671 by torchij on Thursday 28th of February 2013 12:41:35 PM
Old 02-28-2013
Hi, I'll try to be more clear with the example.

Thanks for the response, here is an example, i'll focus on just the second column

input
Code:
$ cat file.2.txt
3
2
1
10
10
10
25
30
60

The purpose is to reduce the data for an x,y scatterplot, because the file is millions of lines long. Instead of plotting every point, I want to take an average of every "n" number of points, and plot that one number. Bin might not be the correct word, perhaps a "rolling-average"? For example a bin of 3 would break the data down like so:

Code:
$ cat file.2.txt
#bin A
3
2         #average all three = 2
1

#bin B
10
10     #average all three = 10
10

#bin C
25
30     # average all 3 = 38.3
60

Output would then be:

Code:
2
10
38.3

For the case where bin is 2

Code:
$ cat file.2.txt
#bin A
3    # average = 2.5
2

#bin B
1    # average = 5.5
10

#binC
10    # average = 10
10

#bin D
25    # average = 27.5
30

#binE - ignored because only one value
60

Finally, doing this for both the x and y axis (the original file), for bin of 3:

Input:
Code:
$ cat file.txt
1      3
2      2
3      1
4      10
5      10
6      10
7      25
8      30
9      60

Output:
2      2
5      10
8      38.8

Many thanks, I hope this is more clear
Torch
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash/AWK Newbie taking on more than he can chew.

A few questions: I'm trying to use Bash (although I'm not against using AWK) to try to accomplish a few things, but I'm stumped on a few points. I'm learning most of the basics quickly: but there are a few things I can't figure out. 1. I'm trying to count the number of .txt files in a... (3 Replies)
Discussion started by: Asylus
3 Replies

2. Shell Programming and Scripting

Is there a awk solution for this??

I am writing a awk script that gathers certain data from certain fields. I needed a awk solution for this, because it will later become a function in the script. I have the following data that I need output on a single line, but record spans across multilple lines and records are not... (7 Replies)
Discussion started by: timj123
7 Replies

3. Shell Programming and Scripting

Awk solution

Hello! Well, I searched and wasn't able to find a specific example of my dilemma, so hopefully someone could assist? Or maybe there was an example but I missed it? I have two files: file1 = order data file file2 = list of 65,000+ order numbers I would like to extract from 'file1' any... (5 Replies)
Discussion started by: rm -r *
5 Replies

4. Shell Programming and Scripting

Calculating frequency of values within bins

Hi, I am working with files containing 2 columns in which i need to come up with the frequency/count of values in col. 2 falling within specifics binned values of col. 1. the contents of a sample file is shown below: 15 12.5 15 11.2 16 0.2 16 1.4 17 1.6 18 4.5 17 5.6 12 8.6 11 7.2 9 ... (13 Replies)
Discussion started by: ida1215
13 Replies

5. UNIX for Dummies Questions & Answers

taking the output of awk command to a new file

cat doc | nawk -v da="${date}" '$23>199 {print $0 > "doc"+da+".txt"}' Every time(need to run every day) i run this, i want to a create a new file "doc_01 Aug.txt". Basically, i want to create a new file with date appended in it. The above command is creating a file with name "0".... (4 Replies)
Discussion started by: vagar11
4 Replies

6. Shell Programming and Scripting

Cannot get the correct ans. Using awk in taking average

Hi all, I think so I’m getting the result is wrong, while using following awk commend, colval=$(awk 'FNR>1 && NR==FNR{a=$4;next;} FNR>1 {a+=$4; print $2"\t"a/3}' filename_f.tsv filename_f2.tsv filename_f3.tsv) echo $colval >> Result.tsv it’s doing the condition 2 times, first result... (5 Replies)
Discussion started by: Shenbaga.d
5 Replies

7. Shell Programming and Scripting

Taking inputs for awk

Hi, i want to print 2nd column value with the below script. I need to take input of the string i need to search in that file and file name. How can i take these two as inputs? using read command? Getting error for below script. echo "enter SID" read SID echo "enter filename" read filename... (8 Replies)
Discussion started by: sam_bd
8 Replies

8. Shell Programming and Scripting

awk command line arguments not taking

# more minusf.awk #!/bin/awk -f BEGIN { FS=":"; } { if ( $2 == "" ) { print $1 ": no password!"; } } # ./minusf.awk aa aa aa aa awk: can't open aa (6 Replies)
Discussion started by: sri.phani
6 Replies

9. Shell Programming and Scripting

awk to select 2D data bins

I wish to use AWK to do something akin: Select all 2D data with 1<$1<2 and -7.5<$2<-6.5 But it's not working awk 'END {print ($1<=2&&$1>=1&&$2<=-6.5&&$2>=-7.5)}' bla Data: -1.06897 -8.04482 -61.469 -1.13613 -8.04482 -61.2271 -1.00182 -8.04482 -61.2081 -1.06897 -8.13518 -60.8544... (2 Replies)
Discussion started by: chrisjorg
2 Replies

10. UNIX for Beginners Questions & Answers

Create bins with totals and percentage

I would like to create bins to get histogram with totals and percentage, e.g. starting from 0. If possible to set the minimum and maximum value in the bins ( in my case value min=0 and max=20 ) Input file 8 5 10 1 11 4 12 4 12 4 13 5 16 7 18 9 16 9 17 7 18 5 19 5 20 1 21 7 (10 Replies)
Discussion started by: jiam912
10 Replies

LEARN ABOUT OPENDARWIN

gzexe

GZEXE(1)						      General Commands Manual							  GZEXE(1)

NAME

       gzexe - compress executable files in place

SYNOPSIS

       gzexe [ name ...  ]

DESCRIPTION

       The  gzexe  utility  allows you to compress executables in place and have them automatically uncompress and execute when you run them (at a
       penalty in performance).  For example if you execute ``gzexe /bin/cat'' it will create the following two files:
	   -r-xr-xr-x  1 root  bin   9644 Feb 11 11:16 /bin/cat
	   -r-xr-xr-x  1 bin   bin  24576 Nov 23 13:21 /bin/cat~
       /bin/cat~ is the original file and /bin/cat is the self-uncompressing executable file.  You can remove /bin/cat~ once  you  are	sure  that
       /bin/cat works properly.

       This utility is most useful on systems with very small disks.

OPTIONS

       -d     Decompress the given executables instead of compressing them.

SEE ALSO

       gzip(1), znew(1), zmore(1), zcmp(1), zforce(1)

CAVEATS

       The  compressed	executable  is a shell script. This may create some security holes. In particular, the compressed executable relies on the
       PATH environment variable to find gzip and some other utilities (tail, chmod, ln, sleep).

BUGS

       gzexe attempts to retain the original file attributes on the compressed executable, but you may have to fix them manually  in  some  cases,
       using chmod or chown.

																	  GZEXE(1)
All times are GMT -4. The time now is 09:37 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy