awk multiple files Post: 302339336

Sponsored Content

Top Forums UNIX for Dummies Questions & Answers awk multiple files Post 302339336 by philstar on Thursday 30th of July 2009 09:20:37 AM

07-30-2009

Registered User

awk multiple files

Hi there!
I'm with a problem because I'm working on a script to do a calculation of a mean using multiple files with 3 columns and N-lines.
My input are like this

File1

Code:

0001 0023  5
0120 0376  6
0412 3412  3
...

File2

Code:

0001 0023  3
0120 0376  nan
0412 3412  7
...

File3

Code:

...

Notice that all the files have the same 2 first columns and only the 3d is variable. I need to read all the files in a row and make the mean of the 3d column for every line.
The nan is "not a number" so it can't go on the calculus.

Output

Code:

0001 0023  4
 0120 0376  6
 0412 3412  5

can Anyone give me a help?

I'm trying somthing like this:

Code:

for i in $(seq 1 3); do awk -f t5.awk -v facc=acc$((i-1)).txt t$i.dat > acc$i.txt; rm acc$((i-1)).txt; done

(in the shell)

and the script

Code:

awk '{
#criar o vector
BEGIN {
#variavel de contagem
i = 1
#ciclo que vai buscar a variavel facc
#cada linha � indexada ao vector a
while ((getline<facc)>0) {
    a[i]=$0
    i++
}
i = 1
soma +=$3
}
{ print a[i] " " soma
  i++
}'

but this is for bilding a matrix of all the columns in $3 and that doest help because it creates a file of ~~5G and I dont have that kind of resourses!

Use CODE-tags next time you post code, data or logs to enhance readability and to preserve formatting like indention etc., ty.

Last edited by zaxxon; 07-30-2009 at 11:41 AM..

philstar

View Public Profile for philstar

Find all posts by philstar

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Splitting input files into multiple files through AWK command

Hi, I needs to split *.txt files from single directory depends on the some mutltiple input values. i have wrote the code like below for file in *.txt do grep -i -h "value1|value2" $file > $file; done. My requirment is more input values needs to be given in grep; let us say 50...

2. Shell Programming and Scripting

awk with multiple files

I have 2 files first is category: 1:Phone 2:Keyboard 3:Printer 4:Scanner 5:Mouse the second is product: 1:iphone:1:..... 2:blackberry:1:..... 3:Mitsumi:2:..... 4:abc:5:..... 5:def:4:..... 6:noi:3:..... which bold numbers are "foreign key" in product file and they are "primary...

3. Shell Programming and Scripting

Multiple search string in multiple files using awk

Hi, filenames: contains name of list of files to search in. placelist contains the names of places to be searched in all files in "filenames" for i in $(<filenames) do egrep -f placelist $i if ] then echo $i fi done >> outputfile Output i am getting:

4. Shell Programming and Scripting

extract multiple cloumns from multiple files; skip rows and include filenames; awk

Hello, I am trying to write a bash shell script that does the following: 1.Finds all *.txt files within my directory of interest 2. reads each of the files (25 files) one by one (tab-delimited format and have the same data format) 3. skips the first 10 rows of the file 4. extracts and...

5. UNIX for Dummies Questions & Answers

best method of replacing multiple strings in multiple files - sed or awk? most simple preferred :)

Hi guys, say I have a few files in a directory (58 text files or somthing) each one contains mulitple strings that I wish to replace with other strings so in these 58 files I'm looking for say the following strings: JAM (replace with BUTTER) BREAD (replace with CRACKER) SCOOP (replace...

6. UNIX for Dummies Questions & Answers

Using AWK: Extract data from multiple files and output to multiple new files

Hi, I'd like to process multiple files. For example: file1.txt file2.txt file3.txt Each file contains several lines of data. I want to extract a piece of data and output it to a new file. file1.txt ----> newfile1.txt file2.txt ----> newfile2.txt file3.txt ----> newfile3.txt Here is...

7. UNIX for Dummies Questions & Answers

awk in multiple files

Hello all, I have two files: one that looks like this string1 string2 string3 string6 and another that looks like this Wstring1 apple Wstring2 vegetable Wstring3 mouse Wstring4

8. Shell Programming and Scripting

perform 3 awk commands to multiple files in multiple directories

Hi, I have a directory /home/datasets/ which contains a bunch (720) of subdirectories called hour_1/ hour_2/ etc..etc.. in each of these there is a single text file called (hour_1.txt in hour_1/ , hour_2.txt for hour_2/ etc..etc..) and i would like to do some text processing in them. Each of...

9. Shell Programming and Scripting

Awk with Multiple files

Hello, I wanted to know the best way to do some matching between two files. I am currently using awk. Example Question: If column 4 from file A is contained in column 1 from file B. Print the whole row from file A of mix of information from both files. i.e. File A ...

10. Shell Programming and Scripting

awk, multiple files input and multiple files output

Hi! I'm new in awk and I need some help. I have a folder with a lot of files and I need that awk do something in each file and print a new file with the output. The input file name should be modified when I print the outpu files. Thanks in advance for help! :-) ciao

LEARN ABOUT MINIX

ppmquantall

ppmquantall(1)                                                General Commands Manual                                               ppmquantall(1)

NAME

       ppmquantall - run ppmquant on a bunch of files all at once, so they share a common colormap

SYNOPSIS

       ppmquantall [-ext extension] ncolors ppmfile ...

DESCRIPTION

       Takes a bunch of portable pixmap as input.  Chooses ncolors colors to best represent all of the images, maps the existing colors to the new
       ones, and overwrites the input files with the new quantized versions.

       If you don't want to overwrite your input files, use the -ext option.  The output files are then named the same as the input files, plus  a
       period and the extension text you specify.

       Verbose  explanation:  Let's  say  you've got a dozen pixmaps that you want to display on the screen all at the same time.  Your screen can
       only display 256 different colors, but the pixmaps have a total of a thousand or so different colors.  For a single pixmap you  solve  this
       problem  with  ppmquant;  this  script  solves  it for multiple pixmaps.  All it does is concatenate them together into one big pixmap, run
       ppmquant on that, and then split it up into little pixmaps again.

       (Note that another way to solve this problem is to pre-select a set of colors and then use ppmquant's -map option  to  separately  quantize
       each pixmap to that set.)

SEE ALSO

       ppmquant(1), ppm(5)

AUTHOR

       Copyright (C) 1991 by Jef Poskanzer.

                                                                   27 July 1990                                                     ppmquantall(1)

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Splitting input files into multiple files through AWK command

Discussion started by: arund_01

2. Shell Programming and Scripting

awk with multiple files

Discussion started by: WuZun

3. Shell Programming and Scripting

Multiple search string in multiple files using awk

Discussion started by: pinnacle

4. Shell Programming and Scripting

extract multiple cloumns from multiple files; skip rows and include filenames; awk

Discussion started by: manishabh

5. UNIX for Dummies Questions & Answers

best method of replacing multiple strings in multiple files - sed or awk? most simple preferred :)

Discussion started by: rich@ardz

6. UNIX for Dummies Questions & Answers

Using AWK: Extract data from multiple files and output to multiple new files

Discussion started by: Liverpaul09

7. UNIX for Dummies Questions & Answers

awk in multiple files

Discussion started by: FelipeAd

8. Shell Programming and Scripting

perform 3 awk commands to multiple files in multiple directories

Discussion started by: amarn

9. Shell Programming and Scripting

Awk with Multiple files

Discussion started by: rafir

10. Shell Programming and Scripting

awk, multiple files input and multiple files output

Discussion started by: gabrysfe

LEARN ABOUT MINIX

ppmquantall

Featured Tech Videos