Sponsored Content
Top Forums Shell Programming and Scripting Calculation in Multiple files using awk Post 302443000 by Christoph Spohr on Friday 6th of August 2010 07:41:34 AM
Old 08-06-2010
O.k.

Code:
awk -F,  # make "," the new field delimiter
'{gsub(/ */,"",$0); # the format of your data is not consistent
                    # -> delete all blanks
split($0,t,",");    # split the current line at each comma
                    # store the values in an array called t
a[t[2]", "t[3]", "t[4]]+=$1} # now the values in t don't contain any spaces of commas any more
                    # so i construct a new string t[2]", "t[3]", "t[4] of the elements in t
                    # and use it as the name for the hash entry
                    # if this string is encountered again, add 1 to the value associated with the hash name
END{for (i in a) print a[i],i}' # when all files are processed
                    # print out first the number stored in the hash
                    # then the corresponding hash string
file1 file2 file3   # names of the three processed files

This User Gave Thanks to Christoph Spohr For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Splitting input files into multiple files through AWK command

Hi, I needs to split *.txt files from single directory depends on the some mutltiple input values. i have wrote the code like below for file in *.txt do grep -i -h "value1|value2" $file > $file; done. My requirment is more input values needs to be given in grep; let us say 50... (3 Replies)
Discussion started by: arund_01
3 Replies

2. Shell Programming and Scripting

Multiple search string in multiple files using awk

Hi, filenames: contains name of list of files to search in. placelist contains the names of places to be searched in all files in "filenames" for i in $(<filenames) do egrep -f placelist $i if ] then echo $i fi done >> outputfile Output i am getting: (0 Replies)
Discussion started by: pinnacle
0 Replies

3. Shell Programming and Scripting

extract multiple cloumns from multiple files; skip rows and include filenames; awk

Hello, I am trying to write a bash shell script that does the following: 1.Finds all *.txt files within my directory of interest 2. reads each of the files (25 files) one by one (tab-delimited format and have the same data format) 3. skips the first 10 rows of the file 4. extracts and... (4 Replies)
Discussion started by: manishabh
4 Replies

4. UNIX for Dummies Questions & Answers

best method of replacing multiple strings in multiple files - sed or awk? most simple preferred :)

Hi guys, say I have a few files in a directory (58 text files or somthing) each one contains mulitple strings that I wish to replace with other strings so in these 58 files I'm looking for say the following strings: JAM (replace with BUTTER) BREAD (replace with CRACKER) SCOOP (replace... (19 Replies)
Discussion started by: rich@ardz
19 Replies

5. UNIX for Dummies Questions & Answers

Using AWK: Extract data from multiple files and output to multiple new files

Hi, I'd like to process multiple files. For example: file1.txt file2.txt file3.txt Each file contains several lines of data. I want to extract a piece of data and output it to a new file. file1.txt ----> newfile1.txt file2.txt ----> newfile2.txt file3.txt ----> newfile3.txt Here is... (3 Replies)
Discussion started by: Liverpaul09
3 Replies

6. Shell Programming and Scripting

echo multiple variable with calculation

$total=500 echo "scale=2; $val1*100/$total" | bc echo "scale=2; $val2*100*100/$total" | bc echo "scale=2; $val3*100/$total" | bc I want to make the above code to be accomplish in a single echo line. For instance output:21.3, 44.2, 51.6 How to achieve that, some one please help, i just... (5 Replies)
Discussion started by: alvin0618
5 Replies

7. Shell Programming and Scripting

perform 3 awk commands to multiple files in multiple directories

Hi, I have a directory /home/datasets/ which contains a bunch (720) of subdirectories called hour_1/ hour_2/ etc..etc.. in each of these there is a single text file called (hour_1.txt in hour_1/ , hour_2.txt for hour_2/ etc..etc..) and i would like to do some text processing in them. Each of... (20 Replies)
Discussion started by: amarn
20 Replies

8. Shell Programming and Scripting

awk, multiple files input and multiple files output

Hi! I'm new in awk and I need some help. I have a folder with a lot of files and I need that awk do something in each file and print a new file with the output. The input file name should be modified when I print the outpu files. Thanks in advance for help! :-) ciao (5 Replies)
Discussion started by: gabrysfe
5 Replies

9. Shell Programming and Scripting

Multiple Column Calculation

Hallo Team, I need you you help. I need to sum up all the columns in a .csv file. Lets call the file file1.csv and it looks like below: $ cat file1.csv... (2 Replies)
Discussion started by: kekanap
2 Replies

10. Shell Programming and Scripting

awk split and awk calculation in the same command

I am trying to run the awk below. My question is when I split the input, then run anotherawk to perform a calculation using that splitas the input there are no issues. When I try to combine them the output is not correct, is the split not working or did I do it wrong? Thank you :). input ... (8 Replies)
Discussion started by: cmccabe
8 Replies
JOIN(1) 						    BSD General Commands Manual 						   JOIN(1)

NAME
join -- relational database operator SYNOPSIS
join [-a file_number | -v file_number] [-e string] [-j file_number field] [-o list] [-t char] [-1 field] [-2 field] file1 file2 DESCRIPTION
The join utility performs an ``equality join'' on the specified files and writes the result to the standard output. The ``join field'' is the field in each file by which the files are compared. The first field in each line is used by default. There is one line in the output for each pair of lines in file1 and file2 which have identical join fields. Each output line consists of the join field, the remaining fields from file1 and then the remaining fields from file2. The default field separators are tab and space characters. In this case, multiple tabs and spaces count as a single field separator, and leading tabs and spaces are ignored. The default output field separator is a single space character. Many of the options use file and field numbers. Both file numbers and field numbers are 1 based, i.e. the first file on the command line is file number 1 and the first field is field number 1. The following options are available: -a file_number In addition to the default output, produce a line for each unpairable line in file file_number. (The argument to -a must not be preceded by a space; see the COMPATIBILITY section.) -e string Replace empty output fields with string. -o list The -o option specifies the fields that will be output from each file for each line with matching join fields. Each element of list has the form 'file_number.field', where file_number is a file number and field is a field number. The elements of list must be either comma (``,'') or whitespace separated. (The latter requires quoting to protect it from the shell, or, a simpler approach is to use multiple -o options.) -t char Use character char as a field delimiter for both input and output. Every occurrence of char in a line is significant. -v file_number Do not display the default output, but display a line for each unpairable line in file file_number. The options -v 1 and -v 2 may be specified at the same time. -1 field Join on the field'th field of file 1. -2 field Join on the field'th field of file 2. When the default field delimiter characters are used, the files to be joined should be ordered in the collating sequence of sort(1), using the -b option, on the fields on which they are to be joined, otherwise join may not report all field matches. When the field delimiter char- acters are specified by the -t option, the collating sequence should be the same as sort(1) without the -b option. If one of the arguments file1 or file2 is ``-'', the standard input is used. The join utility exits 0 on success, and >0 if an error occurs. COMPATIBILITY
For compatibility with historic versions of join, the following options are available: -a In addition to the default output, produce a line for each unpairable line in both file 1 and file 2. (To distinguish between this and -a file_number, join currently requires that the latter not include any white space.) -j1 field Join on the field'th field of file 1. -j2 field Join on the field'th field of file 2. -j field Join on the field'th field of both file 1 and file 2. -o list ... Historical implementations of join permitted multiple arguments to the -o option. These arguments were of the form ``file_num- ber.field_number'' as described for the current -o option. This has obvious difficulties in the presence of files named ``1.2''. These options are available only so historic shell scripts don't require modification and should not be used. SEE ALSO
awk(1), comm(1), paste(1), sort(1), uniq(1) STANDARDS
The join command is expected to be IEEE Std 1003.2 (``POSIX.2'') compatible. BSD
April 28, 1995 BSD
All times are GMT -4. The time now is 07:15 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy