Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: awk - saving results of external script to variable.
Top Forums Shell Programming and Scripting awk - saving results of external script to variable. Post 302956005 by treesloth on Thursday 24th of September 2015 02:55:09 PM
Old 09-24-2015
Thanks for the reply. The input I have is just a very simple CSV that I grabbed for testing. A sample:

Code:
00102,20150501,0001,0,CLR, ,10.00, , , ,24, ,-4.4, ,23, ,-4.9, ,21, ,-6.1, , 88, , 0, ,000, , , ,29.70, , , , , ,30.01, ,AA, , , ,29.88, 
00102,20150501,0101,0,CLR, ,10.00, , , ,22, ,-5.6, ,21, ,-6.0, ,19, ,-7.2, , 88, , 0, ,000, , , ,29.71, , , , , ,30.02, ,AA, , , ,29.89, 
00102,20150501,0201,0,CLR, ,10.00, , , ,19, ,-7.2, ,18, ,-7.5, ,17, ,-8.3, , 92, , 0, ,000, , , ,29.72, , , , , ,30.03, ,AA, , , ,29.90,


The processor scripts that are called by the distributor just go through every field of every line, and if the field contains a positive integer it sums it.

Code:
    {
    for (i = 1; i <= NF; i++)
        {
        if ($i ~ /^[0-9]+$/)
            {
            SUM += $i
            }
        }
    }
END {
    printf "%.0f\n",  SUM
    }

It's a weird, arbitrary task, I know. Its only purpose is to test the distributor/processor idea. The distributor that calls the script above is already posted. Its output looks like:

25280650967054
25280650651146
25280650873710
25280650942866
25280650844079
25280650677879
25280671120373

Each is simply the output from each of, in this case, the 7 processor scripts that were called. So, I would get that as the output of:

Code:
mawk -F , -v count=7 'NR > 1 {mod = NR%count + 1 ; proc = "./proc." mod ".mawk -F ,"; print $0 | proc}' datafile.csv

Currently, I pipe that to:

Code:
awk '{SUM += $1} END {print SUM}'

I'd like to eliminate that pipe and have the distributor receive back each processor's results and sum them itself. It might not be readily doable, but I figured someone here might have an idea. Thank you.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

store awk results in a variable

I try to get the month (of last save) and the filename into a variable, is this possible ? something like this : for month in `ls -la | awk '{print $6}'` do if ] then a=filename of the matching file cp $a /Sep fi thanks, Steffen (1 Reply)
Discussion started by: forever_49ers
1 Replies

2. Shell Programming and Scripting

awk help (external variable)

i need help with an awk question. i am looking to have an external variable be defined outside of awk but used in awk. so if we have fields $1, $2, $3 so on and so forth, i would like to be able to dictate what field is being printed by something like $. so if i had a counter called test, make it 3... (8 Replies)
Discussion started by: pupp
8 Replies

3. Shell Programming and Scripting

Saving output from awk into a perl variable

How would I pass awk output to a perl variable? For example, I want to save the value in the 4th column into the variable called test. My best guess is something as follow, but I am sure this isn't correct. $test = system("awk '/NUMBER/{print \$4}' $_"); (8 Replies)
Discussion started by: userix
8 Replies

4. Shell Programming and Scripting

If statement in awk with external variable

So I have a if statement inside an awk to check if $2 of a awk equals a specific IP but the test fails. So here is what I have. # !/bin/sh echo "Enter client ID" read ID echo "Enter month (01, 02, 03)" read month echo "Enter day (03, 15)" read day echo "Enter Year (07, 08)" read... (6 Replies)
Discussion started by: doublejz
6 Replies

5. Shell Programming and Scripting

saving awk value in a bash array variable

hi all i am trying to save an awk value into an array in bash: total=`awk '{sum+=$3} END {print sum}' "$count".txt"` ((count++)) the above statement is in a while loop.. $count is to keep track of file numbers (1.txt,2.txt,3.txt,etc.) i get the following error: ./lines1:... (1 Reply)
Discussion started by: npatwardhan
1 Replies

6. Shell Programming and Scripting

Insert external variable in a AWK pattern

Dear all, ¿How can i insert a variable in a AWK pattern? I have almost succeeded in solving a puzzle with AWK but now i want to make a script. Let me explain. cat file.txt | awk 'BEGIN {RS="\\n\\n"} /tux/ { print "\n"$0 }' I know that this command makes right what i want to do, but mi... (8 Replies)
Discussion started by: antuan
8 Replies

7. Shell Programming and Scripting

[awk] - how to insert an external variable

I want to incorporate the variable in the for statement as a column of my processed file. In the INCORRECT example below, it is $i which corresponds to the i in my for loop: for i in x86_64 i686; do awk '{ print $1" "$4" "$5" "$i }'awk $file-$i > processed-$i.log doneThanks! (3 Replies)
Discussion started by: graysky
3 Replies

8. Shell Programming and Scripting

Help needed with awk external variable

I'm trying to get the universities result data into different file, where the $9 contains unversity field and field7,4 & 5 contains the keys to sort the students by marks. How to use uni variable to match against $9 inside awk? c=0 for uni in `cat /tmp/global_rank| awk -F ',' '{print... (1 Reply)
Discussion started by: InduInduIndu
1 Replies

9. Shell Programming and Scripting

Passing external variable to awk

Hi, I am trying to write a bash script in which I need to pass a external variable to the awk program. I tired using -v but it not accepting the value. Here is my sample code. #!/usr/bin/bash ###################################################################################### ####... (5 Replies)
Discussion started by: jpkumar10
5 Replies

10. Shell Programming and Scripting

I want to add a variable for the results from the formula of one variable and results of another var

Good morning all, This is the file name in question OD_Orders_2019-02-19.csv I am trying to create a bash script to read into files with yesterdays date on the file name while retaining the rest of the files name. I would like for $y to equal, the name of the file with a formula output with... (2 Replies)
Discussion started by: Ibrahim A
2 Replies

LEARN ABOUT DEBIAN

igawk

IGAWK(1)							 Utility Commands							  IGAWK(1)

NAME

       igawk - gawk with include files

SYNOPSIS

       igawk [ all gawk options ] -f program-file [ -- ] file ...
       igawk [ all gawk options ] [ -- ] program-text file ...

DESCRIPTION

       Igawk is a simple shell script that adds the ability to have ``include files'' to gawk(1).

       AWK programs for igawk are the same as for gawk, except that, in addition, you may have lines like

	      @include getopt.awk

       in your program to include the file getopt.awk from either the current directory or one of the other directories in the search path.

OPTIONS

       See gawk(1) for a full description of the AWK language and the options that gawk supports.

EXAMPLES

       cat << EOF > test.awk
       @include getopt.awk

       BEGIN {
	    while (getopt(ARGC, ARGV, "am:q") != -1)
		 ...
       }
       EOF

       igawk -f test.awk

SEE ALSO

       gawk(1)

       Effective AWK Programming, Edition 1.0, published by the Free Software Foundation, 1995.

AUTHOR

       Arnold Robbins (arnold@skeeve.com).

Free Software Foundation					    Nov 3 1999								  IGAWK(1)
All times are GMT -4. The time now is 05:44 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy