Execute awk output with continuous streaming input


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Execute awk output with continuous streaming input
# 1  
Old 03-09-2016
Execute awk output with continuous streaming input

I need to parse some continuous output from a program (i.e. aScript.py) into a portal that uses curl. I've written a simple awk one-liner to parse the information that is output from aScript.py, but I'm not able to execute it. I can succeed with just one line of the output from aScript.py: echo curl "awk output" | xargs curl , but this doesn't work for continuous output from the aScript.py program. Here is what I want to be able to do:

Code:
 aScript.py --user aUser:userPswd server.org PORT CODE | awk -F, 'NR>6 {print "curl \42http://aServerSomewhere.com/measurements/url_create?instrument_id=1&VAR1="substr($5,1,2)substr($5,5)"&VAR2="substr($7,1,3)"&VAR3="substr($12,4)"\42"}' | execute awk output (what to do here?)

In this example aScript.py is extracting data that is continuously collected. My awk line is parsing the output into a line necessary for porting the data into a program. How can I get the output of my awk line to execute continuously as the data comes in? Perhaps a loop is necessary?

Solved with:

Code:
 aScript.py --user aUser:userPswd server.org PORT CODE | awk -F, 'NR>6 {cmd="curl \42http://aServerSomewhere.com/measurements/url_create?instrument_id=1&VAR1="substr($5,1,2)substr($5,5)"&VAR2="substr($7,1,3)"&VAR3="substr($12,4)"\42";system(cmd)}'

Moderator's Comments:
Mod Comment Please use CODE tags (not ICODE tags) for full-line and multi-line code segments, sample input, and sample output. And, not that opening tags need matching close tags.

Last edited by Don Cragun; 03-10-2016 at 02:48 AM.. Reason: Fix tags.
# 2  
Old 03-10-2016
Have your awk output commands for the shell and pipe that into sh.
# 3  
Old 03-10-2016
Quote:
Originally Posted by cjcox
Have your awk output commands for the shell and pipe that into sh.

This one doesn't work because, it seems, that sh will only execute once the output from my aScript.py is finished. aScript.py runs continuously. I found the solution with the following:

Code:
 aScript.py --user aUser:userPswd server.org PORT CODE | awk -F, 'NR>6 {cmd="curl \42http://aServerSomewhere.com/measurements/url_create?instrument_id=1&VAR1="substr($5,1,2)substr($5,5)"&VAR2="substr($7,1,3)"&VAR3="substr($12,4)"\42";system(cmd)}'

Login or Register to Ask a Question

Previous Thread | Next Thread

6 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk for streaming data - if/then

Hello, I've written an awk one-liner to parse a stream of real-time data. I invoke a program that gives me output from which I extract a few columns, perform some simple calculations, and then stream that data into a portal using an http string. It's tricky because I have to run it every second... (2 Replies)
Discussion started by: jm4smtddd
2 Replies

2. UNIX for Dummies Questions & Answers

Redirect output to the same input file in awk

Hi, I want to compare a value from test file and redirect the o/p value to the same file input file 250 32000 32 128 Below is my code awk '{ if ($1 < "300") print $1 > /tmp/test}' test want to compare 250 < 300 then print 300 to the same place below is the... (24 Replies)
Discussion started by: stew
24 Replies

3. Shell Programming and Scripting

redirect an awk string output to a script input with pipes

Hi, I have a function in a bash script that returns a string after some operations using awk. The following code returns 555 $VARIABLE="EXAMPLE" get_number $VARIABLE this value I'd like to pass it as a second argument of another script with the following usage myscript.sh <param1>... (7 Replies)
Discussion started by: rid
7 Replies

4. Shell Programming and Scripting

Printing the continuous occurence of pattern using awk ?

Hello all, I have a input file like this. input file --------------- abc ab001 + ab002 zca acb ab006 + ab007 caz cba ab003 + ab004 zca bac ab004 - ab005 zac bca ab002 - ab003 cza cba ab005 + ab006 acz cba ab005 ... (5 Replies)
Discussion started by: admax
5 Replies

5. Shell Programming and Scripting

Pass input and output file as parameter to awk script

Hi, i am new to awk. I am using csv2pipe script(shown below) BEGIN { FS=SUBSEP; OFS="|" } { result = setcsv($0, ",") print } # setcsv(str, sep) - parse CSV (MS specification) input # str, the string to be parsed. (Most likely $0.) # sep, the separator between the values. # #... (6 Replies)
Discussion started by: bhaskarjha178
6 Replies

6. Shell Programming and Scripting

awk should output if one input file doesnt have matching key

nawk -F, 'FNR==NR{a= $3 ;next} $2 in a{print $1, 'Person',$2, a}' OFS=, filea fileb Input filea Input fileb output i am getting : (2 Replies)
Discussion started by: pinnacle
2 Replies
Login or Register to Ask a Question