Sponsored Content
Top Forums Shell Programming and Scripting awk for streaming data - if/then Post 303004260 by jm4smtddd on Thursday 28th of September 2017 02:58:12 PM
Old 09-28-2017
awk for streaming data - if/then

Hello,

I've written an awk one-liner to parse a stream of real-time data. I invoke a program that gives me output from which I extract a few columns, perform some simple calculations, and then stream that data into a portal using an http string. It's tricky because I have to run it every second to parse the data I need, but it works.

My question is about adding in an if/then statement. I need to check if the output of Ntripclient2.py actually has the data I need. If it doesn't I need to move on, not parse it, and run the awk one-liner again until I actually get the data I need. The problem is I end up parsing an error message, and it bogs down my portal.

I need to check NR==13, if($5==0) skip the parse step and keep going. Does that make sense?

Here's the one-liner:

Code:
Ntripclient2.py --user $1 $2.unavco.org $3 $4_RTX | awk -F, 'NR>7 {cmd="curl \42http://'$5'.chordsrt.com/measurements/url_create?instrument_id='$6'&lat="substr($5,1,2)+(substr($5,3)/60)"&lon="substr($7,1,3)+(substr($7,4)/60)"&height="substr($12,4)"&at=""20"substr($4,5,2)"-"substr($4,1,2)"-"substr($4,3,2)"T"substr($3,1,2)":"substr($3,3,2)":"substr($3,5,2)"\42";system(cmd)}'



Moderator's Comments:
Mod Comment Please use CODE tags correctly as required by forum rules!

Last edited by RudiC; 09-28-2017 at 04:00 PM.. Reason: Changed ICODE to CODE tags.
 

4 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

save streaming video data for later viewing?

I have limited bandwidth in my apartment and no cable TV service. I was wondering if it is possible to write a script that would download the video data from my favorite TV shows (legally... many networks post their videos online for free) while I sleep, so that later I can watch them on my TV... (5 Replies)
Discussion started by: euclid3628800
5 Replies

2. Shell Programming and Scripting

Help with parsing data with awk , eliminating unwanted data

Experts , Below is the data: --- Physical volumes --- PV Name /dev/dsk/c1t2d0 VG Name /dev/vg00 PV Status available Allocatable yes VGDA 2 Cur LV 8 PE Size (Mbytes) 8 Total PE 4350 Free PE 2036 Allocated PE 2314 Stale PE 0 IO Timeout (Seconds) default --- Physical volumes ---... (5 Replies)
Discussion started by: rveri
5 Replies

3. Shell Programming and Scripting

awk --> math-operation in data-record and joining with second file data

Hi! I have a pretty complex job - at least for me! i have two csv-files with meassurement-data: fileA ...... (2 Replies)
Discussion started by: IMPe
2 Replies

4. Shell Programming and Scripting

Execute awk output with continuous streaming input

I need to parse some continuous output from a program (i.e. aScript.py) into a portal that uses curl. I've written a simple awk one-liner to parse the information that is output from aScript.py, but I'm not able to execute it. I can succeed with just one line of the output from aScript.py: echo... (2 Replies)
Discussion started by: jm4smtddd
2 Replies
mailq(1)							   User Commands							  mailq(1)

NAME
mailq - print the mail queue SYNOPSIS
/usr/bin/mailq [-Ac] [-q subarg] [-v] DESCRIPTION
The mailq utility displays a summary of the mail messages queued for future delivery. The first line displayed for each mail message shows the internal identifier used on this host for the message, the size of the message in bytes, the date and time the message was accepted into the queue, and the envelope sender of the message. The second line of the display shows the error message that caused this message to be retained in the queue. This line will not be displayed if the message is being pro- cessed for the first time. The mailq utility used to be identical to sendmail -bp. Now it checks for the authorization attribute, solaris.mail.mailq. If the check for the invoking user succeeds, sendmail -bp is executed with the remaining argument vector. Otherwise, an error message is printed. This authorization attribute is by default enabled for all users. It can be disabled by modifying the Basic Solaris User entry in prof_attr(4). OPTIONS
The following options are supported: -Ac Like sendmail(1M), this flag tells mailq to use submit.cf rather than sendmail.cf even if the operation mode does not indicate an initial mail submission. This will result in the client queue /var/spool/clientmqueue being displayed rather than the default server queue /var/spool/mqueue. -qp[time] Similar to -qtime, except that instead of periodically forking a child to process the queue, sendmail forks a single per- sistent child for each queue that alternates between processing the queue and sleeping. The sleep time is given as the argument. The sleep time default is 1 second. The process will always sleep at least 5 seconds if the queue was empty in the previous queue run. -qf Processes saved messages in the queue once and does not fork(), but runs in the foreground. -qG name Processes jobs in the queue group called name only. -q[!]I substr Limits processed jobs to those containing substr as a substring of the queue id, or not when ! is specified. -q[!]R substr Limits processed jobs to those containing substr as a substring of one of the recipients, or not when ! is specified. -q[!]S substr Limits processed jobs to those containing substr as a substring of the sender, or not when ! is specified. -v Prints verbose information. This adds the priority of the message and a single character indicator (+ or blank) indicating whether a warning message has been sent on the first line of the message. Additionally, extra lines may be intermixed with the recipients that indicate the "controlling user" information. This shows who will own any programs that are executed on behalf of this message and the name of the alias this command is expanded from, if any. EXIT STATUS
0 Successful completion. >0 An error occurred. FILES
/etc/security/prof_attr local source for execution profile attributes /var/spool/mqueue default server queue /var/spool/clientmqueue client queue ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWsndmu | +-----------------------------+-----------------------------+ SEE ALSO
sendmail(1M), prof_attr(4), attributes(5) SunOS 5.11 10 Jul 2002 mailq(1)
All times are GMT -4. The time now is 11:29 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy