Sponsored Content
Top Forums Shell Programming and Scripting Pass an array to awk to sequentially look for a list of items in a file Post 303033591 by LMHmedchem on Monday 8th of April 2019 10:04:57 PM
Old 04-08-2019
Quote:
Originally Posted by vgersh99
something to start with and improve upon...
assuming all the statistics are displayed in the same order in a file AND across the files:
awk -f lmh.awk myFiles where lmh.awk is:
Code:
FNR==1 {if (stat) print file OFS stat; file=FILENAME;stat=""; next}
!/statistics/{
   stat=(!stat)? $2:stat OFS $2
}
END {
   print file OFS stat
}

Thank you for the reply but I don't see in the above how my stats will be found in the huge stats output file when there is no notation of how to find what I am looking for. The stats I need are the second field of the first 5 lines following "train statistics", etc. I don't see how you can find what I am looking for without "train statistics" being in there somewhere. I could post an actual file if that would help.

This version works more or less. It prints the stats I need to the logfile.
Code:
#!/bin/sh

# name of file being processed
STATS_FILE=$1
# file we are writing to
LOGFILE=$1

# 4 sets of labels we are looking for
LABELS='train_statistics,test_statistics,validate_statistics,ival_statistics'

cat $STATS_FILE | \
awk -v var="$LABELS" '                 BEGIN { split(var,label_array,","); pos = 1 }
                                      F == 1 { line_array[++a_count] = $2; line_count++ }
                             line_count == 5 { for(i=1; i<=a_count; i++) print line_array[i];
                                               delete line_array;
                                               a_count = 0;  
                                               F = 0;
                                               line_count = 0;
                                               pos++ }
                       $0 ~ label_array[pos] { F = 1; line_count = 0; }
                     ' > $LOGFILE

I need to format the output a bit better and trap for if pos is larger that the size of label_array. I suspect that I also don't need both a_count and line_count.

Is there anything badly wrong with this approach?

LMHmedchem
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Pass array variabel to awk from shell

Hi I need to pass an array to Awk script from Shell. Can you please tell how to do it? How to pass this array add_ct_arr to an awk script or access it in awk? i=1 while ; do add_ct_arr=$(echo ${adda_count} | awk -v i=$i -F" " '{print $i;}') echo ${add_ct_arr} ... (1 Reply)
Discussion started by: appsguy616
1 Replies

2. Shell Programming and Scripting

hw to insert array values sequentially in a file

Hi All :), I am very new to unix. I am requiring ur help in developing shell script for below problem. I have to replace the second field of file with values of array sequentially where first field is ValidateKeepVar <File> UT-ExtractField 1 | &LogEntry &Keep(DatatoValidate)... (3 Replies)
Discussion started by: rohiiit.sharma
3 Replies

3. Shell Programming and Scripting

awk between items including items

OS=HP-UX ksh The following works, except I want to include the <start> and <end> in the output. awk -F '<start>' 'BEGIN{RS="<end>"; OFS="\n"; ORS=""} {print $2} somefile.log' The following work in bash but not in ksh sed -n '/^<start>/,/^<end>/{/LABEL$/!p}' somefile.log (4 Replies)
Discussion started by: Ikon
4 Replies

4. Shell Programming and Scripting

Pass awk array variable to shell

Hi, all suppose I have following myfile (delimited by tab) aa bb cc dd ee ffand I have following awk command: awk 'BEGIN{FS="\t"}{AwkArrayVar_1=$1;AwkArrayVar_2=$2};END{for(i=0; i<NR; i++) print i, AwkArrayVar_1, AwkArrayVar_2,}' myfileMy question is: how can I assign the awk array... (7 Replies)
Discussion started by: littlewenwen
7 Replies

5. Shell Programming and Scripting

[Solved] awk command to read sequentially from a file until last record

Hello, I have a file that looks like this: Generated geometry (...some special descriptor) 1 0.56784 1.45783 -0.87965 8 1.29873 -0.8767 1.098789 ... ... ... ... Generated geometry (....come special descriptor) ... .... ... ... ... ... ... ... and... (4 Replies)
Discussion started by: jaldo0805
4 Replies

6. Shell Programming and Scripting

How to pass an array containing file names to a sftp script?

hi, i want to pass an array parameters to a sftp script so that i can transfer each file in the array to the remote server by connecting only once to the sftp remote server. i thought of using a variable that contains list of file names separated by a space and pass the variable to the sftp... (3 Replies)
Discussion started by: Little
3 Replies

7. Shell Programming and Scripting

Split list of files into an array and pass to function

There are two parts to this. In the first part I need to read a list of files from a directory and split it into 4 arrays. I have done that with the following code, # collect list of file names STATS_INPUT_FILENAMES=($(ls './'$SET'/'$FOLD'/'*'in.txt')) # get number of files... (8 Replies)
Discussion started by: LMHmedchem
8 Replies

8. Shell Programming and Scripting

sed to delete items in an array from a file

I need to create a shell script to delete multiple items (Strings) at a time from a file. I need to iterate through a list of strings. My plan is to create an array and then iterate through the array. My code is not working #!/bin/bash -x declare -a array=(one, two, three, four)... (5 Replies)
Discussion started by: bash_in_my_head
5 Replies

9. Shell Programming and Scripting

Read a lis, find items in a file from the list, change each item

Hello, I have some tab delimited text data, file: final_temp1 aname val NAME;r'(1,) 3.28584 r'(2,)<tab> NAME;r'(3,) 6.13003 NAME;r'(4,) 4.18037 r'(5,)<tab> You can see that the data is incomplete in some cases. There is a trailing tab after the first column for each incomplete row. I... (2 Replies)
Discussion started by: LMHmedchem
2 Replies

10. Shell Programming and Scripting

Script to process a list of items and uncomment lines with that item in a second file

Hello, I have a src code file where I need to uncomment many lines. The lines I need to uncomment look like, C CALL l_r(DESNAME,DESOUT, 'Gmax', ESH(10), NO_APP, JJ) The comment is the "C" in the first column. This needs to be deleted so that there are 6 spaces preceding "CALL".... (7 Replies)
Discussion started by: LMHmedchem
7 Replies
wmnet(1x)																 wmnet(1x)

NAME
wmnet - an IP accounting monitoring tool SYNOPSIS
wmnet [-h,--help] [-v,--version] [-T,--txrule=NUM] [-R,--rxrule=NUM] [-l,--logscale] [-t,--txcolor=COLOR] [-r,--rxcolor=COLOR] [-d DELAY] [-x,--maxrate=BYTES] [-F,--labelfg=COLOR] [-B,--labelbg=COLOR] [-L,--label=LABEL] [-e,--execute=COMMAND] [-p,--promisc=DEVICE] [-u,--unpromisc=DEVICE] [-w,--withdrawn | -n,--normalstate] [-D,--driver=DRIVER] [-W,--device=DEVICE] DESCRIPTION
wmnet polls network statistics and does a few things with the data it gets. It has small blinking lights for the rx and tx of IP packets, a digital speedometer of your networks current speed and a bar graph like xload plotting your throughput. It has a tx speed graph from bot- tom-up and rx speed graph from the top-down. The speedometer keeps track of the current speed per second and shows it in a color corre- sponding to which of rx or tx that has the highest speed at the moment. Also, the graph is drawn in a way that the highest speed is drawn on top of the other while the other is in the background. OPTIONS -h,--help displays a brief help message -v,--version displays version information -T,--txrule=NUM or NAME in the case of the ipfwadm driver, this is the accounting rule number to monitor for tx. For the ipchains driver, this is the chain name to watch. -R,--rxrule=NUM or NAME in the case of the ipfwadm driver, this is the accounting rule number to monitor for rx. For the ipchains, this is the chain name to watch. -t,--txcolor=COLOR specifies the tx color -r,--rxcolor=COLOR specifies the rx color -x,--maxrate=BYTES maximum transfer rate for graph scale. Defaults to 6000, which should be in the right area for modem connections. The key is to experiment with this setting and the --logscale option to get the kind of graph that fits your connection type. A general rule of thumb is to set this to 4 to 5 times greater than your maximum throughput. The author finds using --logscale and --maxrate=10000000 to work nicely for the entire range of his dorms ethernet based connection to the internet. -l,--logscale sets logarithmic scale, which is good for fast connections. This will allow, for example, the graph still being informative at extremely low speeds (telnet), and extremely fast speeds (local FTP) simultaneously without the scale constantly being blank or solid at those respective extremes. -L,--label=LABEL prints a given text label on the bottom of the window -F,--labelfg=COLOR specifies the color for the text of the label -B,--labelbg=COLOR specifies the color for the background of the label text --withdrawn --normalstate sets the initial state of wmnet. WMnet tries to automatically determine which state to start up in by starting up in withdrawn state if a WindowMaker defined atom is present, and in normalstate otherwise. This behavior is overriden by specifying one of these options. -e,--execute=COMMAND executes COMMAND on a single click from button 1 (left mouse button). -u,--unpromisc=DEVICE -p,--promisc=DEVICE put DEVICE in promiscuous mode to start applying accounting rules to all network packets on your network segment. You either need to be root or have the wmnet binary suid root to use this feature. This option may be given more than once on the command line to specify more than one device. -d DELAY delay time for polling /proc/net/ip_account (in microseconds). Defaults to 25000, that is 0.025 seconds, or 40 Hz -D,--driver=DRIVER use DRIVER to get the stats we monitor. Compiled in drivers can be listed with the -h switch. -W,--device=DEVICE watch statistics for DEVICE . This option is only used for certain stat drivers, namely: kmem, devstats, and pppstats. The ipchains and ipfwadm stat drivers do not use this parameter. STAT DRIVERS
wmnet uses different stat drivers to get the stats it needs to monitor your network. Exactly what drivers are available is determined at compile time. The driver wmnet ultimately uses at runtime is dependent on your system. There are 4 drivers specific to Linux and 1 to *BSD. The driver used can be overridden by the --driver option. The available drivers are pppstats, devstats, ipfwadm, ipchains and kmem. pppstats this driver works on Linux 2.0 or Linux 2.1 for ONLY ppp type devices. Specify the --device option for the interface to monitor. By default it uses interface ppp0. Please note, that if the ppp device is not available or active, wmnet will continue to try in the hopes that it is only temporarily offline. devstats use this driver on Linux 2.1 kernels for any interface. Pass the --device option for the device you want monitored, otherwise, the default is eth0. This will be available for ONLY Linux 2.1 kernels and will always be there on those kernels. ipfwadm use this driver on Linux 2.0 kernels compiled with IP accounting. It won't work on Linux 2.1. You'll also need to specify the --txrule and --rxrule options. By default, wmnet uses the first two rules it finds. ipchains this driver will only work in Linux 2.1 kernels with IP chains compiled in. You'll want to also specify the --txrule and --rxrule options and specify the chain names. By default it uses the chains "acctin" and "acctout" There must be at least one rule on the named ipchain, if there is more than one rule in the specified chain, it uses the first. The chain must not immediately return to the parent chain, it has to pass through a rule first. Otherwise, the kernel will not collect the stats we need. kmem this driver is available on FreeBSD and OpenBSD systems and must be passed a device through the --device option. By default, it uses ec0 but will accept any valid device name. FILES
/proc/net/ip_acct /proc/net/dev /proc/net/ipchains kernel net accounting information AUTHORS
wmnet was created by Jesse B. Off <joff@iastate.edu> and is maintained by Katharine Osborne <kaos@digitalkaos.net>. This manpange was originally written by Marcelo Magallon <mmagallo@debian.org> for the Debian Project, and is GNU Copyright 1998 Marcelo Magallon and later modifed by Jesse Off and Katharine Osborne for WMnet versions 1.05 and above. SEE ALSO
wmaker(1x) 4 May 2000 wmnet(1x)
All times are GMT -4. The time now is 09:33 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy