Monitoring awk script progress


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Monitoring awk script progress
# 1  
Old 06-30-2009
Monitoring awk script progress

Is there a way to have awk output its regular output to a file and some other stuff to a log file to monitor progress?

ie: print first field of a file and for every 100000 lines dealt with, print time in file.log
# 2  
Old 06-30-2009
Code:
awk ' { print $0    
        if(FNR % 1000000 ==0 ) {printf ("processed %d lines", FNR) > "./file.log"  } } ' inputfilename

# 3  
Old 06-30-2009
here's something I've been using for quite some time for 'large' files to entertain a user:
nawk -f myAWK.awk myFile
OR
nawk -v twirle="${USER}" -f myAWK.awk myFile
OR
nawk -v twirle='str' -f myAWK.awk myFile

myAWK.awk:
Code:
#!/usr/bin/nawk -f
#
#---------------------------------------------------------------------------
function makeTwirleS(twirle, sep,     s,twLength,twSlength,i,left,l,r,right,arrN ,arr)
{

    s=twirle
    twLength=length(twirle)
    twSlength=length(twirle) * ((length(twirle) <= 3) ? 5 : 2)

    for(right=twLength+1; right<=twSlength; right++)
       s=s "."

    # left -> right
    for(i=1; i+twLength<=twSlength; i++) {
      l=r=""
      for(left=1; left<=i; left++)
         l=(l=="")? "." : l "."
      for(right=i+twLength+1; right<=twSlength; right++)
         r=(r=="")? "." : r "."
      s=s sep l twirle r
    }

    # reverse the direction - right -> left
    arrN=split(s, arr, sep)
    for(i=arrN-1; i>1; i--)
       s=s sep arr[i]

    return s
}
#---------------------------------------------------------------------------

BEGIN {

# Progress bar
  PROGRESSdiv=1000
  if (!twirle) twirle="*"
  # deal with the TWIRLE
  twirleS=makeTwirleS(twirle, SUBSEP)
  ntwirleA=split(twirleS, twirleA, SUBSEP)

  stderr="cat 1>&2"

}
#---------------------------------------------------------------------------

!( FNR % PROGRESSdiv) {
     printf("\rProgress [%s]-> [%d] lines [%s]", FILENAME, FNR, twirleA[(++_twirle % ntwirleA+1)])  | stderr;
}
#
# here goes the rest of the awk script......
#---------------------------------------------------------------------------

Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell Script and Progress Bar or GUI Wrapper

Hi, I have shell script that I am running under Ubuntu as root. Is it possible to hide the command window and show the user some sort of progress /random progress bar / or other form of GUI interaction? On MAC, I have been using Platypus but on Ubuntu I am not sure what to do. (4 Replies)
Discussion started by: naveedanwar4u
4 Replies

2. Programming

Script to monitor progress of find/exec command

hi all, i want to monitor the progress of a find and exec command, this is the code i use - find . -type f -exec md5sum {} \; >> /md5sums/file.txt this command works and produces a text file with all the md5sums but while running it doesnt show the progress is there anyway i can do this... (4 Replies)
Discussion started by: robertkwild
4 Replies

3. Programming

Program or bash script to see total progress of copy

hi all, i want a program or to make a bash script to find out the total ETA/percent (would be nice aswell a progress bar) of a copy recursive command so lets say i do - cp -r /source_folder/ /destinatation_folder/ and when i run it i get no information on the screen of how the copy is... (20 Replies)
Discussion started by: robertkwild
20 Replies

4. Emergency UNIX and Linux Support

Monitoring progress of a process in Linux server

Hi friends, I am quite used to using glance in HPUX servers for analysis performance issues with a particular process as requested by app. folks. The options which are very helpful to me are the "F" , "W" , "L" stats. How can i get the similar details on Linux servers without using glance?... (2 Replies)
Discussion started by: kunwar
2 Replies

5. Shell Programming and Scripting

progress bar in ftp script

dear all , i made script to transfer data through the FTP but i need to add something if you can help me i want in the output message like progress bar |||||||||||||||||||||||||| 30% to know when it will finish (3 Replies)
Discussion started by: maxim42
3 Replies

6. Shell Programming and Scripting

Progress bar using lsof and awk

Hey all. I added this to my script, hoping for a progress bar or percentage complete lsof -o0 -o -p $$ | awk ' BEGIN { CONVFMT = "%.2f" } $4 ~ /^+r$/ && $7 ~ /^0t/ { offset = substr($7, 3) fname = $9 "stat... (6 Replies)
Discussion started by: midniteslice
6 Replies

7. Shell Programming and Scripting

Urgent: Script.sh: syntax error at line 72: `PROGRESS=$' unexpected

I have written a shell script to Automatically FTP a file. The script runs fine when doing it manually but when I schedule it using a crontab it gives me an error. . . . echo "-----------------Starting File FTP---------------------" >> $PROS_LOAD_LOG echo "open X.XX.XX.XXX" >>... (13 Replies)
Discussion started by: tanhajoy
13 Replies

8. Shell Programming and Scripting

Progress indicator script

Can anybody suggest you a good script to show progress of a process. I figured it out some thing like this. But cursor goes to the end of the line and after every loop it goes to the next line. while true ; do for i in \| \/ \- \\ \| \/ \- \\ do echo "\b\b$i \c" sleep 1 done done (8 Replies)
Discussion started by: ./hari.sh
8 Replies

9. Shell Programming and Scripting

Making a progress gauge in a bash script

Hello once again: One thing that seems to be a nice feature is a progress gauge... so I can see how long an operation will take for a task to complete if it is requiring a lot of processing or the file is enormous. I have seen references to gauge operations, but I don't know anything about it or... (1 Reply)
Discussion started by: ccox85
1 Replies
Login or Register to Ask a Question