Sponsored Content
Top Forums Shell Programming and Scripting adding counter to a variable while moving in a loop Post 302459406 by DGPickett on Monday 4th of October 2010 03:13:20 PM
Old 10-04-2010
Yes, fork/exec was too costly on your attempt, but for big data sets, bash may be slower than a multi-process script of more specialized tools. You can use multiple sed's to spread the load, divert reject records to separate files while streaming numbers turned into a bc script to stdout | bc, then just 'wc -l <file' each file for counts. What are all those CPUs for, after all?
Code:
(
echo 'x=0'
sed '
 /^45/!{
   w rej_file
   d
   }
 ' infile | tee good_file | sed '
 s/^.\{99\}.\{99\}.\{82\}\(.\{8\}\).*/x=x+\1/
 '
echo 'x'
)|bc
wc -l <good_file
wc -l <rej_file

On Solaris, in ksh you have <(...) >(...) from /dev/fd/#, so you can do it all in a tree of piped processes in one pass, like:

Code:
( . . . | tee >( ( echo 'count: \c' ; wc -l ) >&2 ) | . . . . )2>report_file

counts records as they go by, and puts the count on stderr, which ends up in the report_file.



Moderator's Comments:
Mod Comment Please use code tags, thank you!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Loop counter variable display

Hello everyone, how can I send output to the screen from a running script or tcl, in such a way that if a loop is executing I will see the rolling counter on my screen as the records are processed in the loop. I do not want the screen to scroll, though. In other words can a var's value be painted... (2 Replies)
Discussion started by: lifespan
2 Replies

2. Shell Programming and Scripting

Adding string variable dynamically in for loop

Hi, I need to generate the text name dynamically in for loop, ex, VAR_COPY_FILE1= file path 1 VAR_COPY_FILE2= file path 2 VAR_COPY_FILE3= file path 3 for i in 1 2 3 do if then "do some process here" fi done (3 Replies)
Discussion started by: msubash26
3 Replies

3. Shell Programming and Scripting

S# in a for loop - concatenate $(loop counter)

Hi, hope I am posting in the right section. My problem is that I have 2 or more arguments passed and I want to check if the arguments passed exists or not. The first argument should not exist and the remaining others should exist. example: ./shells.sh argument1 argument2 argument3 ... (5 Replies)
Discussion started by: fight4love
5 Replies

4. Shell Programming and Scripting

loop with a counter on a constant in bash

Hello Everyone, I'm in need of assistance on creating a script with a counter on a certain string. Basically this script opens a log file and displays certain log data. There are two key words in the log. START and FINISH. In between the START and FINISH is a variable ACTNUMBER. It will... (1 Reply)
Discussion started by: rxc23816
1 Replies

5. Shell Programming and Scripting

How to implement the counter in loop?

Hi, I am working on a script where I need to add one functionality i.e. to could the number of tar files at particular location...but the script is working in below way. 1) create sandbox 2) Drop old member function 3) addmember function 4) Apply checkpoint lable 5) Resync operation(This... (1 Reply)
Discussion started by: anuragpgtgerman
1 Replies

6. Shell Programming and Scripting

Loop usage with counter

Hi friends, I just want to try some thing with shell using loop and counter I have 30 Directory, in each directory number of files say 5, 10, 20 etc... directory_1 directory_2 directory_n what I want to do is read files from directory_1 say 5 files if my counter is like this m=2000 ... (6 Replies)
Discussion started by: Akshay Hegde
6 Replies

7. Shell Programming and Scripting

Help on Adding one counter loop at the end of each line in a file

Hello All, I have file a.txt I want to add a counter loop at the end of each line in a file ill explain: i have a site h**p://test.test=Elite#1 i want to add a a counter to the number at the end of the file, that it will be like this urlLink//test.test=Elite#1 urlLink//test.test=Elite#2... (3 Replies)
Discussion started by: nexsus
3 Replies

8. Shell Programming and Scripting

Avoid multiple emails being sent - Counter, Loop?

Hello, I have currently coded a bash script below in which it does the following: # Archives compressed file from another location. Basically it moves *.gz files to another location. The script also sends an email whenever a new compressed file is placed. This is the issue that i... (5 Replies)
Discussion started by: Wizard_1979
5 Replies

9. Shell Programming and Scripting

Loop counter resets by itself

Hi! Can anyone explain this? The counter CDR_count should go on forever, but it suddenly resets at every step of the FOR loop (I know this because $_file has exactly 378 records). However, the counter reset is OUTSIDE the FOR loop. What's going on?? #!/bin/bash if || ; then echo ... (8 Replies)
Discussion started by: Flavius
8 Replies

10. Shell Programming and Scripting

While loop with limit counter

#!/usr/bin/ksh c=0 while ]; do echo /tmp/unex NOT found, iter : $c; ((c = $c + 1)); sleep 2; done so, the above counter doesn't work, already tried both -lt & -gt, and changed || to && so what am I missing? Thanks in advance (5 Replies)
Discussion started by: unexistance
5 Replies
SIEVESHELL(1)						User Contributed Perl Documentation					     SIEVESHELL(1)

NAME
sieveshell - remotely manipulate sieve scripts SYNOPSIS
sieveshell [--user=user] [--authname=authname] [--realm=realm] [--password=password] [--exec=script] [--execfile=file] server[:port] sieveshell --help DESCRIPTION
sieveshell allows users to manipulate their scripts on a remote server. It works via MANAGESIEVE, a work in progress. The following commands are recognized: list list scripts on server. put <filename> upload script to server. get <name> [<filename>] get script. if no filename display to stdout delete <name> delete script. activate <name> activate script. deactivate deactivate all scripts. OPTIONS
-u user, --user=user The authorization name to request; by default, derived from the authentication credentials. -a authname, --authname=authname The user to use for authentication (defaults to current user). -r realm, --realm=realm The realm to attempt authentication in. -p password, --password=password The password to use when authenticating to server. Note that this parameter can be seen in the process list. Use with caution! -e script, --exec=script Instead of working interactively, run commands from script, and exit when done. -f file, --execfile=file Instead of working interactively, run commands from file file and exit when done. REFERENCES
[MANAGESIEVE] Martin, T.; "A Protocol for Remotely Managing Sieve Scripts", draft-ietf-managesieve-03.txt, Mirapoint, Inc.; May 2001, work in progress. AUTHOR
Tim Martin <tmartin@mirapoint.com>, and the rest of the Cyrus team <cyrus-bugs@andrew.cmu.edu>. perl v5.14.2 2014-06-03 SIEVESHELL(1)
All times are GMT -4. The time now is 03:37 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy