Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

adding counter to a variable while moving in a loop

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting adding counter to a variable while moving in a loop
# 1  
Old 09-30-2010
Question adding counter to a variable while moving in a loop
The scenario is like this :
I need to read records from a file one by one and increment counter1, if a certain field matches with a number say "40"..the script should increment the counter2 and also extract a corresponding field from the same line and adding them one by one and redirecting the the entire line to another file. If that certain field doesn't match with "40" then increment counter3.

I followed the below approach :
Used while -do-done to move through the file line by line...used expr to add up the counter.

Result I am getting :

The script is resulting in an infinite loop, while giving all the counter values as 1.

The code is like following :

cat read_file.txt | while read each_record
do
read_record_counter=0
write_record_counter=0
skipped_record_counter=0
temp_rec=0
total_amount=0
read_record_counter=`expr ${read_record_counter} + 1`
echo "read_record_counter : "${read_record_counter}
counter=`echo "${each_record}" | cut -c1-2`
if [[ ${count} -eq 45 ]]
then
write_record_counter=`expr ${write_record_counter} + 1`
echo "write_record_counter : "${write_record_counter}
temp_rec=`echo "${each_rec}" | cut -280-287`
echo "temp_ocd_chargeback : " ${temp_rec}
total_amount=`expr ${temp_ocd_chargeback} + 1`
echo "total_amount : "${total_amount}
temp_ocd_chargeback=0
cat ${each_record} >> temp_file.txt
else
skipped_record_counter=`expr ${skipped_record_counter} + 1`
echo "skipped_record_counter : "${skipped_record_counter}
fi
done

The file looks like follwing :

01RD0108200CARDINAL HLTH DC15436629+00002 09-24-101148758000U010000210900426642BG9471222772770151 50458 60528 0727101127801+0000001 530 N MAIN ST PROVIDENCERI02904576200000 +00016827.44+00353.71+00279.17+00000074.00+00000074.54 GENOA HLTHCARE 2010092402C
01RD0108200CARDINAL HLTH DC15436629+00002 09-24-101148758000U010000210900426642BG9471222772770151 50458 60528 0727101127801-0000001 530 N MAIN ST PROVIDENCERI02904576200000 +00016827.44+00353.71+00279.71+00000074.00-00000074.00 GENOA HLTHCARE 2010092406D
01RO0153609CARDINAL HEALTHCARE DC15436881+00004 09-24-101148759000U010004014200139419BB3487166772770151 50458 60528 0727101127778+0000001 120 LYME ST BELLEVUE OH44811142800000 +00028724.31+00353.71+00279.17+00000074.00+00000074.54 BELLEVUE HOME MEDICAL 2010092402C
01RO0153609CARDINAL HEALTHCARE DC15436881+00004 09-24-101148759000U010004014200139419BB3487166772770151 50458 60528 0727101127778-0000001 120 LYME ST BELLEVUE OH44811142800000 +00028724.31+00353.71+00279.71+00000074.00-00000074.00 BELLEVUE HOME MEDICAL 2010092406D
01RO0153609CARDINAL HEALTHCARE DC15436881+00004 09-24-101148759000U010004014201083219FN0383997772770151 50458 60528 0727101127778+0000002 1500 SHERMAN AVNORWOOD OH45212251000000 +00028724.31+00353.71+00279.17+00000148.00+00000149.08 NORWOOD PHARM 2010092402C
01RO0153609CARDINAL HEALTHCARE DC15436881+00004 09-24-101148759000U010004014201083219FN0383997772770151 50458 60528 0727101127778-0000002 1500 SHERMAN AVNORWOOD OH45212251000000 +00028724.31+00353.71+00279.71+00000148.00-00000148.00 NORWOOD PHARM 2010092406D
01RW0231908CARDINAL HLTH-AUROA DC15437804+00002 09-24-101148760000U010003349001024665FG0026193772770151 50458 60528 0727101127832+0000001 625 W WASHINGTOMADISON WI53703263700000 +00021698.43+00353.71+00279.17+00000074.00+00000074.54 GENOA HLTHCARE OF WI LLC 2010092402C
01RW0231908CARDINAL HLTH-AUROA DC15437804+00002 09-24-101148760000U010003349001024665FG0026193772770151 50458 60528 0727101127832-0000001 625 W WASHINGTOMADISON WI53703263700000 +00021698.43+00353.71+00279.71+00000074.00-00000074.00 GENOA HLTHCARE OF WI LLC 2010092406D
01RW0269654CARDINAL HLTH DC15440727+00002 09-24-101148761000U010031906900272911BP6647222772770151 50458 60528 0727101127802+0000001 70 JACKSON DR CRANFORD NJ07016351000000 +00018293.63+00353.71+00279.17+00000074.00+00000074.54 PARTNERS HEALTHCARE, LLC 2010092402C
01RW0269654CARDINAL HLTH DC15440727+00002 09-24-101148761000U010031906900272911BP6647222772770151 50458 60528 0727101127802-0000001 70 JACKSON DR CRANFORD NJ07016351000000 +00018293.63+00353.71+00279.71+00000074.00-00000074.00 PARTNERS HEALTHCARE, LLC 2010092406D
45RA0316958AMERISOURCEBERGEN DRUG 9129371835+00372 09-24-101148692000U010041354800424938BW6834255773000031 10147 17007 0916101127758+0000001 3010 GARRETT DRPERRYTON TX79070532200000 +00247836.02+00208.40+00128.75+00000079.65+00000079.65 WAIDE'S PHARMACY INC 2010092401C
45RA0316958AMERISOURCEBERGEN DRUG 9129371835+00372 09-24-101148692000U010041354800501759BW3150086773000031 10147 17007 0910101127758+0000002 1919 S WHEELINGTULSA OK74104563800000 +00247836.02+00208.40+00128.75+00000159.30+00000159.30 PROFESSIONAL PHARMACY 2010092401C
45RA0316958AMERISOURCEBERGEN DRUG 9129371835+00372 09-24-101148692000U010041354800501759BW3150086773000031 10147 17007 0910101127758+0000002 1919 S WHEELINGTULSA OK74104563800000 +00247836.02+00208.40+00128.75+00000159.30+00000159.30 PROFESSIONAL PHARMACY 2010092401C
45RA0316958AMERISOURCEBERGEN DRUG 9129371835+00372 09-24-101148692000U010041354800502781AI2541565773000031 10147 17007 0817101127758-0000001 205 ATLANTA ST GRAVETTE AR72736936400000 +00247836.02+00208.40+00128.75-00000079.65-00000079.65 TEASLEY DRUG 2010092405D
45RA0316958AMERISOURCEBERGEN DRUG 9129371835+00372 09-24-101148692000U010041354800517169AF2912120773000031 10147 17007 0916101127758+0000004 245 E OAK ST CONWAY AR72032468400000 +00247836.02+00208.40+00128.75+00000318.60+00000318.60 FREDS PHARMACY 2010092401C
60RA0290736AMERISOURCEBERGEN DRUG 9129364366+00093 09-24-101149374000U010004665600344756BH6111291762252011 50458 58801 0914101128416+0000001 6100 N HAGGERTYCANTON MI48187368300000 +00067329.69+00454.74+00149.66+00000305.08+00000305.08 HENRY FRD MEDCL CTR PHARM20100924Y1C
60RA0289000AMERISOURCEBERGEN DRUG 9129366455+00184 09-24-101148684000U01000614890000782000007820 761701481 50458 58801 0902101127747-0000012 47149 BUSE RD PATUXENT RMD20670154000000 +00237192.08+00454.74+00132.04-00003872.40-00003872.40 NAVAL MEDCL CLINIC 2010092405D
60RA0289000AMERISOURCEBERGEN DRUG 9129366455+00184 09-24-101148684000U010006148900055120AU4675027761701481 50458 58801 0915101127747+0000006 73 NEALY BLVD LANGLEY AFVA23665202300000 +00237192.08+00454.74+00132.04+00001936.20+00001936.20 1 MDG SGSL 2010092401C
60RA0289000AMERISOURCEBERGEN DRUG 9129366455+00184 09-24-101148684000U010006148900110626BS700513476716331 50458 58801 0915101127747+0000003 6501 N CHARLES TOWSON MD21204681900000 +00237192.08+00454.74+00454.51+00000000.69+00000000.69 SHEPPARD PRATT HLTH SYS 2010092401C
60RA0316958AMERISOURCEBERGEN DRUG 9129371835+00372 09-24-101148692000U010041354800008136BD2247636761701481 50458 58801 0916101127758+0000048 36000 DARNALL LFORT HOOD TX76544 00000 +00247836.02+00454.74+00132.04+00015489.60+00015489.60 DARNALL ARMY COMM HOSP 2010092401C
60RA0316958AMERISOURCEBERGEN DRUG 9129371835+00372 09-24-101148692000U010041354800008149AW5333303761701481 50458 58801 0916101127758+0000007 2200 BERGQUIST SAN ANTONITX78236990700000 +00247836.02+00454.74+00132.04+00002258.90+00002258.90 WHMC/MSLS (FM3047) 2010092401C
60RA0316958AMERISOURCEBERGEN DRUG 9129371835+00372 09-24-101148692000U010041354800248892BM5725138762252011 50458 58801 0916101127758+0000001 400 W 4TH ST ODESSA TX79761504500000 +00247836.02+00454.74+00149.66+00000305.08+00000305.08 MEDICAL CTR HOSP OUTPATIE20100924Y1C
60PF0000012MCKESSON DRUG CO (DC147) 1265VJOMSO+00007 09-24-101149210000U010002936400186689BL4662563750444521 50580 87411 0921101128333+0000001 6 PAIUTE DR LAS VEGAS NV89106326000000 +00002500.44+00174.88+00119.94+00000054.94+00000054.94 LAS VEGAS PAIUTE TRIBE 2010092401C


# 2  
Old 09-30-2010
Optimized script:
bash code:
  1. #!/bin/bash
  2. while read each_record
  3. do
  4.    ((read_record_counter++))
  5.    echo -ne "read_record_counter: ${read_record_counter}\t"
  6.    if [[ ${each_record::2} -eq 45 ]]
  7.    then
  8.       ((write_record_counter++))
  9.       echo -ne "write_record_counter: ${write_record_counter}\t"
  10.       echo -ne "temp_ocd_chargeback:  ${each_rec:279:18}\t"
  11.       total_amount=$((temp_ocd_chargeback + 1))
  12.       echo "total_amount: ${total_amount}"
  13.       temp_ocd_chargeback=0
  14.       echo "${each_record}" >> temp_file.txt
  15.    else
  16.       ((skipped_record_counter++))
  17.       echo "skipped_record_counter: ${skipped_record_counter}"
  18.    fi
  19. done <read_file.txt
  20. echo "--------------------------"
  21. # echo "cat temp_file.txt"
  22. # cat temp_file.txt
Output
Code:
read_record_counter: 1	skipped_record_counter: 1
read_record_counter: 2	skipped_record_counter: 2
read_record_counter: 3	skipped_record_counter: 3
read_record_counter: 4	skipped_record_counter: 4
read_record_counter: 5	skipped_record_counter: 5
read_record_counter: 6	skipped_record_counter: 6
read_record_counter: 7	skipped_record_counter: 7
read_record_counter: 8	skipped_record_counter: 8
read_record_counter: 9	skipped_record_counter: 9
read_record_counter: 10	skipped_record_counter: 10
read_record_counter: 11	write_record_counter: 1	temp_ocd_chargeback:  	total_amount: 1
read_record_counter: 12	write_record_counter: 2	temp_ocd_chargeback:  	total_amount: 1
read_record_counter: 13	write_record_counter: 3	temp_ocd_chargeback:  	total_amount: 1
read_record_counter: 14	write_record_counter: 4	temp_ocd_chargeback:  	total_amount: 1
read_record_counter: 15	write_record_counter: 5	temp_ocd_chargeback:  	total_amount: 1
read_record_counter: 16	skipped_record_counter: 11
read_record_counter: 17	skipped_record_counter: 12
read_record_counter: 18	skipped_record_counter: 13
read_record_counter: 19	skipped_record_counter: 14
read_record_counter: 20	skipped_record_counter: 15
read_record_counter: 21	skipped_record_counter: 16
read_record_counter: 22	skipped_record_counter: 17
read_record_counter: 23	skipped_record_counter: 18

There's a problem with temp_ocd_chargeback : at what position is it extracted?
# 3  
Old 09-30-2010
Bug thanks a lot
Yes..I did also notice that....I will verify it once and post the verified one...
bye the way...
[[ ${each_record::2} -eq 45 ]]
...please let me know why you used
${each_record::2}
# 4  
Old 09-30-2010
Quote:
Originally Posted by mady135
...please let me know why you used
${each_record::2}
It's equivalent to ${each_record:0:2} which means :
take the 2 characters after position 0 in the variable each_record
It's a shell builtin, so there's no need for echo and the external command cut
# 5  
Old 10-01-2010
all in one awk:

Code:
awk '
{
   if ($0~/^45/) 
     {w++;t=substr($0,280,8);printf "read_record_counter: %d write_record_counter: %d temp_ocd_chargeback: %s total_amount: 1\n",NR,w,t}
   else{s++;printf "read_record_counter: %d skipped_record_counter: %d\n",NR,s}
}' read_file.txt

# 6  
Old 10-04-2010
Yes, fork/exec was too costly on your attempt, but for big data sets, bash may be slower than a multi-process script of more specialized tools. You can use multiple sed's to spread the load, divert reject records to separate files while streaming numbers turned into a bc script to stdout | bc, then just 'wc -l <file' each file for counts. What are all those CPUs for, after all?
Code:
(
echo 'x=0'
sed '
 /^45/!{
   w rej_file
   d
   }
 ' infile | tee good_file | sed '
 s/^.\{99\}.\{99\}.\{82\}\(.\{8\}\).*/x=x+\1/
 '
echo 'x'
)|bc
wc -l <good_file
wc -l <rej_file

On Solaris, in ksh you have <(...) >(...) from /dev/fd/#, so you can do it all in a tree of piped processes in one pass, like:

Code:
( . . . | tee >( ( echo 'count: \c' ; wc -l ) >&2 ) | . . . . )2>report_file

counts records as they go by, and puts the count on stderr, which ends up in the report_file.



Moderator's Comments:
Mod Comment Please use code tags, thank you!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

While loop with limit counter

#!/usr/bin/ksh c=0 while ]; do echo /tmp/unex NOT found, iter : $c; ((c = $c + 1)); sleep 2; done so, the above counter doesn't work, already tried both -lt & -gt, and changed || to && so what am I missing? Thanks in advance (5 Replies)
Discussion started by: unexistance
5 Replies

2. Shell Programming and Scripting

Loop counter resets by itself

Hi! Can anyone explain this? The counter CDR_count should go on forever, but it suddenly resets at every step of the FOR loop (I know this because $_file has exactly 378 records). However, the counter reset is OUTSIDE the FOR loop. What's going on?? #!/bin/bash if || ; then echo ... (8 Replies)
Discussion started by: Flavius
8 Replies

3. Shell Programming and Scripting

Avoid multiple emails being sent - Counter, Loop?

Hello, I have currently coded a bash script below in which it does the following: # Archives compressed file from another location. Basically it moves *.gz files to another location. The script also sends an email whenever a new compressed file is placed. This is the issue that i... (5 Replies)
Discussion started by: Wizard_1979
5 Replies

4. Shell Programming and Scripting

Help on Adding one counter loop at the end of each line in a file

Hello All, I have file a.txt I want to add a counter loop at the end of each line in a file ill explain: i have a site h**p://test.test=Elite#1 i want to add a a counter to the number at the end of the file, that it will be like this urlLink//test.test=Elite#1 urlLink//test.test=Elite#2... (3 Replies)
Discussion started by: nexsus
3 Replies

5. Shell Programming and Scripting

Loop usage with counter

Hi friends, I just want to try some thing with shell using loop and counter I have 30 Directory, in each directory number of files say 5, 10, 20 etc... directory_1 directory_2 directory_n what I want to do is read files from directory_1 say 5 files if my counter is like this m=2000 ... (6 Replies)
Discussion started by: Akshay Hegde
6 Replies

6. Shell Programming and Scripting

How to implement the counter in loop?

Hi, I am working on a script where I need to add one functionality i.e. to could the number of tar files at particular location...but the script is working in below way. 1) create sandbox 2) Drop old member function 3) addmember function 4) Apply checkpoint lable 5) Resync operation(This... (1 Reply)
Discussion started by: anuragpgtgerman
1 Replies

7. Shell Programming and Scripting

loop with a counter on a constant in bash

Hello Everyone, I'm in need of assistance on creating a script with a counter on a certain string. Basically this script opens a log file and displays certain log data. There are two key words in the log. START and FINISH. In between the START and FINISH is a variable ACTNUMBER. It will... (1 Reply)
Discussion started by: rxc23816
1 Replies

8. Shell Programming and Scripting

S# in a for loop - concatenate $(loop counter)

Hi, hope I am posting in the right section. My problem is that I have 2 or more arguments passed and I want to check if the arguments passed exists or not. The first argument should not exist and the remaining others should exist. example: ./shells.sh argument1 argument2 argument3 ... (5 Replies)
Discussion started by: fight4love
5 Replies

9. Shell Programming and Scripting

Adding string variable dynamically in for loop

Hi, I need to generate the text name dynamically in for loop, ex, VAR_COPY_FILE1= file path 1 VAR_COPY_FILE2= file path 2 VAR_COPY_FILE3= file path 3 for i in 1 2 3 do if then "do some process here" fi done (3 Replies)
Discussion started by: msubash26
3 Replies

10. Shell Programming and Scripting

Loop counter variable display

Hello everyone, how can I send output to the screen from a running script or tcl, in such a way that if a loop is executing I will see the rolling counter on my screen as the records are processed in the loop. I do not want the screen to scroll, though. In other words can a var's value be painted... (2 Replies)
Discussion started by: lifespan
2 Replies
Login or Register to Ask a Question