Sponsored Content
Top Forums UNIX for Advanced & Expert Users Comparison and For Loop Taking Too Long Post 302331871 by hanie123 on Tuesday 7th of July 2009 11:54:45 AM
Old 07-07-2009
Comparison and For Loop Taking Too Long

I'd like to
1. Check and compare the 10,000 pnt files contains single record from the /$ROOTDIR/scp/inbox/string1 directory against 39 bad pnt files from the /$ROOTDIR/output/tma/pnt/bad/string1 directory based on the fam_id column value start at position 38 to 47 from the record below. Here is an example of the record from the file in both directories:
PNT0220060503081122003700100000091049000005629001005146417001407712SFirstname Lastname
2. If fam_id is matched then move current file from the /$ROOTDIR/scp/inbox/string1 directory into the /$ROOTDIR/output/tma/pnt/bad/string1 directory.
If not then continue the normal process
The below code is worked but it took 2 plus hours to complete the comparison process. Please advice if there is a better way to re-write or improve the comparison process to make it run faster and better. Thanks
Code:
pntcnt1=`ls -l /$ROOTDIR/scp/inbox/string1 | grep 'PNT.*' | wc -l`
if [[ $pntcnt1 -gt 0 ]] then
 
for gfile in `ls -1 /$ROOTDIR/scp/inbox/string1/PNT.2*`
 do
   gline=`sed '1q' $gfile`
   x=`echo "$gline" | awk '{ print substr( $0, 38, 9 ) }'`
   for bfile in `ls -1 /$ROOTDIR/output/tma/pnt/bad/string1/PNT.2*`
    do
      bline=`sed '1q' $bfile`
      y=`echo "$bline" | awk '{ print substr( $0, 38, 9 ) }'`
if [ "$x" -eq "$y" ]
then
  echo "file moved $gfile"
  mv -f $gfile /$ROOTDIR/output/tma/pnt/bad/string1
 
break
fi
 
done
 
done
fi

 

10 More Discussions You Might Find Interesting

1. Red Hat

login process taking a long time

I'm having a bit of a login performance issue.. wondering if anyone has any ideas where I might look. Here's the scenario... Linux Red Hat ES 4 update 5 regardless of where I login from (ssh or on the text console) after providing the password the system seems to pause for between 30... (4 Replies)
Discussion started by: retlaw
4 Replies

2. Shell Programming and Scripting

For Loop Taking Too Long

I'm new from UNIX scripting. Please help. I have about 10,000 files from the $ROOTDIR/scp/inbox/string1 directory to compare with the 50 files from /$ROOTDIR/output/tma/pnt/bad/string1/ directory and it takes about 2 hours plus to complete the for loop. Is there a better way to re-write the... (5 Replies)
Discussion started by: hanie123
5 Replies

3. Shell Programming and Scripting

<AIX>Problem in purge script, taking very very long time to complete 18.30hrs

Hi, I have here a script which is used to purge older files/directories based on defined purge period. The script consists of 45 find commands, where each command will need to traverse through more than a million directories. Therefore a single find command executes around 22-25 mins... (7 Replies)
Discussion started by: sravicha
7 Replies

4. UNIX for Dummies Questions & Answers

gref -f taking long time for big file

grep -f taking long time to compare for big files, any alternate for fast check I am using grep -f file1 file2 to check - to ckeck dups/common rows prsents. But my files contains file1 contains 5gb and file 2 contains 50 mb and its taking such a long time to compare the files. Do we have any... (10 Replies)
Discussion started by: gkskumar
10 Replies

5. UNIX for Dummies Questions & Answers

Job is taking long time

Hi , We have 20 jobs are scheduled. In that one of our job is taking long time ,it's not completing. If we are not terminating it's running infinity time actually the job completion time is 5 minutes. The job is deleting some records from the table and two insert statements and one select... (7 Replies)
Discussion started by: ajaykumarkona
7 Replies

6. Solaris

Re-sync Taking Extremely Long.

It's almost 3 days now and my resync/re-attach is only at 80%. Is there something I can check in Solaris 10 that would be causing the degradation. It's only a standby machine. My live system completed in 6hrs. (9 Replies)
Discussion started by: ravzter
9 Replies

7. Solaris

How to find out bottleneck if system is taking long time in gzip

Dear All, OS = Solaris 5.10 Hardware Sun Fire T2000 with 1 Ghz quode core We have oracle application 11i with 10g database. When ever i am trying to take cold backup of database with 55GB size its taking long time to finish. As the application is down nobody is using the server at all... (8 Replies)
Discussion started by: yoojamu
8 Replies

8. UNIX for Dummies Questions & Answers

ls is taking long time to list

Hi, All the data are kept on Netapp using NFS. some directories are so fast when doing ls but few of them are slow. After doing few times, it becomes fast. Then again after few minutes, it becomes slow again. Can you advise what's going on? This one directory I am very interested is giving... (3 Replies)
Discussion started by: samnyc
3 Replies

9. Shell Programming and Scripting

While loop problem taking too long

while read myhosts do while read discovered do echo "$discovered" done < $LOGFILE | grep -Pi "|" | egrep... (7 Replies)
Discussion started by: SkySmart
7 Replies

10. Shell Programming and Scripting

Rm -rf is taking very long, will it timeout?

I have so many (hundreds of thousands) files and directories within this one specific directory that my "rm -rf" command to delete them has been taking forever. I did this via the SSH, my question is: if my SSH connection times out before rm -rf finishes, will it continue to delete all of those... (5 Replies)
Discussion started by: phpchick
5 Replies
TR(1)							      General Commands Manual							     TR(1)

NAME
tr - translate characters SYNOPSIS
tr [ -cds ] [ string1 [ string2 ] ] DESCRIPTION
Tr copies the standard input to the standard output with substitution or deletion of selected characters (runes). Input characters found in string1 are mapped into the corresponding characters of string2. When string2 is short it is padded to the length of string1 by dupli- cating its last character. Any combination of the options -cds may be used: -c Complement string1: replace it with a lexicographically ordered list of all other characters. -d Delete from input all characters in string1. -s Squeeze repeated output characters that occur in string2 to single characters. In either string a noninitial sequence -x, where x is any character (possibly quoted), stands for a range of characters: a possibly empty sequence of codes running from the successor of the previous code up through the code for x. The character followed by 1, 2 or 3 octal digits stands for the character whose 16-bit value is given by those digits. The character sequence followed by 1, 2, 3, or 4 hexadecimal digits stands for the character whose 16-bit value is given by those digits. A followed by any other character stands for that character. EXAMPLES
Replace all upper-case ASCII letters by lower-case. tr A-Z a-z <mixed >lower Create a list of all the words in one per line in where a word is taken to be a maximal string of alphabetics. String2 is given as a quoted newline. tr -cs A-Za-z ' ' <file1 >file2 SOURCE
/sys/src/cmd/tr.c SEE ALSO
sed(1) TR(1)
All times are GMT -4. The time now is 05:36 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy