The UNIX and Linux Forums  

Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
.
google unix.com



Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
tftp troubleshooting Bobby76 UNIX for Dummies Questions & Answers 0 02-29-2008 09:44 AM
reading from a file and pass as variables and ignore # in the file konark Shell Programming and Scripting 4 11-08-2007 03:55 AM
Reading a file and writing the file name to a param file. thebeginer UNIX for Advanced & Expert Users 1 10-05-2007 05:38 PM
Reading file names from a file and executing the relative file from shell script anushilrai Shell Programming and Scripting 4 03-10-2006 05:25 AM
Reading specific contents from a file and appending it to another file dnicky Shell Programming and Scripting 5 10-04-2005 06:45 AM

Closed Thread
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish Bulgarian Greek Powered by Powered by Google
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 10-03-2006
Indalecio Indalecio is offline
Registered User
  
 

Join Date: Sep 2006
Location: Sweden
Posts: 59
Troubleshooting: file reading clash?

Hi all
I´m trying to fully understand an issue I had yesterday and was just hoping you'd give me your opinion about it. I believe I fixed the problem but I still feel ignorant about some parts of the behaviour I could experience in the following scenario. Here was the situation (old simplified code version):


Code:
for FILE in $INPUT_DIR ; do
   grep -q $FILE $PROCESSED 2>&1
   if [ $? != 0 ] ; then
      echo $FILE >> $PROCESSED
   else
      continue
   fi
   SOURCE=`grep ^S $FILE | wc -l | awk '{print $1}'`
   END_RCD=`grep ^E $FILE | wc -l | awk '{print $1}'`
   if [ $SOURCE -ne $END_RCD ] ; then
      echo "Error in $FILE: "$SOURCE" source keys detected, but "$END_RCD" end keys detected.
   fi
done

The content of the INPUT_DIR contained a lot of files, and it reported the above error on one of them (SOURCE = 1, END_RCD = 0). However the file was ok as far as I could see. I quickly ensured my commands were correct and even re-ran the script on the whole file list to get a final ok as an output, so this looked weird at first sight.

After some time of investigation I finally found that I had a bunch of n processes running in the background calling that bit of code. Despite of this I originally supposed this script would only be running once at the same time, so the files processed from my process(n) falling into INPUT_DIR would be updating the PROCESSED file so they don't get picked up by the next calls.

Unfortunately for my theory, the truth was that the script got called several times at the same time (from different sessions), processing in parallel, this with a PROCESSED file that wasn't unique (you begin to see what it can look like here). So to summarize, the above script reported an error when running on the process (n) for a file created by the process(n-1).

Now the above situation should still be ok as long as a file is opened once a time, it's just that this file gets checked by the wrong process but the result remains the same. So what I believe is that the file on which it reported the error was opened in two sessions at the same time.

I have removed the PROCESSED file and now run this script once after all processes are completed so I don't get any issue anymore. The only thing I still don't understand is why END_RCD gave 0. Since two sessions have different variable buffers, I don't see how a potential clash on a file would produce such an error. If anybody here has a credible explanation, I would be very happy to know about it.

Many thanks,
Yann
Closed Thread

Bookmarks

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On




All times are GMT -4. The time now is 11:49 PM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0