UNIX for Dummies Questions & Answers

Ok so I have files that are going to land on /apps/, but I need to load balance them so I need to load balance them to four different folders.

The three file extensions I get are .mpe .mpd and mpf which will land here... /apps/ I can't move them until the mpf is there it triggers the next process within the ./a/ or. /b/ or. /c/ or ./d/ folder etc.

The names of the three files will be the same but I get three files for a file suite once they all land on the /apps/ dir I need to mv the file suite that's the oldest to

/apps/a/
/apps/b/
/apps/c/
/apps/d/


So for example filesuite1.mpe filesuite1.mpd and filesuite.mpf land on /apps/

So I need something that will pick them up each three and move to A then the next suite that comes in move to b and the next to C and so on ?

#!/bin/bash
oldest_mpd=`ls -1 -t /apps/*.mpd | head -1`
oldest_mpe=`ls -1 -t /apps/*.mpe | head -1`
oldest_mpf=`ls -1 -t /apps/*.mpf | head -1`

mv $oldest_mpd /apps/d1/
mv $oldest_mpe /apps/d1/
mv $oldest_mpf /apps/d1/

mv $oldest_mpd /apps/d2/
mv $oldest_mpe /apps/d2/
mv $oldest_mpf /apps/d2/

mv $oldest_mpd /apps/d3/
mv $oldest_mpe /apps/d3/
mv $oldest_mpf /apps/d3/

mv $oldest_mpd /apps/d4/
mv $oldest_mpe /apps/d4/
mv $oldest_mpf /apps/d4/

I can put this into cron every minute right but I don't think this is going to be that fast, what if the moves happen in less than a minute and it moves four suites and then it's done til the next minute? Is there a way to make it continuous? will this work?

Would I have to do something like this to reload the variable again?

The above is likely to produce errors as it assumes you'll have at least four sets of files in there any time it's run.

I'd think it would be better to leave something in memory instead. That way it will know where it last moved a file.

Code:

#!/bin/sh
TARGETLIST="a:b:c:d"
INTERVAL=60 # seconds

targetnum=1
targetlistsize=`echo $TARGETLIST | sed 's/[^:]//g' | wc -c`
while true
do
  for mpf in *.mpf
  do
    files="$mpf `echo $mpf | sed 's/f$/e/'` `echo $mpf | sed 's/f$/d/'`"
    if ls $files > /dev/null 2>&1
    then
      mv $files `echo $TARGETLIST | cut -d ':' -f $targetnum`
      targetnum=`expr $targetnum % $targetlistsize`
      targetnum=`expr $targetnum + 1`
    fi
  done
  sleep $INTERVAL
done

However, are you sure you are looking at the right problem? Could you not use a striped volume across those 4 devices instead?

Thank you I'm going to try this out,

I am only an application administrator, so if I could design it perhaps we could do that, so I have to work around existing constraints.

Could you also explain to me what this is doing?

files="$mpf `echo $mpf | sed 's/f$/e/'` `echo $mpf | sed 's/f$/d/'`"

Sure, this creates a string containing the three files we're going to be moving in one set (blah.mpf, blah.mpd and blah.mpe).
You say that the .mpf generally arrives last so I'm looking for those to keep things efficient. This means that we start with all .mpf files and for each one, and generate a 3 element, space seperated list from it.
This is done by using sed to replace the last letter of the filename (ie 'f') with e in the second field and d in the third. The sed segments sare saying:
"take the .mpf filename, and look for a part of the name that has an 'f' followed by the end of line, then replace it with an e (or a d)".

oh by the way, in case you hadn't spotted it, you can set the TARGETLIST variable at the top to be a : seperated list of directories, whatever and however many or few you wish (provided there's at least 1) and it should automatically figure out what to do.
INTERVAL is how many seconds to wait once it's processed all the files it's found this run before looking for more files. If it finds none it silently goes back to sleep for INTERVAL seconds again (you could set this to 1 if you want it to be highly responsive, or 300 to be pretty gentle on the system).