The UNIX and Linux Forums  

Go Back   The UNIX and Linux Forums > Top Forums > UNIX for Dummies Questions & Answers
Google UNIX.COM


UNIX for Dummies Questions & Answers If you're not sure where to post a UNIX or Linux question, post it here. All UNIX and Linux newbies welcome !!

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
IBM Informix Load and Insert with multiple files rauphelhunter Shell Programming and Scripting 0 04-09-2008 08:52 AM
balance 3.42 (Default branch) iBot Software Releases - RSS News 0 04-08-2008 09:50 PM
Load balance summerpeh Linux 2 12-30-2007 04:05 AM
Need help in wrting Load Script for a Load-Resume type of load. ankitgupta Shell Programming and Scripting 1 11-09-2006 08:46 PM
load Balance yorkyboy UNIX for Dummies Questions & Answers 1 05-13-2005 08:00 AM

Reply
 
LinkBack Thread Tools Display Modes
  #1 (permalink)  
Old 01-31-2008
Registered User
 

Join Date: Dec 2006
Posts: 28
mv load balance files

Ok so I have files that are going to land on /apps/, but I need to load balance them so I need to load balance them to four different folders.

The three file extensions I get are .mpe .mpd and mpf which will land here... /apps/ I can't move them until the mpf is there it triggers the next process within the ./a/ or. /b/ or. /c/ or ./d/ folder etc.

The names of the three files will be the same but I get three files for a file suite once they all land on the /apps/ dir I need to mv the file suite that's the oldest to

/apps/a/
/apps/b/
/apps/c/
/apps/d/


So for example filesuite1.mpe filesuite1.mpd and filesuite.mpf land on /apps/

So I need something that will pick them up each three and move to A then the next suite that comes in move to b and the next to C and so on ?

Last edited by xgringo; 01-31-2008 at 11:53 AM.
Reply With Quote
Forum Sponsor
  #2 (permalink)  
Old 01-31-2008
Registered User
 

Join Date: Dec 2006
Posts: 28
#!/bin/bash
oldest_mpd=`ls -1 -t /apps/*.mpd | head -1`
oldest_mpe=`ls -1 -t /apps/*.mpe | head -1`
oldest_mpf=`ls -1 -t /apps/*.mpf | head -1`

mv $oldest_mpd /apps/d1/
mv $oldest_mpe /apps/d1/
mv $oldest_mpf /apps/d1/

mv $oldest_mpd /apps/d2/
mv $oldest_mpe /apps/d2/
mv $oldest_mpf /apps/d2/

mv $oldest_mpd /apps/d3/
mv $oldest_mpe /apps/d3/
mv $oldest_mpf /apps/d3/

mv $oldest_mpd /apps/d4/
mv $oldest_mpe /apps/d4/
mv $oldest_mpf /apps/d4/

I can put this into cron every minute right but I don't think this is going to be that fast, what if the moves happen in less than a minute and it moves four suites and then it's done til the next minute? Is there a way to make it continuous? will this work?

Would I have to do something like this to reload the variable again?

Last edited by xgringo; 01-31-2008 at 01:25 PM.
Reply With Quote
  #3 (permalink)  
Old 01-31-2008
Smiling Dragon's Avatar
Disorganised User
 
Join Date: Nov 2007
Location: New Zealand
Posts: 674
The above is likely to produce errors as it assumes you'll have at least four sets of files in there any time it's run.

I'd think it would be better to leave something in memory instead. That way it will know where it last moved a file.
Code:
#!/bin/sh
TARGETLIST="a:b:c:d"
INTERVAL=60 # seconds

targetnum=1
targetlistsize=`echo $TARGETLIST | sed 's/[^:]//g' | wc -c`
while true
do
  for mpf in *.mpf
  do
    files="$mpf `echo $mpf | sed 's/f$/e/'` `echo $mpf | sed 's/f$/d/'`"
    if ls $files > /dev/null 2>&1
    then
      mv $files `echo $TARGETLIST | cut -d ':' -f $targetnum`
      targetnum=`expr $targetnum % $targetlistsize`
      targetnum=`expr $targetnum + 1`
    fi
  done
  sleep $INTERVAL
done
However, are you sure you are looking at the right problem? Could you not use a striped volume across those 4 devices instead?
Reply With Quote
  #4 (permalink)  
Old 01-31-2008
Registered User
 

Join Date: Dec 2006
Posts: 28
Thank you I'm going to try this out,

I am only an application administrator, so if I could design it perhaps we could do that, so I have to work around existing constraints.
Reply With Quote
  #5 (permalink)  
Old 01-31-2008
Registered User
 

Join Date: Dec 2006
Posts: 28
Could you also explain to me what this is doing?

files="$mpf `echo $mpf | sed 's/f$/e/'` `echo $mpf | sed 's/f$/d/'`"
Reply With Quote
  #6 (permalink)  
Old 01-31-2008
Smiling Dragon's Avatar
Disorganised User
 
Join Date: Nov 2007
Location: New Zealand
Posts: 674
Quote:
Originally Posted by xgringo View Post
Could you also explain to me what this is doing?
files="$mpf `echo $mpf | sed 's/f$/e/'` `echo $mpf | sed 's/f$/d/'`"
Sure, this creates a string containing the three files we're going to be moving in one set (blah.mpf, blah.mpd and blah.mpe).
You say that the .mpf generally arrives last so I'm looking for those to keep things efficient. This means that we start with all .mpf files and for each one, and generate a 3 element, space seperated list from it.
This is done by using sed to replace the last letter of the filename (ie 'f') with e in the second field and d in the third. The sed segments sare saying:
"take the .mpf filename, and look for a part of the name that has an 'f' followed by the end of line, then replace it with an e (or a d)".
Reply With Quote
  #7 (permalink)  
Old 01-31-2008
Smiling Dragon's Avatar
Disorganised User
 
Join Date: Nov 2007
Location: New Zealand
Posts: 674
oh by the way, in case you hadn't spotted it, you can set the TARGETLIST variable at the top to be a : seperated list of directories, whatever and however many or few you wish (provided there's at least 1) and it should automatically figure out what to do.
INTERVAL is how many seconds to wait once it's processed all the files it's found this run before looking for more files. If it finds none it silently goes back to sleep for INTERVAL seconds again (you could set this to 1 if you want it to be highly responsive, or 300 to be pretty gentle on the system).
Reply With Quote
Google UNIX.COM
Reply

Thread Tools
Display Modes




All times are GMT -7. The time now is 09:49 AM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited.
The UNIX and Linux Forums Content Copyright ©1993-2008 The CEP Blog All Rights Reserved -Ad Management by RedTyger Visit The Global Fact Book

Content Relevant URLs by vBSEO 3.2.0