Balance the load of files across queues using shell script
Hi,
I need to balance the load to be processed by the system using shell script.
The scenario is like below:
Suppose there are some files in a directory:
Code:
-rw-rw-r-- 1 usr usrgrp 456432306 Oct 23 07:53 abc_queue_q1
-rw-rw-r-- 1 usr usrgrp 4123934 Oct 23 07:53 def_queue_q1
-rw-rw-r-- 1 usr usrgrp 14 Oct 25 09:01 ghi_queue_q1
-rw-rw-r-- 1 usr usrgrp 10 Oct 25 09:01 jkl_queue_q1
-rw-rw-r-- 1 usr usrgrp 10 Oct 25 09:01 mno_queue_q1
-rw-rw-r-- 1 usr usrgrp 50765487 Oct 25 09:01 mno_queue_q1
-rw-rw-r-- 1 usr usrgrp 98765412 Oct 25 09:01 mno_queue_q1
and there are 4 queues available across which we can distribute the load
queue_q1,queue_q2,queue_q3,queue_q4
but the files were submitted haphazardly...Now we need to manage the load across all 4 queues. So that the equal load is shared among the queues.
So in our case the output should be like:
Code:
-rw-rw-r-- 1 usr usrgrp 456432306 Oct 23 07:53 abc_queue_q1
-rw-rw-r-- 1 usr usrgrp 4123934 Oct 23 07:53 def_queue_q2
-rw-rw-r-- 1 usr usrgrp 14 Oct 25 09:01 ghi_queue_q2
-rw-rw-r-- 1 usr usrgrp 10 Oct 25 09:01 jkl_queue_q2
-rw-rw-r-- 1 usr usrgrp 10 Oct 25 09:01 mno_queue_q2
-rw-rw-r-- 1 usr usrgrp 50765487 Oct 25 09:01 mno_queue_q3
-rw-rw-r-- 1 usr usrgrp 98765412 Oct 25 09:01 mno_queue_q4
there could be n number of conditions to balance the load..conditions could be:
there are thousands of files on queue_q1. suppose the total size of all the file on queue_q1 is 1000 MB..then 250 MB will be distributed across all 4 queues...
If there is 1 big file which is on queue_q1 and some small files ... then you need to balance the load.,..Remember you will not break any file...We just try to manage the load that is possible.
Last edited by Don Cragun; 10-25-2013 at 10:36 PM..
Reason: Add CODE tags.
Hi,
I need to balance the load to be processed by the system using shell script.
The scenario is like below:
...
If there is 1 big file which is on queue_q1 and some small files ... then you need to balance the load.,..Remember you will not break any file...We just try to manage the load that is possible.
Your consistent unwillingness to use CODE tags and to rely on moderators to clean up your posts is troubling.
The fact that you never show any code that you're working on and seem only to want us to write code for you is troubling.
The fact that this posting sounds more like a homework assignment than something you're working on on your own is troubling.
In the future, post homework assignments in the Homework & Coursework Questions forum, show us that you are making an effort to solve problems on your own, and PLEASE use CODE tags as required by forum rules.
This thread is closed.
This User Gave Thanks to Don Cragun For This Post:
Hello everyone,
I am new to shell scripting/ loading data into a database.
I want to load data into Oracle database using SQL loader. Can some one please explain why do we need unix shell script to load the data into the database? Also can someone please explain what has to be in that script?... (5 Replies)
Hi All,
I want to check load balance(or CPU utilzation) on server dynamically based on that value i want to perform some operation .
Means if load balance is <10 then start server .
Am using prstat -a 1
commadn it is listing CPU utilzation/load balancing dynamically ..
How to take this... (7 Replies)
I need help as to how to write a script in Unix for the following:
We have 3 servers;
The mainframe will FTP them to a folder. In that folder we will need the script to look and see if the specific file name is there and load it to the correct table.
Can anyone pls help me out with... (2 Replies)
friends , need a shell script to create a disk load. can any one pls guide me with how this can be implemented. Pls provide the concept. from there i will try to design my script. (3 Replies)
Hi,
I have a piece of shell script which will connect to mysql database and execute a load statement(which will load datas in a file to the database table).The code is working and the data is in the tables.
Now my requirement is, i need to grab the output from the load statement... (4 Replies)
Ok so I have files that are going to land on /apps/, but I need to load balance them so I need to load balance them to four different folders.
The three file extensions I get are .mpe .mpd and mpf which will land here... /apps/ I can't move them until the mpf is there it triggers the next... (8 Replies)
hi all
need your help.
I am wrting a script that will load data into the table.
then on another load will append the data into the existing table.
Regards
Ankit (1 Reply)
Hi,
I am a beginner. I need an example of using DB2 Load Utility in Shell script.
I appreciate if anyone could help me about it.
Thanks,
Paris (0 Replies)
Guys im trying to create a script that will load ballance 3 devices. Unix is still very alien to me having only been in this position for 3 months now. Not to mention the fact that i've been thrown in the deep end. Anyway so far i've managed to make a script that looks at all 4 devices and make a... (1 Reply)
10-25-2013
Balance the load of files across queues using shell script