04-12-2007
Quote:
Originally Posted by Amruta Pitkar
Hi MatrixMadhan,
I am not sure how to code this exactly....
Also..from what I read...do I have to stop the background processes explicitly..
What if there are some problems with the data parsing...how will the error.log will be created...
Can u explain more, guide for some sample scripts ?
There is no need to stop your background process unless they is a situation to be done so!
would something like this be of help!
split the files ...100k to 10 ( 10k s )
now the process that was used to run against the 100k sample, should be used against each of the smaller chunks
i=1
while[ $i -le 10 ]
do
/somedir/process chunk$i & //Make that a background process
i=$(($i + 1))
done
Now with the above loop, smaller chunks are fed to the individual process and would start processing.
By default background process have a lower priority when compared to the foreground process.
You need to arrive / determine at a threshold value ( more of a bench mark stuff ) where running several processes with smaller chunks is actually not bringing down the performance when compared to running it with a single chunk and just a single process.
Creating the error logs is as the usual way as you had been doing it for the foreground process!
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
OS - Sun OS7
What sources can I go to to figure out what is the maximun number of processes for OS7 with 2GB of memory.
I believe it is 64K processes, but this number reflects resources being swaped.
Any help is appreciated
SmartJuniorUnix (1 Reply)
Discussion started by: SmartJuniorUnix
1 Replies
2. UNIX for Dummies Questions & Answers
we've got solaris 5.6 installed in a ultra 5 box that serves as gateway server going to the main unix box.
just like to find out how to determine the number of concurrent terminal connections and processes that the ultra 5 box can handle? and handling at present time?
thanks in advance! (1 Reply)
Discussion started by: eddie_villarta
1 Replies
3. UNIX for Advanced & Expert Users
Hi,
I notice in my Sun Solaris 8 sparc workstatin, I am able to login concurrently using a same user ID.
Is there a way to disallow this? That is, at anyone time, the user can have only 1 login session.
How can it be done?
Thanks (10 Replies)
Discussion started by: champion
10 Replies
4. Shell Programming and Scripting
Hi
I have a ksh that can have multiple instances running at the same time.
The script (each instance) uses the SAME log file to write to.
Should this cause a problem or is the ksh clever enough to queue write requests to the file?
Thanks.
GMMIKE (2 Replies)
Discussion started by: GNMIKE
2 Replies
5. UNIX for Advanced & Expert Users
Is there a way to monitor certain processes and if they hang too long to kill them, but certain scripts which are expected to take a long time to let them go?
Thank you
Richard (4 Replies)
Discussion started by: ukndoit
4 Replies
6. Solaris
Hi
Is there an easy way to identify and group currently running processes into OS processes and APP processes. Not all applications are installed as packages.
Any free tools or scripts to do this?
Many thanks. (2 Replies)
Discussion started by: wilsonee
2 Replies
7. AIX
Hi,
on normal (non concurrent) vgs, it's possible to extend a lun on san-storage , and use chvg -g to rewrite vgda, and use disks with the new size for lvm operations
is the same procedure possbile on a hacmp-cluster (2 node in our case) with concurrent vgs in active/passive mode?
cheers... (5 Replies)
Discussion started by: funksen
5 Replies
8. Shell Programming and Scripting
Hi all,
I have a folder with sql files that need to be inserted in a DB with SQL*Plus. The thing is that it takes too long to insert them all one by one, so I want to insert them five at a time.
Currently what I use is this:
for $FILENAME in *.sql
do
sqlplus -s $DBUSER@$SID << EOF
... (0 Replies)
Discussion started by: Tr0cken
0 Replies
9. Shell Programming and Scripting
Hi,
The problem detail is follows
I have three individual scripts .
SCRIPT A
sh -x sqoop_channels_nc_daily_01.sh & sh -x sqoop_channels_nc_daily_02.sh & sh -x sqoop_channels_nc_daily_03.sh
SCRIPT B
sh -x sqoop_contacts_nc_daily_01.sh & sh -x sqoop_contacts_nc_daily_02.sh & sh -x... (1 Reply)
Discussion started by: H_bansal
1 Replies
10. AIX
Hi,
What are the differences between concurrent and enhanced concurrent VGs.?
Any advantages of enhanced concurrent VG over normal concurrent vg
Regards,
Siva (2 Replies)
Discussion started by: ksgnathan
2 Replies
LEARN ABOUT OSX
getpriority
GETPRIORITY(2) BSD System Calls Manual GETPRIORITY(2)
NAME
getpriority, setpriority -- get/set program scheduling priority
SYNOPSIS
#include <sys/resource.h>
int
getpriority(int which, id_t who);
int
setpriority(int which, id_t who, int prio);
DESCRIPTION
The scheduling priority of the process, process group, or user as indicated by which and who is obtained with the getpriority() call and set
with the setpriority() call. Which is one of PRIO_PROCESS, PRIO_PGRP, or PRIO_USER, and who is interpreted relative to which (a process
identifier for PRIO_PROCESS, process group identifier for PRIO_PGRP, and a user ID for PRIO_USER). A zero value of who denotes the current
process, process group, or user. prio is a value in the range -20 to 20. The default priority is 0; lower priorities cause more favorable
scheduling.
The getpriority() call returns the highest priority (lowest numerical value) enjoyed by any of the specified processes. The setpriority()
call sets the priorities of all of the specified processes to the specified value. Only the super-user may lower priorities.
Additionally, the current thread or process can be placed in a background state by specifying PRIO_DARWIN_THREAD or PRIO_DARWIN_PROCESS for
which. Only a value of zero (the current thread or process) is supported for who when setting or getting background state. prio is either 0
(to remove current thread from background status) or PRIO_DARWIN_BG (to set current thread into background state). When a thread or process
is in a background state the scheduling priority is set to the lowest value, disk IO is throttled (with behavior similar to using
setiopolicy_np(3) to set a throttleable policy), and network IO is throttled for any sockets opened after going into background state. Any
previously opened sockets are not affected. The getpriority() call returns 0 when current thread or process is not in background state or 1
when the current thread is in background state. Any thread or process can set itself into background state.
RETURN VALUES
Since getpriority() can legitimately return the value -1, it is necessary to clear the external variable errno prior to the call, then check
it afterward to determine if a -1 is an error or a legitimate value. The setpriority() call returns 0 if there is no error, or -1 if there
is.
ERRORS
Getpriority() and setpriority() will fail if:
[EINVAL] Which is not one of PRIO_PROCESS, PRIO_PGRP, PRIO_USER, PRIO_DARWIN_THREAD, or PRIO_DARWIN_PROCESS.
[EINVAL] Who is not a valid process, process group, or user ID.
[EINVAL] Who is not 0 when which is PRIO_DARWIN_THREAD or PRIO_DARWIN_PROCESS.
[ESRCH] No process can be located using the which and who values specified.
In addition to the errors indicated above, setpriority() will fail if:
[EACCES] A non super-user attempts to lower a process priority.
[EPERM] A process is located, but neither its effective nor real user ID matches the effective user ID of the caller.
LEGACY SYNOPSIS
#include <sys/types.h>
#include <sys/resource.h>
The include file <sys/types.h> is necessary.
int
getpriority(int which, int who);
int
setpriority(int which, int who, int value);
The type of who has changed.
SEE ALSO
nice(1), fork(2), setiopolicy_np(3), compat(5), renice(8)
HISTORY
The getpriority() function call appeared in 4.2BSD.
4th Berkeley Distribution June 4, 1993 4th Berkeley Distribution