Doubt about multiple processes


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Doubt about multiple processes
# 1  
Old 10-27-2007
Doubt about multiple processes

Suppose that I am performing some operation on an sql database. Lets say process of Searching and then if a value is found, updating it... Now, when I have some millions of records on which the operation has to be performed... Does it help to spawn multiple processes each executing the same script(assuming that I coded the script in php)? Will that make a difference in speed of execution?

Last edited by Legend986; 10-28-2007 at 12:05 AM..
# 2  
Old 10-28-2007
Quite the reverse, a database will serialise access and as you access records will add them to the list of records touched by the current transaction.

Rather than doing searches and updates, you should let the database do the work with an "update .... where ...." statement. Let the database do all the searches internally and update internally.
# 3  
Old 10-28-2007
Oops... Sorry... I meant to say that... What I was asking is that if I spawn new processes that execute this script, will the job get done faster? The computer I am referring to is a full fledged server which only I am using currently...
# 4  
Old 10-28-2007
i would not expect so if the transactions touch the same records.
# 5  
Old 10-28-2007
Oh... For example consider the following:

Table 1:
121::::ABCDE::::::0.2
230:::::ASDS:::::::0.6
xxx::::::jaksd:::::::0.7
xxx::::::dasdd:::::::0.3
xxx::::::asaaa:::::::0.1
xxx::::::ddddd:::::::0.5

Table 2:
121::::ABCDE
230:::::ASDS
343::::::jaksd
545::::::dasdd
656::::::asaaa
344::::::ddddd


What my script is doing is takes the second column of Table 1 and searches for a key in the table 2 and then when it finds a relvant key it updates table 1 with that value... So basically, I'm trying to merge the two tables with the only difference is that the number of rows in both the tables are not the same due to some corruption in table 2... What is the fastest way to achieve this when I have to perform millions of such operations?
# 6  
Old 10-28-2007
Can you provide some more information ?

Which database are you using ?
and what about the status of the columns being used for searching and updating.

Are the fields indexed, if so obviously search is going to be faster.

I have tried this in Informix and not in Oracle.
When you have to do an update on a table : column based on a value from table2 : column2, there is an option in Informix to do parallelized search where you have to specify only the number of threads in which the search has to run.

And the approach is much faster on the indexed columns.

for n number of rows from 1 .. n
Based on ' m ' threads number of records would be split as
1 .. k, k+1 .. k<n> , and k <m> .. n

and the results are internally manipulated.

Would this help you ?
# 7  
Old 10-28-2007
Thank you so much for your concern... I'm using a MySQL database... As for the status of the columns, I not quite sure what you meant, but they are of the following types:

Table_1
Column_1: int
Column_2: varchar(18)
Column_3: decimal

Table_2
Column_1: int
Column_2: varchar(18)

I had a couple of text files from which i used a php script + regular expressions to grab the text I want and then inserted them into the database's table_1 and table_2. Hope I was able to convey my requirement...
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

need help ps -e on multiple processes

:)Hi there, I am new to scripting and wanted to see if someone can show me how to grep on multiple processes and send the output to a file in /home/mydir/output. I am aware of ps -ef | grep on 1 process but need help looking up multiple processes, can you use this command ps -elf | grep |pid1... (4 Replies)
Discussion started by: abbya
4 Replies

2. Solaris

Multiple hanged FTP processes

Hello people, I got one problem with a script. I have a script which runs every five mins and in the script an ftp process is invoked which sends files to a particular location. The problem is that the ftp process hangs every now and then which causes the whole script to hang. As the... (4 Replies)
Discussion started by: m_usmanayub
4 Replies

3. Shell Programming and Scripting

kill multiple processes by name

Want to kill multiple processes by name. for the example below, I want to kill all 'proxy-stagerd_copy' processes. I tried this but didn't work: >> ps -ef|grep proxy_copy root 991 986 0 14:45:34 ? 0:04 proxy-stagerd root 1003 991 0 14:45:49 ? 0:01... (2 Replies)
Discussion started by: catalinawinemxr
2 Replies

4. Programming

Capture stdout from multiple processes

I have a number of binaries which I currenlty have no control over. They alright data to stdout. I would like to kick off any number of these binaries and capture and process their stdout. Doing this for one process is straight forward for example - comments and error checking removed for... (6 Replies)
Discussion started by: dvales
6 Replies

5. Programming

Creating Multiple Processes

I am having problems creating multiple forks. I want create a certain number of forks, each call a program and each wait for a different value. How is this accomplished my loop is not doing the trick. for (i = 0; i < 5; i++) { if (fork() < 0) { //print error } ... (3 Replies)
Discussion started by: Vikings1201
3 Replies

6. Programming

Program to spawn multiple processes

I'm trying to make a program that will spawn multiple child processes then exit. I'm having trouble figuring out how to do this since after I fork, the child process begins running the program again (never ending). int main(void){ for(int i = 0; i < 3; i++){ fork(); }... (1 Reply)
Discussion started by: cagney58
1 Replies

7. UNIX for Dummies Questions & Answers

Running multiple processes in Linux

Hi guys, I want to run the multiple scripts at the same time using a ksh script. For example, I have three scripts to run: a.ksh, b.ksh and c.ksh How to start the above 3 scripts simultaneously and then on the completion of the above scripts I have other tasks to schedule. Thanks Gary (6 Replies)
Discussion started by: abcabc1103
6 Replies

8. Shell Programming and Scripting

multiple processes overlap

Hello I've got a script that creates multiple processes, in ksh, to bcp out 6 tables at a time. In the script, we write messages to the log to show our progress; most of the time, the log messages are nice and neat with one per line, like they should be. But every once in awhile, at random, the... (2 Replies)
Discussion started by: stonemonolith
2 Replies

9. UNIX for Advanced & Expert Users

Crontab spawning multiple at processes

Hi - I need help. My user crontab is spawning multiple at processes (and multiple mencoder program starts, that exit, then restart, repeatedly), locking up my system. For example I have this entry in my crontab: $ sudo crontab -u victoria -e * * * * * ~/recordings/pvr1 * * * * *... (10 Replies)
Discussion started by: gstuart
10 Replies

10. UNIX for Advanced & Expert Users

processes statistics doubt

The following is the o/p I got when i run prstat -t command on SunOs5.8 NPROC USERNAME SIZE RSS MEMORY TIME CPU 2 shah 3152K 2384K 0.0% 0:00.00 0.0% 8 oracle 14G 13G 100% 5:09.04 0.0% 35 root 88M 54M 0.4% 0:54.27 0.0% 1 bb ... (3 Replies)
Discussion started by: shahnazurs
3 Replies
Login or Register to Ask a Question