Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

While loop - how to run processes one after another (2nd starts after first completes, and so on)

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting While loop - how to run processes one after another (2nd starts after first completes, and so on)
# 8  
Old 02-20-2013
Below is the terminal part of stdout following including the status of the usearch on the fifth file, followed by output of the following three commands, which I appended to the end of my script.

Unfortunately, it's not clear to me what information about usearch would be useful to your efforts to help me. It's a program that aligns sequences in one file to sequences in another file. 32 bit (the 64 bit version would likely do what I need, but it's expensive whereas the 32 bit version is free).USEARCH manual

Thanks for your efforts here.

Code:
ps -eo rss,vsz,pid,comm | sort -n | tail
ps -eo pcpu,pid,comm | sort -n | tail
uname

Quote:
00:03 94Mb 100.0% Masking
00:07 94Mb 100.0% Word stats
00:12 348Mb 100.0% Build index
00:14 2.1Gb 100.0% Searching, 12.4% matched
5080 26344 2113 hald
5216 133044 3006 gconfd-2
7132 242676 3025 polkit-gnome-au
7632 263560 3003 gnome-session
8420 268220 3024 gnome-power-man
9636 284108 3021 metacity
12428 345924 3011 gnome-settings-
15476 379380 3026 gdm-simple-gree
24588 125504 2982 Xorg
372812 2981496 7658 usearch_linux
0.0 9578 ps
0.0 9579 sort
0.0 9580 tail
0.0 95 md_misc/6
0.0 96 md_misc/7
0.0 97 khungtaskd
0.0 98 kswapd0
0.0 99 ksmd
0.0 9 ksoftirqd/1
%CPU PID COMMAND
Linux
# 9  
Old 02-20-2013
Disk is cheap, can you make more swap on some other local file system? http://www.cyberciti.biz/faq/linux-a...ap-file-howto/
# 10  
Old 02-20-2013
Code:
00:14 2.1Gb 100.0% Searching, 12.4% matched

That's interesting, how the process has managed to rename itself 'searching'. I wonder if it forks every once in a while to tell ps how much farther it's gotten...

Whatever it is, it's on the hair's edge of biting the 32-bit memory limit and dying... There may be a reason the 32-bit version is free. More swap won't help, since there's still a per-process limit in 32-bit.

Does this process quit after it's finished? You can pgrep for 'Searching'...
# 11  
Old 02-20-2013
If the limit is per process, allocating all of the 4G address space, then running in parallel or not has no effect, and yes, swap does not add address space.

If they crash because they exhaust swap running in parallel, more swap may help, but in the end they may thrash with excess paging unless run singly.

It is pecular that the process gives its parent the slip, almost like it is registering the search with a server and exiting.
# 12  
Old 02-20-2013
4 gigs in theory, in practice often half of that due to what ranges have been reserved for what purposes. No more room for more heap but plenty left for stack, etc.
# 13  
Old 02-20-2013
Yes, they do no seem to be flexible about allowing you either a 3.9G heap, stack or mmap(). Even in 64 bit compilers, there are options for several different pointer options with varying sub-64-bit address ranges.
# 14  
Old 02-20-2013
One (hacky) method to force each iteration of the loop to wait until usearch and all of its children have exited is to use strace -f.

A simple demonstration ...


sneaky.sh: prints 1 and exits while a sleeping subshell is left behind to print 2 at a later time.
Code:
#!/bin/sh

echo 1
( sleep 5; echo 2 ) &


loop.sh: runs sneaky.sh, almost certainly completing all loop iterations before any of the sleeping subshells awake.
Code:
#!/bin/sh

for i in 1 2 3; do
    ./sneaky.sh
done

Sample run:
Code:
$ ./loop.sh
1
1
1
$2
2
2


loop-strace.sh: exploits the fact that strace -f does not return until all children have exited:
Code:
#!/bin/sh

for i in 1 2 3; do
    strace -f -o /dev/null -e trace=process ./sneaky.sh
done

Sample run:
Code:
$ ./loop-strace.sh
1
2
1
2
1
2
$

If you are interested in seeing the processes that usearch is creating and how it's managing them, use a real file instead of /dev/null.

Regards,
Alister
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need to write a shell script that starts one, then kills it, then starts another?

This is on a CentOS box, I have two scripts that need to run in order. I want to write a shell script that calls the first script, lets it run and then terminates it after a certain number of hours (that I specify of course), and then calls the second script (they can't run simultaneously) which... (3 Replies)
Discussion started by: btramer
3 Replies

2. Homework & Coursework Questions

When I run the script, the cursor starts on the wrong line?

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: It's a shell script using a looping logic, trap, tput, if, while. Most of the scripts in this book aren't written... (2 Replies)
Discussion started by: ckleinholz
2 Replies

3. Shell Programming and Scripting

Array Variable being Assigned Values in Loop, But Gone when Loop Completes???

Hello All, Maybe I'm Missing something here but I have NOOO idea what the heck is going on with this....? I have a Variable that contains a PATTERN of what I'm considering "Illegal Characters". So what I'm doing is looping through a string containing some of these "Illegal Characters". Now... (5 Replies)
Discussion started by: mrm5102
5 Replies

4. UNIX for Dummies Questions & Answers

Maximum no of processes that can run

Is there any limit in UNIX that only a MAXIMUM no of processes can run at a time? If so, it exceeds then what would be the error we would receive? (4 Replies)
Discussion started by: jansat
4 Replies

5. UNIX and Linux Applications

how to run more than two processes parallely

I would like to call a function called CIRCLE which is further beind called by other function but in a loop that CIRCLE fuction is being called. And this CIRCLE function starts another process which takes 3 hours to complete again, if i put that process in nohup &, I can go to the next command... (2 Replies)
Discussion started by: venugopalsmartb
2 Replies

6. UNIX for Dummies Questions & Answers

Two questions. First one; What are the ways in which a program starts to run.

This is my first post here so hello everyone! I know that a command of the programs name can start a program and clicking on a icon in GUI can as well as a startup shell script but how do I educate myself of the method that starts an application? Does the GUI run a script? What are the ways/ way... (2 Replies)
Discussion started by: theKbStockpiler
2 Replies

7. UNIX for Dummies Questions & Answers

how can I run something as root (modprobe, to be exact) every time computer starts.

I have the root password for my box, but I'm ignorant. So, every time I start my computer, I have to run this command /sbin/modprobe fuse as su, so that I can do other stuff (like mount remote directories locally using sshfs) I guess there's some file, like .bashrc, only it's applicable... (4 Replies)
Discussion started by: tphyahoo
4 Replies

8. UNIX for Advanced & Expert Users

script to run different shells which run different processes

Hi, Would like to ask the experts if anyone knows how to run a script like this: dtterm -title shell1 run process1 on shell1 dtterm -title shell2 run process2 on shell2 cheers! p/s: sorry if i used the wrong forum, quite concussed after watching world cup for several nights; but I... (2 Replies)
Discussion started by: mochi
2 Replies

9. UNIX for Advanced & Expert Users

Run away processes

Hi, My server runnning on SUN Solaris rel.5.5.1. I have been facing this issues for years. I have some Xbase databases running on the server. User is using emulation software to telnet to server for accessing application. If user logout application abnormally - by closing windows session, then... (2 Replies)
Discussion started by: lowtaiwah
2 Replies

10. Shell Programming and Scripting

How to run processes in parallel?

In a korn shell script, how can I run several processes in parallel at the same time? For example, I have 3 processes say p1, p2, p3 if I call them as p1.ksh p2.ksh p3.ksh they will run after one process finishes. But I want to run them in parallel and want to display "Process p1... (3 Replies)
Discussion started by: sbasak
3 Replies
Login or Register to Ask a Question