Slow Producer - Fast consumer


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Slow Producer - Fast consumer
# 1  
Old 11-05-2010
Slow Producer - Fast consumer

I would like to loop through a set of directories, performing operation(s) on each one. The basic script** is

dirs=`find . -name .svn -print`

for f in $dirs; do
echo "Processing $f directory .."
done

Fine and dandy, but here is the problem: the find expression must complete before the do ... done begins. The find process is relatively slow. I would like to have the do .. done work on entries as they are appended to the list. So in essence there are two threads, the find works in the background, and the do consumes $dirs until empty.

How would the do .. done loop "know" that the $dirs is still being populated? Maybe this needs to be changed completely using another method.

Thanks in advance.



** Some of you will recognize the ".svn", as Subversion stores additional copies of data on the local machine in ".svn" directories.
# 2  
Old 11-05-2010
Enter the pipe:
Code:
find . . . | while read d
do
 . . .
done

or better yet, if cmd can do the process, like rm or fgrep or the like:
Code:
find . . . | xargs -n999 'cmd'

# 3  
Old 11-05-2010
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Homework & Coursework Questions

Abnormal producer consumer problem driving me nuts

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: normally, i hate asking someone to do my homework for me but am getting desperate right now. i have a project... (1 Reply)
Discussion started by: alexantosh
1 Replies

2. Homework & Coursework Questions

producer consumer semaphore

Control two exclusively shared resources(semaphore). The two resources are two files. The producer will write even numbers to one file, and odd numbers to another one. The consumer respectively reads from each file until it gets 5 even numbers and 5 odd numbers. Can any one help me with the... (1 Reply)
Discussion started by: gokult
1 Replies

3. UNIX for Dummies Questions & Answers

producer consumer

Control two exclusively shared resources. The two resources are two files. The producer will write even numbers to one file, and odd numbers to another one. The consumer respectively reads from each file until it gets 5 even numbers and 5 odd numbers. Can any one help me with the code. ... (3 Replies)
Discussion started by: gokult
3 Replies

4. Programming

producer consumer semaphore

Control two exclusively shared resources(semaphore). The two resources are two files. The producer will write even numbers to one file, and odd numbers to another one. The consumer respectively reads from each file until it gets 5 even numbers and 5 odd numbers. Can any one help me with the... (0 Replies)
Discussion started by: gokult
0 Replies

5. UNIX for Advanced & Expert Users

producer consumer

Control two exclusively shared resources(semaphore). The two resources are two files. The producer will write even numbers to one file, and odd numbers to another one. The consumer respectively reads from each file until it gets 5 even numbers and 5 odd numbers. Can any one help me with the... (0 Replies)
Discussion started by: gokult
0 Replies

6. Shell Programming and Scripting

producer consumer

Control two exclusively shared resources. The two resources are two files. The producer will write even numbers to one file, and odd numbers to another one. The consumer respectively reads from each file until it gets 5 even numbers and 5 odd numbers. Can any one help me with the code. ... (1 Reply)
Discussion started by: gokult
1 Replies

7. Programming

producer consumer

Control two exclusively shared resources. The two resources are two files. The producer will write even numbers to one file, and odd numbers to another one. The consumer respectively reads from each file until it gets 5 even numbers and 5 odd numbers. Can any one help me with the code. ... (0 Replies)
Discussion started by: gokult
0 Replies

8. UNIX for Dummies Questions & Answers

Slow Producer - Fast consumer

I would like to loop through a set of directories, performing operation(s) on each one. The basic script** is dirs=`find . -name .svn -print` for f in $dirs; do echo "Processing $f directory .." done Fine and dandy, but here is the problem: the find expression must complete... (7 Replies)
Discussion started by: jakeo25
7 Replies

9. Programming

producer-consumer problem

The intention of the program. Create N threads with 2 shared memories. One shared memory to write, one shared memory to recieve. Consumer creates 2 shared memory to share with producer. I need H threads for the producer as well(somebody help on it..) Also another question, would segment_id... (2 Replies)
Discussion started by: joey
2 Replies

10. UNIX for Dummies Questions & Answers

Producer/Consumer

Hi all, I have an Producer/Consumer program in C(using shared memory and semaphores).It is working fine with sleep calls after every read/write operation.Each one is a independent program.Now if i remove the sleep calls in consumer,it goes on waiting in the loop till the producer puts some valid... (3 Replies)
Discussion started by: poorni_uma
3 Replies
Login or Register to Ask a Question