Login or Register to Ask a Question and Join Our Community


UNIX for Beginners Questions & Answers

Executing statements inside forloop parallely

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Executing statements inside forloop parallely
# 1  
Old 04-29-2018
Linux Executing statements inside forloop parallely
Hi All,

I have made a forloop as given below.

Code:
if [[ -f "serv_AAA_DS.ldif" && -s "serv_AAA_DS.ldif" ]]; then 
    echo -e "serv_AAA_DS.ldif file created and will be splitted into 8 smaller files for ldapadd \n"
    sh ./file_splitter.sh
    for (( w=1; w <= 8; ++w ))
    do 
    	nohup ldapadd -x -c -h PL0 -p 389 -D "$LDAP_USER" -w $LDAP_PWD -f serv_AAA_DS.ldif_$w > serv_AAA_DS.ldif_$w.log
    done 
    wait
    serv_AAA_DS_count=`cat serv_AAA_DS.ldif | grep '^dn: ' | wc -l`
    echo -e "Count of serv AAA entries present in the Input file is $serv_AAA_DS_count, cross-check the value with total_dm.sh counters & also the check the failures if any in serv_AAA_DS.ldif_*.log \n"
else 
    echo -e "serv_AAA_DS.ldif is empty and will be removed \n"
    rm serv_AAA_DS.ldif
fi

This one runs sequentially waiting for each iteration to finish inside the forloop, I want to execute all the 8 ldapadd parallel and then the code should wait for all the 8 threads to complete, before it moves to the next lines.

Can you let me know what changes to be done on this to achieve the above mentioned.

Thank you in advance.
Moderator's Comments:
Mod Comment Please use code tags, not bolded text.
Last edited by jim mcnamara; 04-29-2018 at 11:03 AM..
# 2  
Old 04-29-2018
Assuming your code works correctly, I cannot verify that at the moment:

Code:
nohup ldapadd -x -c -h PL0 -p 389 -D "$LDAP_USER" -w $LDAP_PWD -f serv_AAA_DS.ldif_$w > serv_AAA_DS.ldif_$w.log  &

Add an ampersand (& red above) at the end of the line. This works for bash which seems to be the shell you use for this code. If it is another different shell please let us know.

Note:
Code:
> serv_AAA_DS.ldif_$w.log

will overwrite the log file every time the script runs, so you lose the content of previous logs. That may or may not be okay.
Last edited by jim mcnamara; 04-29-2018 at 12:19 PM..
This User Gave Thanks to jim mcnamara For This Post:
Kamesh G
# 3  
Old 04-29-2018
@jim mcnamara - One more query, I have a script.sh that waits for user input to begin. i.e. it has "read" in the script. Now im not able to log the script.sh out as when i do ./script.sh > out.log it doesnt prompt for the input. Also the preferrable mode is to run it as "nohup ./script.sh &" . Can you help me how to log all the outputs from the script ????
# 4  
Old 04-29-2018
Quote:
Originally Posted by Kamesh G
Also the preferrable mode is to run it as "nohup ./script.sh &" . Can you help me how to log all the outputs from the script ????
The real question is not where the output goes (when you start a process with nohup per default its output goes to ./nohup.out) but where the input is supposed to come from.

When you start a process on the commandline this commandline is displayed in a terminal. Per default input comes from this terminal (the keyboard) and output goes to this terminal (the screen). Starting a process with nohup cuts this connection to a terminal - actually this is the whole point of nohup: allowing a process to continue even if its terminal is closed ("hangs up").

Bottom line: if you want to start the process with nohup you need something which replaces the terminal as a source of input. That can be some file, a named pipe or whatever - but something has to be there and you need to use redirection to make the process use this instead of waiting indefinitely.

I hope this helps.

bakunin
This User Gave Thanks to bakunin For This Post:
Kamesh G
# 5  
Old 04-30-2018
@bakunin - thank you for the help. one more query while the script runs. I'm getting the below

script:
Code:
    for (( w=1; w <= 8; ++w ))
    do 
    	nohup ldapadd -x -c -h PL0 -p 389 -D "$LDAP_USER" -w $LDAP_PWD -f serv_AAA_DS.ldif_$w > serv_AAA_DS.ldif_$w.log &
    done 
    wait

To be suppressed:-
Code:
nohup: redirecting stderr to stdout
nohup: redirecting stderr to stdout
nohup: redirecting stderr to stdout
nohup: redirecting stderr to stdout
nohup: redirecting stderr to stdout
nohup: redirecting stderr to stdout
nohup: redirecting stderr to stdout
nohup: redirecting stderr to stdout

Is there a way to suppress this.???
Last edited by vbe; 04-30-2018 at 04:05 AM.. Reason: more code tags
# 6  
Old 04-30-2018
Code:
nohup: redirecting stderr to stdout

You get that message as UNIX is known to be quite verbous, to say :
As you havent defined anything special for stderr, it will redirect stderr to stdout which is normally your screen...
I would not suggest to redirect to /dev/null as you are in developing stage, you need to be able to read what errors you may get so the best is to redirect in a file, a separate one, or in your log
So your command line could be
Code:
nohup ldapadd -x -c -h PL0 -p 389 -D "$LDAP_USER" -w $LDAP_PWD -f serv_AAA_DS.ldif_$w \
> serv_AAA_DS.ldif_$w.log 2>&1 &

# 7  
Old 04-30-2018
Dears,

Please let me know what is the way to print the time taken to complete execution of the below script.

The below will run 8 parallel threads, would like to print the total time taken for all the threads to complete the execution.

Code:
    for (( w=1; w <= 8; ++w ))
    do 
    	nohup ldapadd -x -c -h PL0 -p 389 -D "$LDAP_USER" -w $LDAP_PWD -f serv_AAA_DS.ldif_$w > serv_AAA_DS.ldif_$w.log &
    done 
    wait
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Executing sed command inside a bash script

I want to run commands inside a bash script. An example is I want to pass the command in a string as regexp as an argument to the script, then run sed on the bash variable sed.sh regexp sed.sh "-i \"s/<p>//g\"" then call sed "$regexp" $fl (3 Replies)
Discussion started by: Kangol
3 Replies

2. Shell Programming and Scripting

Double forloop?

how do I do a double forloop or any other loops like this reads txt file a read first line read txt file b read first line then run command lun map $line1_from_txtA $line1_from_txtB exit (14 Replies)
Discussion started by: tdubb123
14 Replies

3. Shell Programming and Scripting

How to run scripts parallely inside shell script?

Hi , I have 4 scripts example script1,script2,script3,script4 . I have to run script1,script2 and script3 parallely since this 3 scripts dont have dependencies . Once script1,script2 and script3 got completed successfully , I have to trigger script4. Can someone help me on this how to... (10 Replies)
Discussion started by: vinothsekark
10 Replies

4. Shell Programming and Scripting

Executing multiple ssh commands inside a shell simultaneously

I would like to execute a commands in four different servers through ssh at a single instance(simultaneously). Below are the details with examples, ssh user1@server1 "grep xxxx logs" ssh user1@server2 "grep xxxx logs" ssh user1@server3 "grep xxxx logs" Each statement will take some... (4 Replies)
Discussion started by: Amutha
4 Replies

5. Programming

Executing a awk command inside perl script --

Hello experts I want to execute a awk command, which reads from txt files and sums the numbers from the first column for those listed only inside a <init> block -- The awk command is like awk '/<\/?init>/{x = !x}x{a++}x && a > 2{sum+=$1}END{printf"%E" "\n", sum} So, I want to execute... (2 Replies)
Discussion started by: Alkass
2 Replies

6. Shell Programming and Scripting

compare parallely

hi, i have folder backup and it contains 3 sub-directories , i want to compare /backup/1 with /recover/1 and /backup/2 with /recover/2 etc . parallelly with using shell scripting . pls help me thanks (2 Replies)
Discussion started by: shankr3
2 Replies

7. UNIX for Dummies Questions & Answers

Problem with executing command inside a cron job

Hi All, I have scheduled a script in cron which writes output to the below file. ....>> /data/Target/wrapper_invoke_ds_job_`date '+%Y%m%d'`.ksh_out 2>&1 But the date command is not getting resolved in the format specified. It just resolves to the following. wrapper_invoke_MQ_ds_job_Tue... (3 Replies)
Discussion started by: pkm_oec
3 Replies

8. Shell Programming and Scripting

executing 2 or more files inside another file parallely

hi All, i have 3 files file1 file2 file3 these 3 files are shell (KSH) scripts. i want to place all these 3 shell scripts in another file file99 and execute these files simultaneoulsy inside the file99 these 3 files should be present and executed simultaneously when i excute file99 (nohup... (1 Reply)
Discussion started by: dareman123
1 Replies

9. Shell Programming and Scripting

Problem in executing a comand liying inside a variable

Hi everyone! I need some help with my shell script :( I am sending a shell command from a html text input to a cgi. Then, I store it into a variable. For example var="ps -axu" echo `$var` This functions properly. But consider the following... var="ps -axu | grep root" Now, I want... (2 Replies)
Discussion started by: Nene
2 Replies

10. Shell Programming and Scripting

Executing statements after quit in FTP.

In my shell script I am doing FTP along with other process. But all the statements after the "quit" command in FTP are not getting executed. Example: echo "This is a shell script to FTP a file" #ftp -inv <server name> <<eol>> <logfilename> #user <userid> <password> #put <filename> #quit... (2 Replies)
Discussion started by: dharmesht
2 Replies
Login or Register to Ask a Question