Retention of Variable Value when a script is called by different processes in parallel- Linux 2.6.9


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Retention of Variable Value when a script is called by different processes in parallel- Linux 2.6.9
# 1  
Old 03-18-2010
Retention of Variable Value when a script is called by different processes in parallel- Linux 2.6.9

Hi,

I have a generic FTP script which will be called by 28 different processes in parallel (through a GUI tool) may or may not be at the exact moment (there could be a delay of about a minute or so).

Code:
 
./FTP.ksh 1 (1 through 28)

This script after importing file from remote m/c inserts a record for that particular file in an Oracle table (see Fig 2). The "ID" column takes value from 1 ... 28. Column "SEQ" takes value from Oracle Sequence generator (so its unique). Column "TABLE_RECORD_COUNT" is inserted with '0'."Column "FILE_STATUS_FLAG" will have default value of 'A' (arrived) while inserting.

Code:
FILE_NAME|ID|SEQ|FILE_STATUS_FLAG|FILE_RECORD_COUNT|TABLE_RECORD_COUNT|CREATED_DATE|LAST_UPDATED_DATE|FILE_SIZE
r1_file1_<timestamp>.gz|1|1|A|899800|0|16-Mar-10|16-Mar-10|4000000
r1_file2_<timestamp>.gz|2|2|A|894700|0|16-Mar-10|16-Mar-10|187852398
r2_file1_<timestamp>.gz|3|3|A|799800|0|16-Mar-10|16-Mar-10|187852398
r2_file2_<timestamp>.gz|4|4|A|799720|0|16-Mar-10|16-Mar-10|187852398
r1_file1_<timestamp>.gz|1|5|A|899800|0|17-Mar-10|17-Mar-10|4000000
r1_file2_<timestamp>.gz|2|6|A|894700|0|17-Mar-10|17-Mar-10|187852398
r2_file1_<timestamp>.gz|3|7|A|799800|0|17-Mar-10|17-Mar-10|187852398
r2_file2_<timestamp>.gz|4|8|A|799720|0|17-Mar-10|17-Mar-10|187852398

Fig 2 (INSERT)

At the end of the process (which loads data from the file to a table) again a Unix script will be called which will update the record inserted by the FTP script.

Now the updating script some how should know which record to update? And then it will update the table with values as below:

Code:
FILE_NAME|ID|SEQ|FILE_STATUS_FLAG|FILE_RECORD_COUNT|TABLE_RECORD_COUNT|CREATED_DATE|LAST_UPDATED_DATE|FILE_SIZE
r1_file1_<timestamp>.gz|1|1|P|899800|899800|16-Mar-10|16-Mar-10|4000000
r1_file2_<timestamp>.gz|2|2|P|894700|894700|16-Mar-10|16-Mar-10|187852398
r2_file1_<timestamp>.gz|3|3|P|799800|799800|16-Mar-10|16-Mar-10|187852398
r2_file2_<timestamp>.gz|4|4|P|799720|799720|16-Mar-10|16-Mar-10|187852398
r1_file1_<timestamp>.gz|1|5|P|899800|899800|17-Mar-10|17-Mar-10|4000000
r1_file2_<timestamp>.gz|2|6|P|894700|894700|17-Mar-10|17-Mar-10|187852398
r2_file1_<timestamp>.gz|3|7|P|799800|799800|17-Mar-10|17-Mar-10|187852398
r2_file2_<timestamp>.gz|4|8|P|799720|799720|17-Mar-10|17-Mar-10|187852398

Fig 3 (UPDATE)

The script has updated column "FILE_STATUS_FLAG" to 'P' (processed) and column "TABLE_RECORD_COUNT" by executing "SELECT COUNT(1) FROM TABLE". Now I wonder if there's a way of remebering SEQ value till one process gets complete. Even if another process has kicked off the FTP script. Please suggest.

-dips
# 2  
Old 03-18-2010
The loading script, which calls the updating script, should "communicate" the value of the sequence, after inserting the value with sequence_name.nextval,
it should save/memorize this value (calling sequence_name.currval) and pass it to the updating process.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

(bash) Script Processes in Parallel

Hello all, I tried to parralise my treatments but after a while 'ps -ef' display all child process <defunct> (zombie) Parent bash script to process all files (>100000) in directory: for filename in /Data/*.txt; do ./child_pprocess.sh $filename & done exit(0)I understand that the... (1 Reply)
Discussion started by: namnetes
1 Replies

2. Shell Programming and Scripting

Shell script to pass the config file lines as variable on the respective called function on a script

I want to make a config file which contain all the paths. i want to read the config file line by line and pass as an argument on my below function. Replace all the path with reading config path line by line and pass in respective functions. how can i achieve that? Kindly guide. ... (6 Replies)
Discussion started by: sadique.manzar
6 Replies

3. Shell Programming and Scripting

Passing variable from called script to the caller script

Hi all, Warm regards! I am in a difficult situation here. I have been trying to create a shell script which calls another shell script inside. Here is a simplified version of the same. Calling Script. #!/bin/ksh # want to run as a different process... (6 Replies)
Discussion started by: LoneRanger
6 Replies

4. Shell Programming and Scripting

Passing the value of variable which is read from command line in called script

Hi, I am calling a Perl script in my shell script. When Perl script is executed it asks for a answer to be entered by user from terminal. How can i pass that value from my shell script ?? I know I can change perl script to default the answer but i dont have access to do that so only option i... (5 Replies)
Discussion started by: varun22486
5 Replies

5. Shell Programming and Scripting

Keep up constant number of parallel processes

Hi guys, I am struggling with adapting my script to increase the performance. I created a ksh script to process a lot of files in parallel. I would like to know how can I do in such a way that a constant number of processes is always up (until all is finished). What I have is (not actual... (8 Replies)
Discussion started by: lurkerro
8 Replies

6. Shell Programming and Scripting

Deleting all the parallel processes launched when the main script receives a ctrl+c

Hi, I have a shell script that creates 2 parallel processes. When I press ctrl+c, i want the parallel process to get killed as well. #!/bin/bash cmd1="script1.py" cmd2="script2.py" ${cmd1} & pid1=$! echo ${pid1} ${cmd2} & pid2=$! (7 Replies)
Discussion started by: sana.usha
7 Replies

7. UNIX for Dummies Questions & Answers

How to retrieve the value of variable in shell script which is called by crontab

There are two files one is shell script (sample.sh) and another is configuration file (sampl_conf.cfg) configuration file contains one variable $FTP_HOME. the value of this variable vaires for user to user. If user is say jadoo then value is /home/jadoo/ftp/, for user1 - /home/user1/ftp. The... (4 Replies)
Discussion started by: jadoo_c2
4 Replies

8. Shell Programming and Scripting

environment variable in shell script called through crontab

Please help me on below.. https://www.unix.com/shell-programming-scripting/141533-retrieve-value-environment-variable-shell-script-called-crontab.html#post302442024 I'm still here. I can still see you! (0 Replies)
Discussion started by: jadoo_c2
0 Replies

9. Shell Programming and Scripting

Retrieve the value of environment variable in shell script which called from crontab

There are two files one is shell script (sample.sh) and another is configuration file (sampl_conf.cfg) configuration file contains one variable $FTP_HOME. the value of this variable vaires for user to user. If user is say jadoo then value is /home/jadoo/ftp/, for user1 - /home/user1/ftp. The... (0 Replies)
Discussion started by: jadoo_c2
0 Replies

10. Shell Programming and Scripting

How to run processes in parallel?

In a korn shell script, how can I run several processes in parallel at the same time? For example, I have 3 processes say p1, p2, p3 if I call them as p1.ksh p2.ksh p3.ksh they will run after one process finishes. But I want to run them in parallel and want to display "Process p1... (3 Replies)
Discussion started by: sbasak
3 Replies
Login or Register to Ask a Question