Multiple jobs reading from same file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Multiple jobs reading from same file
# 1  
Old 04-14-2010
Multiple jobs reading from same file

I have a sequence of tasks that I routinely run and I'm trying to parallelize certain portions of the sequence. Specifically, there are 3 tasks which all read from the same file, each performing different operations and writing to their own seperate file. I was wondering if I could execute these three steps at the same time.

Side note: I didn't write the code for these steps, so I can't just wrap them all up into one or anything like that.
# 2  
Old 04-14-2010
You can run jobs in the background. If you have the tasks

- in separate scripts:

Code:
./script1 &
./script2 &

- as functions in a script:

Code:
your_function1 &
your_function2 &

# 3  
Old 04-14-2010
Thanks, Franklin52, that's precisely what I had in mind, but what I was actually getting at was the safety of this technique.

I know that multiple processes writing to the same file is a recipe for trouble, understandably, but I was curious whether multiple jobs/processes reading from the same file is also dangerous.

For example, what happens if they both try to read the same block of memory at the same time?

My googling turned up some forum topics about this in C code, where it did seem to be an issue, but I was going to submit jobs to the background like in Franklin52's post. Can I rely on the OS to sort out any conflicts between the processes?
# 4  
Old 04-15-2010
Quote:
Originally Posted by erichpowell
Thanks, Franklin52, that's precisely what I had in mind, but what I was actually getting at was the safety of this technique.

I know that multiple processes writing to the same file is a recipe for trouble, understandably, but I was curious whether multiple jobs/processes reading from the same file is also dangerous.

For example, what happens if they both try to read the same block of memory at the same time?

My googling turned up some forum topics about this in C code, where it did seem to be an issue, but I was going to submit jobs to the background like in Franklin52's post. Can I rely on the OS to sort out any conflicts between the processes?
It shouldn't be a problem IMHO to read a file with 3 processes simultaneously.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to read file and run multiple jobs

I have a txt file line1 line2 line3 $!/bin/sh cat /tmp/lus.txt | while read line do esxcli storage vmfs unmap -u $lin -n 4000 done this works but does in one line at a time. how do I do all lines at once simutaeously? Please use CODE tags as required by forum rules! (4 Replies)
Discussion started by: tdubb123
4 Replies

2. Shell Programming and Scripting

Reading multiple lines in a file

Hello, I am new in shell scripting. I need help regarding following. I have 4 files generated by backups daily. I have stored the names of these 4 files into one file. i.e I have 4 files names as a, b, c & d and these names have been put into one file abcd.txt. Now I want to cat each file in... (7 Replies)
Discussion started by: Ali Sarwar
7 Replies

3. Shell Programming and Scripting

Shell script to run multiple jobs and it's dependent jobs

I have multiple jobs and each job dependent on other job. Each Job generates a log and If job completed successfully log file end's with JOB ENDED SUCCESSFULLY message and if it failed then it will end with JOB ENDED with FAILURE. I need an help how to start. Attaching the JOB dependency... (3 Replies)
Discussion started by: santoshkumarkal
3 Replies

4. Shell Programming and Scripting

Reading multiple directories and file from them

Hello I'm making script for Dallas temperature sensors (DS1820). When a sensor is connected, it shows up as a directory in /sys/bus/w1/devices in format 10-xxxxxxx. Inside the directory is a file called w1_slave which holds the temperature in format t=xxxxx. Each sensor has unique... (2 Replies)
Discussion started by: Klipeti
2 Replies

5. Shell Programming and Scripting

Search & Replace in Multiple Files by reading a input file

I have a environment property file which contains: Input file: value1 = url1 value2 = url2 value3 = url3 and so on. I need to search all *.xml files under directory for value1 and replace it with url1. Same thing I have to do for all values mentioned in input file. I need script in unix bash... (7 Replies)
Discussion started by: Shamkamde
7 Replies

6. Shell Programming and Scripting

While loop reading file with multiple conditions

Hi Am trying to print the PIDs of process in a file and trying to grep any PID from that file I set the if condition as $value != "PID" and $value != "-" Assign that number to a variable Am confused since am using while loop to read the line from file and again if condition to check those... (2 Replies)
Discussion started by: Priya Amaresh
2 Replies

7. Shell Programming and Scripting

How to ignore single or multiple lines between /* and */ while reading from a file in unix?

I have a file proc.txt: if @debug = 1 then message 'Start Processing ', @procname, dateformat(now(*), 'hh:mm:ss'), @julian type info to client; end if; /* execute immediate with quotes 'insert into sys_suppdata (property, value, key_name) location ''' || @supp_server || '.' ||... (5 Replies)
Discussion started by: kidncute
5 Replies

8. Shell Programming and Scripting

Create Multiple files by reading a input file and changing the contents

Being new to this area .I have been assigned a task which i am unable to do . Can any one please help me . Hi I have requirement where i have input file XYZ_111_999_YYYYMMDD_1.TXT and with header and series of Numbers and Footer. I want to create a mutiple output files with each file having a... (2 Replies)
Discussion started by: bhargavkr
2 Replies

9. UNIX for Dummies Questions & Answers

Re : Set multiple cron jobs in one crontab file

Hello All, Hopw all is fine. I am newbie to Unix. I am using Bourne Shell (sh). One of the question I have is that I am trying to read XML file and based on reading that XML file I want to run different java programs at different hours. Meaning 05 14 * * * java ./program1 10 14 * * * java... (3 Replies)
Discussion started by: samshaw
3 Replies

10. UNIX for Dummies Questions & Answers

Ksh Storing Multiple Files and reading each line in each file.

How would I go about storing multiple file paths in a directory that begin like: 20080402* and run a loop that reads each line of each file thats in a given directory. So far this is what I have: #!/bin/ksh echo "ENTER Reprint Date (YYYYMMDD): " read ReprintDate echo ""... (1 Reply)
Discussion started by: developncode
1 Replies
Login or Register to Ask a Question