Consolidating multiple outputs in one file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Consolidating multiple outputs in one file
# 1  
Old 02-21-2018
Consolidating multiple outputs in one file

Dears,
i am stuck here
i have 3 scripts running at one time and all the three scripts finish at different time and each script generate 1 file with different name. so i will have three files.

i dnt know which script finish first
i want to have a script which check if all the there files are created and process is over then join the files and make 1 file.

please help me to archive this
# 2  
Old 02-21-2018
Please provide information on your attempts to resolve.

Until so, we will refrain from sharing any guidance.

The purpose of this Board is the assist users in solving their problems. We are not a coding service. Further, we are not a homework service - and sometimes posts appear to be an attempt to have someone solve a school assignment.

Finally, we urge all members of the Forum to NOT post a solution to this question until effort to resolve is demonstrated.
This User Gave Thanks to joeyg For This Post:
# 3  
Old 02-22-2018
Quote:
Originally Posted by joeyg
Please provide information on your attempts to resolve.

Until so, we will refrain from sharing any guidance.

The purpose of this Board is the assist users in solving their problems. We are not a coding service. Further, we are not a homework service - and sometimes posts appear to be an attempt to have someone solve a school assignment.

Finally, we urge all members of the Forum to NOT post a solution to this question until effort to resolve is demonstrated.
I was not looking for someone to do my homework, i was seeking for some suggestions,
i can try to do it on my own.. but was seeking for some pattern.
you misunderstood my post
# 4  
Old 02-22-2018
Conceptual...

OK, then you have 3 programs running that create output.
Would seem that you would want to create one watcher program, perhaps run at regular interval via a cron job.
The watcher program would look to see if the three output files have been created. It would then combine the three files into one.

Some things to consider - is this daily?
Because then you may want to also check to see if you have already created the combined file; thus perhaps checking for the "NOT" existence of the daily combined file.

Another consideration - the order or organization of the combined file. Should the files be combined in the order completed or another order.
This User Gave Thanks to joeyg For This Post:
# 5  
Old 02-22-2018
Quote:
Originally Posted by joeyg
OK, then you have 3 programs running that create output.
Would seem that you would want to create one watcher program, perhaps run at regular interval via a cron job.
The watcher program would look to see if the three output files have been created. It would then combine the three files into one.
I would rather suggest to do it via a semaphore and i suggest the thread starter to read up on the concept.

Basically the first script to be started will open the semaphore and do the starting setup and the last one to stop will do the end processing and close the semaphore.

I hope this helps.

bakunin
# 6  
Old 02-22-2018
Another important information is how the scripts are started.
Would it be possible to have a wrapper script to start those 3 scripts in background and use the wait command to wait for all of them to finish, then combine the output?
This User Gave Thanks to cero For This Post:
# 7  
Old 02-23-2018
Quote:
Originally Posted by joeyg
OK, then you have 3 programs running that create output.
Would seem that you would want to create one watcher program, perhaps run at regular interval via a cron job.
The watcher program would look to see if the three output files have been created. It would then combine the three files into one.

Some things to consider - is this daily?
Because then you may want to also check to see if you have already created the combined file; thus perhaps checking for the "NOT" existence of the daily combined file.

Another consideration - the order or organization of the combined file. Should the files be combined in the order completed or another order.
i have worked it out by making a script and scheduling in cron
it keep on running and wait for the three file, once three files exist, it concatenate all the three in 1 and then send email

please close this thread
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Returning multiple outputs of a single line based on previous repeated lines

Hello, I am trying to return a time multiple times from a file that has varying output just before the time instance, i.e. cat jumped cat jumped cat jumped time = 1.1 cat jumped cat jumped time = 1.2 cat jumped cat jumped time = 1.3 In this case i would like to output a time.txt... (6 Replies)
Discussion started by: ryddner
6 Replies

2. UNIX for Dummies Questions & Answers

Writing multiple outputs in to separate cells of CSV file

Hi I am writing a script which has multiple awk statements and each statement gives me a numeric count as an output. I want those output to be stored in different cells of a csv file. say 12 awk statements give 12 output and i want them in diffrenet cells of csv file. Thank you guys..!! (4 Replies)
Discussion started by: prabhat.diwaker
4 Replies

3. Shell Programming and Scripting

Array in awk outputs multiple values

Disclaimer: OP is 100% Awk beginner. I use this code on ASCII files I need to report against. awk 'BEGIN { tokens = 0 tokens = 0 tokens = 0 } { for (token in tokens) { if ($1 == token){print $0; tokens++;}}} END {for (token in tokens){ if( tokens ==... (1 Reply)
Discussion started by: alan
1 Replies

4. Shell Programming and Scripting

simple join for multiple files and produce 3 outputs

sh script file1 filea fileb filec ................filez. >>output1 & output2 &output3 file1 z10 1873 1920 z_number1_E59 z10 2042 2090 z_number2_E59 Z22 2476 2560 z_number3_E59 Z22 2838 2915 z_number4_E59 z1 1873 1920 z_number1_E60 z1 ... (9 Replies)
Discussion started by: stateperl
9 Replies

5. UNIX for Dummies Questions & Answers

multiple variables assignement (stdout/stderr outputs)

Hi all, I've been looking around for this for a while and can't seem to find a satifactory way to do what I want: I would like to assign the output of stdout to a variable and that of stderr to another one, and this without using temporary files/named pipes. In other words be able to assign... (4 Replies)
Discussion started by: anthalamus
4 Replies

6. Shell Programming and Scripting

create outputs from other command outputs

hi friends, The code: i=1 while do filename=`/usr/bin/ls -l| awk '{ print $9}'` echo $filename>>summary.csv #Gives the name of the file stored at column 9 count=`wc -l $filename | awk '{print $1}'` echo $count>>summary.csv #Gives just the count of lines of file "filename" i=`expr... (1 Reply)
Discussion started by: rajsharma
1 Replies

7. UNIX for Dummies Questions & Answers

Multiple Substring Outputs

Hello, I am reading a file with millions of lines in it. Each line is big line containing several xml tags. I need to Output just the value of two tags in a seperate flat file. For eg- I need to output whats present in <ComponentName> something </ComponentName> and another tag is... (2 Replies)
Discussion started by: sunnybehl
2 Replies

8. Shell Programming and Scripting

How to store multiple outputs from an awk command?

x=`echo $line | awk -F "|" '{print $1;print NR}'` How will I get the 2 return values ($1 and NR) from awk to variables? (4 Replies)
Discussion started by: tene
4 Replies

9. Shell Programming and Scripting

problems with a script that outputs data to a file

First of all, im a total newbie to the point that i do not know what are the terms to search for my problem. I did however spend the rest of the day today trying to figure out what is wrong with my bash script. ive always thought that the best way to learn is to tackle a problem heads on. but at... (1 Reply)
Discussion started by: joeribut
1 Replies

10. UNIX for Advanced & Expert Users

Consolidating Pipes

This is something I've given a lot of thought to and come up with no answer. Say you have a data stream passing from a file, through process A, into process B. Process A only modifies a few bytes of the stream, then prints the rest of the stream unmodified. Is there any way to stream the file... (4 Replies)
Discussion started by: Corona688
4 Replies
Login or Register to Ask a Question