Sponsored Content
Top Forums Shell Programming and Scripting Using Make to batch process files Post 302473273 by jujumbura on Friday 19th of November 2010 10:30:18 AM
Old 11-19-2010
Question Using Make to batch process files

Hello all,

I have a make question, and I was hoping somebody here might be able to point me in the right direction.

Here is my issue; I have a command-line tool that I use to run a conversion on an input XML file that results in an output binary file. However, this particular tool needs to load up an overarching project file when it starts, and the time involved in that initial parsing is significant. Therefore, if I have 100 XML files that need processing, I waste a lot of time starting the tool over and over.

I like to use make for this kind of processing, because the dependency handling is so good. However, in this instance I would like to send all of the XML files that need processing to the tool in a single batch. Typically my make rules look like this:

Code:
%.bin : %.xml
           tool -export %.xml

... but this of course will run "tool" once for each XML file encountered in a previous dependency. I had considered this:

Code:
$(ALL_BIN) : $(ALL_XML)
                 tool -export $(ALL_XML)

... which will batch up the job, but it will rebuild every binary file if just a single XML file changes. Does anybody know a way that I keep make's dependency checking on a per-file basis, but run the actual command only once? Thanks much for any advice!

Moderator's Comments:
Mod Comment
Please use code tags when posting data and code samples!

Last edited by vgersh99; 11-19-2010 at 11:51 AM.. Reason: code tags, please!
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

trying to read batch process but need some command help

I am trying to access batch process that take place each nite. I am using Solaris 5.8 (and i am used to redhat). however I am trying to access say a certain directory. The home/oradev , is the directory...in there i am trying to access say a batch file within this, how can see if they are in... (1 Reply)
Discussion started by: etravels
1 Replies

2. Shell Programming and Scripting

User Prompt during batch script process

The script below performs an incremental database backup. When the increment backup is out of sequence, the process prompts the user: "Incremental backup out of sequece. Do you wish to continue? Y or N". This script is set to run as a scheduled process in background mode. When the script... (2 Replies)
Discussion started by: bond007jlv
2 Replies

3. Shell Programming and Scripting

how to start a process and make it sleep for 5 mins and then kill that process

how to start a process and make it sleep for 5 mins and then kill that process (6 Replies)
Discussion started by: shrao
6 Replies

4. Shell Programming and Scripting

How to make the parent process to wait for the child process

Hi All, I have two ksh script. 1st script calls the 2nd script and the second script calls an 'C' program. I want 1st script to wait until the 'C' program completes. I cant able to get the process id for the 'C' program (child process) to make the 1st script to wait for the second... (7 Replies)
Discussion started by: sennidurai
7 Replies

5. Shell Programming and Scripting

Processing different jobs as a batch process

Hi All, I want to process consecutive jobs in a sequence but when I execute 1 job ,the control does not return to the command prompt to continue with the next job. Can anyone help me here? Thanks (3 Replies)
Discussion started by: Taranjeet Singh
3 Replies

6. Shell Programming and Scripting

Executing a batch of files within a shell script with option to refire the individual files in batch

Hello everyone. I am new to shell scripting and i am required to create a shell script, the purpose of which i will explain below. I am on a solaris server btw. Before delving into the requirements, i will give youse an overview of what is currently in place and its purpose. ... (2 Replies)
Discussion started by: goddevil
2 Replies

7. Shell Programming and Scripting

Cannot make pipe for process substitution: Too many open files

Hi, I've came across an issue with a script I've been writing to check DHCP addresses on an Solaris system, the script has been running reasonably well, until it hit the following problem: ./sub_mon_v2: redirection error: cannot duplicate fd: Too many open files ./sub_mon_v2: cannot make... (3 Replies)
Discussion started by: CiCa
3 Replies

8. Shell Programming and Scripting

Make background process interact with fg process

Hi, I have written a menu driven shell script in which as per the choice, I run the another script on background. For eg: 1. get info 2)process info 3)modify info All the operations have different scripts which i schedule in background using &. However I wish to display the error... (0 Replies)
Discussion started by: ashima jain
0 Replies

9. Shell Programming and Scripting

Start process on X number of files and then wait for the next batch

Thanks for RudiC for his extraordinary help on organizing files in a batch of 10 using below code. FL=($(ls)); for ((i=0;i<=${#FL};i++)); do for j in ${FL:$i:10}; do $batch ${j} ${j}.txt done; echo "Pausing for next iteration"; echo... (6 Replies)
Discussion started by: busyboy
6 Replies
GNOME-XML(1)							       1.1.0							      GNOME-XML(1)

NAME
xml-config - script to get information about the installed version of GNOME-XML SYNOPSIS
xml-config [--prefix[=DIR]] [--libs] [--cflags] [--version] [--help] DESCRIPTION
xml-config is a tool that is used to determine the compile and linker flags that should be used to compile and link programs that use GNOME-XML. OPTIONS
xml-config accepts the following options: --version Print the currently installed version of GNOME-XML on the standard output. Add the --static option to print the linker flags that are necessary to statically link a GNOME-XML program. --libs Print the linker flags that are necessary to link a GNOME-XML program. --cflags Print the compiler flags that are necessary to compile a GNOME-XML program. --prefix=PREFIX If specified, use PREFIX instead of the installation prefix that GNOME-XML was built with when computing the output for the --cflags and --libs options. This option must be specified before any --libs or --cflags options. AUTHOR
This manual page was written by Fredrik Hallenberg <hallon@lysator.liu.se>, for the Debian GNU/linux system (but may be used by others). Version 3 July 1999 GNOME-XML(1)
All times are GMT -4. The time now is 05:47 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy