09-04-2009
parallel processing
hi i am preparing a set of batches for a set of files sequentially
There is a folder /xyz where all the files reside
now all the files starting with
01 - will be appended for one below other to form a batch batch01
then all the files starting with
02 - will be appended for one below other to form a batch batch02
then
03 - will be appended for one below other to form a batch batch03
then
04 - will be appended for one below other to form a batch batch04
..
and so on, now this is taking a lot of time while processing.
How can i improve the performance say include some type of parallel processing
to minimize the time.
Please Advice
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi All,
I am working on solaris 8 sparc machine with 2 cpu.
I am trying to run my application which generates files. I run multiple instance of the application, but the results don't seem to show as if it were runing parallely.
When i run the application once it takes 12 secs to generate a... (1 Reply)
Discussion started by: zing
1 Replies
2. Shell Programming and Scripting
Hi
I want to run two shell script files parallely. These two scripts are interacting with the database. can any body help on this Pls
Regards
Audippa naidu.M (3 Replies)
Discussion started by: audippa
3 Replies
3. Shell Programming and Scripting
Hi
I am looking for some kind of feature in unix that will help me write a script that can invoke multiple processes in parallel. And make sure that the multiple parallel processes complete successfully before I proceed to the next step.
Someone suggested something called timespid or... (6 Replies)
Discussion started by: imnewtothis23
6 Replies
4. Shell Programming and Scripting
Hello everybody,
I have a little problem with one of my program. I made a plugin for collectd (a stats collector for my servers) but I have a problem to make it run in parallel.
My program gathers stats from logs, so it needs to run in background waiting for any new lines added in the log... (0 Replies)
Discussion started by: Samb95
0 Replies
5. Shell Programming and Scripting
How to write script which run multiple scripts parllely,
i have script called A,which has to execute B,C,D,E scripts parllely.. (2 Replies)
Discussion started by: machpee
2 Replies
6. Shell Programming and Scripting
HI All,
I have scenerio where I need to call sub modules through for loop
for (i=0; i<30 ;i++)
{
..
..
..
subroutine 1;
subroutine 2;
}
I want this to be run in parallel
process1
{
...
...
subroutine 1;
subroutine 2; (0 Replies)
Discussion started by: gvk25
0 Replies
7. Shell Programming and Scripting
Hi
Say I am interested in processing a big data set over shell, and each process individually takes a long time, but many such processes can be pipe-lined, is there a way to do this automatically or efficiently in shell?
For example consider pinging a list addresses upto 5 times each. Of... (5 Replies)
Discussion started by: jamie_123
5 Replies
8. Shell Programming and Scripting
Unix OS : Linux 2.6x
Shell type : Korn
Hi all ,
This is a requirement to incorporate parallel processing of a Unix code .
I have two pieces of unix code , one of which will act as a parent process .
This script will invoke multiple ( say four ) instances of the second script at one go... (13 Replies)
Discussion started by: kumarjt
13 Replies
9. Shell Programming and Scripting
Hi,
I am taking up the cue from where I was left in my earlier post ( link given below )
https://www.unix.com/shell-programming-scripting/231107-implement-parallel-processing.html
I actually wanted to know the significance of using the Unix "wait" , which returns
the control from background to... (3 Replies)
Discussion started by: kumarjt
3 Replies
10. Shell Programming and Scripting
I have 10,000 + files, each of which I need to zip using bzip2.
Is ti possible to use bash to create 8 parallel streams sending a new file to be processed from the list when one of the others has finished? (1 Reply)
Discussion started by: garethsays
1 Replies
LEARN ABOUT SUSE
git-cat-file
GIT-CAT-FILE(1) Git Manual GIT-CAT-FILE(1)
NAME
git-cat-file - Provide content or type and size information for repository objects
SYNOPSIS
git cat-file (-t | -s | -e | -p | <type>) <object>
git cat-file (--batch | --batch-check) < <list-of-objects>
DESCRIPTION
In its first form, the command provides the content or the type of an object in the repository. The type is required unless -t or -p is
used to find the object type, or -s is used to find the object size.
In the second form, a list of objects (separated by linefeeds) is provided on stdin, and the SHA1, type, and size of each object is printed
on stdout.
OPTIONS
<object>
The name of the object to show. For a more complete list of ways to spell object names, see the "SPECIFYING REVISIONS" section in git-
rev-parse(1).
-t
Instead of the content, show the object type identified by <object>.
-s
Instead of the content, show the object size identified by <object>.
-e
Suppress all output; instead exit with zero status if <object> exists and is a valid object.
-p
Pretty-print the contents of <object> based on its type.
<type>
Typically this matches the real type of <object> but asking for a type that can trivially be dereferenced from the given <object> is
also permitted. An example is to ask for a "tree" with <object> being a commit object that contains it, or to ask for a "blob" with
<object> being a tag object that points at it.
--batch
Print the SHA1, type, size, and contents of each object provided on stdin. May not be combined with any other options or arguments.
--batch-check
Print the SHA1, type, and size of each object provided on stdin. May not be combined with any other options or arguments.
OUTPUT
If -t is specified, one of the <type>.
If -s is specified, the size of the <object> in bytes.
If -e is specified, no output.
If -p is specified, the contents of <object> are pretty-printed.
If <type> is specified, the raw (though uncompressed) contents of the <object> will be returned.
If --batch is specified, output of the following form is printed for each object specified on stdin:
.ft C
<sha1> SP <type> SP <size> LF
<contents> LF
.ft
If --batch-check is specified, output of the following form is printed for each object specified on stdin:
.ft C
<sha1> SP <type> SP <size> LF
.ft
For both --batch and --batch-check, output of the following form is printed for each object specified on stdin that does not exist in the
repository:
.ft C
<object> SP missing LF
.ft
AUTHOR
Written by Linus Torvalds <torvalds@osdl.org[1]>
DOCUMENTATION
Documentation by David Greaves, Junio C Hamano and the git-list <git@vger.kernel.org[2]>.
GIT
Part of the git(1) suite
NOTES
1. torvalds@osdl.org
mailto:torvalds@osdl.org
2. git@vger.kernel.org
mailto:git@vger.kernel.org
Git 1.7.1 07/05/2010 GIT-CAT-FILE(1)