Sponsored Content
Top Forums Shell Programming and Scripting Parallel Runs in UNIX/Linix Shell Scripting Post 302975465 by Don Cragun on Monday 13th of June 2016 03:55:09 PM
Old 06-13-2016
I do not understand what you're trying to do.

Are you saying that you want 100 copies of split running at the same to split a single file into 100 chunks? That is not the way split works! And, the split utility takes at most two file operands; not an unlimited number as indicated by the filec... in your example split command (and the comma in filea, will be used as the base part of the names of your output files (the first four of which will be named filea,aa, filea,ab, filea,ac, and filea,ad) assuming you remove the remaining file operands which would cause split
to fail with a syntax error.

You say your input file has about 1 million records. Are the records fixed length or variable length? Are you absolutely positive that files of 2,097,152 bytes will contain complete records (i.e., that the start of a record won't be in one of your split files and the end of that record won't be at the start of the next split file)?

After you have created a bunch of files (all but the last of which contain exactly 2,097,152 bytes), what do you want to do with these files? You say you want to run them. Are each of these 2,097,152 byte files a shell script???

Does your system have 100 CPUs? If not, trying to run 100 jobs in parallel will slow down processing; not provide faster processing!

Are you going to distribute the output files from split onto different disk drives? Do you have 100 different disk drives? Will running these 100 files in parallel overwhelm your I/O subsystem?

What operating system are you using?

What shell are you using?
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

UNIX problem? Unix programm runs windows 2000 CPU over 100%

Okee problems...!! What is happening: Unix server with some programms, workstations are windows 2000, the workstations work good but when you start a programm on the Unix server the CPU of the workstations go to 100% usage resulting that the system gets very slow. The programm well its running so... (2 Replies)
Discussion started by: zerocool
2 Replies

2. Shell Programming and Scripting

difference between AIX shell scripting and Unix shell scripting.

please give the difference between AIX shell scripting and Unix shell scripting. (2 Replies)
Discussion started by: haroonec
2 Replies

3. Shell Programming and Scripting

unix shell scripting

i have a file blah_blah_blah_yyyymmdd.dat . I need to write a shell script to extract this date and compare it with the current date. How do i do this?? Can someone provide me with the code please? (1 Reply)
Discussion started by: swarna bhat
1 Replies

4. Shell Programming and Scripting

Unix Shell Scripting

I 'm new to unix shell scripting can some one guide me to any e-book or link from where i can learn unix shell scripting .. i want to learn create interactive scripts for my day to day solaris work. Any help would be appreciated (1 Reply)
Discussion started by: fugitive
1 Replies

5. Shell Programming and Scripting

$# in unix shell scripting what does it mean

Hi can some one let me know what is the meaning of $# $@ $_ @_ in unix shell scripting. I really appreciate your time for replying to my post. Thanks In Advance (4 Replies)
Discussion started by: einsteinBrain
4 Replies

6. Shell Programming and Scripting

Unix shell scripting

Hi All, I have one file called date1.txt and it contains dates like 130112 140112 150112 160112 170112 180112 190112 201012 so i need a script to read this file line by line and find out the day of each date and assign this value in one variable. And validate Weekday="Mon" then... (4 Replies)
Discussion started by: vichuelaa
4 Replies

7. UNIX for Dummies Questions & Answers

Unix Shell Scripting( Calling from Unix to PLSQL)

Hello Experts, I have the following questions to be discussed here at this esteemed discussion forum. I have two Excel sheets which contain Unix Commands llike creating directory the structure/ftp/Copy/Zip etc to basically create an environment. I need help in understanding some of... (1 Reply)
Discussion started by: faizsaadq
1 Replies

8. UNIX for Dummies Questions & Answers

Unix Shell Scripting

I'm sorry if this doesn't go here, but I'm in depserate need of help with my last unix homework. Anyways, I'm taking summer classes, and one of them is UNIX. I've understood everything thus far, but I'm having a killer time with how my instructor has worded the problems for shell scripting. I... (3 Replies)
Discussion started by: dw15
3 Replies

9. Shell Programming and Scripting

Parallel processing in AIX (bash shell scripting)

Hi all, I am struggling to build a utility which can do the parallel processing. I achieved same in Linux using xargs -P but same is not working on AIX. I am building file copy utility where I will have all required info in a file (like souce file info and target location details), now i need... (2 Replies)
Discussion started by: ankur singh
2 Replies
CPANPLUS::Shell(3pm)					 Perl Programmers Reference Guide				      CPANPLUS::Shell(3pm)

NAME
CPANPLUS::Shell - base class for CPANPLUS shells SYNOPSIS
use CPANPLUS::Shell; # load the shell indicated by your # config -- defaults to # CPANPLUS::Shell::Default use CPANPLUS::Shell qw[Classic] # load CPANPLUS::Shell::Classic; my $ui = CPANPLUS::Shell->new(); my $name = $ui->which; # Find out what shell you loaded $ui->shell; # run the ui shell DESCRIPTION
This module is the generic loading (and base class) for all "CPANPLUS" shells. Through this module you can load any installed "CPANPLUS" shell. Just about all the functionality is provided by the shell that you have loaded, and not by this class (which merely functions as a generic loading class), so please consult the documentation of your shell of choice. BUG REPORTS
Please report bugs or other issues to <bug-cpanplus@rt.cpan.org<gt>. AUTHOR
This module by Jos Boumans <kane@cpan.org>. COPYRIGHT
The CPAN++ interface (of which this module is a part of) is copyright (c) 2001 - 2007, Jos Boumans <kane@cpan.org>. All rights reserved. This library is free software; you may redistribute and/or modify it under the same terms as Perl itself. SEE ALSO
CPANPLUS::Shell::Default, CPANPLUS::Shell::Classic, cpanp perl v5.18.2 2014-01-06 CPANPLUS::Shell(3pm)
All times are GMT -4. The time now is 07:56 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy