Sponsored Content
Top Forums Shell Programming and Scripting parallel excution for 2000 files. Post 302441703 by onesuri on Monday 2nd of August 2010 05:52:58 AM
Old 08-02-2010
parallel excution for 2000 files.

Code:
Hi,

I have a function abc(). i am calling the function 9 times. it is working fine and completed the script execution 10 hours.


input files:
CUSTOMER_INFO_1111_12345.csv
CUSTOMER_INFO_1222_12345.csv
CUSTOMER_INFO_1322_12345.csv
CUSTOMER_INFO_1333_12345.csv
CUSTOMER_INFO_1151_12345.csv
CUSTOMER_INFO_2221_12345.csv
CUSTOMER_INFO_3333_12345.csv
CUSTOMER_INFO_4444_12345.csv

like that i have 2000 files are present inthe source directory.

Code:
abc()
{
#Calculations are added in the function
for file in `ls |egrep '(CUSTOMER_INFO_.*_'$Flag'.*csv$)'`
do
tail -1 $file1| awk '/^"9"/'  > new_$file
done
}

Flag=1
abc $p1 $p2 $p3 $p4  $Flag &
Flag=2
abc $p1 $p2 $p3 $p4  $Flag &
Flag=3
abc $p1 $p2 $p3 $p4  $Flag &
Flag=4
abc $p1 $p2 $p3 $p4  $Flag &
Flag=5
abc $p1 $p2 $p3 $p4  $Flag &
Flag=6
abc $p1 $p2 $p3 $p4  $Flag &
Flag=7
abc $p1 $p2 $p3 $p4  $Flag &
Flag=8
abc $p1 $p2 $p3 $p4  $Flag &
Flag=9
abc $p1 $p2 $p3 $p4  $Flag &

wait

echo "script completed"


Code:
Currently execution based on flag and at a time 9 files are executed. i want to run the script for all the 2000 files at the same time. 
when i run the scipt all the 2000 files get start for the excution. all the files are executed in parallel.
i am unable to execute files at a time.

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

xterm closing automatically after command excution

Hi, I have a commands in a file called commands.file, I am using this file with the following commnad xterm -e "commands.file" After executing this file in the new xterm, it is closing automatically. I want to use that xterm after that. Please help me regarding this? :) ... (4 Replies)
Discussion started by: chaitubek
4 Replies

2. UNIX for Advanced & Expert Users

To find the time of excution of a particular unix command

Hi experts, It would be grateful if you can guide me on the following, 1.How to find the time of excution of a particular unix command. 2. How to find the terminal (IP Address) from where that particular unix command was excuted. 3. How to find the user who executed that particular unix... (1 Reply)
Discussion started by: apsprabhu
1 Replies

3. UNIX for Dummies Questions & Answers

Copy files in Parallel

Is there any out of box Command in Unix or is it possible through shell scripting to copy all the files in a directory in Parallel. Example. I am doing a COLD database backup. I have the data/dbx directory which has around 1000 data files. I was thinking is there a way to spawn the copy... (10 Replies)
Discussion started by: simonsimon
10 Replies

4. Shell Programming and Scripting

Unable to transfer files from Windows 2000 Server to Sun Solaris..

Dear Friends, I need to transfer few files from a Windows 2000 server to Sun Solaris system, connected in the same network. This copy should be done as a batch job without asking for password to be entered every time. How to make this possible ??? At present I am using cygwin in my laptop... (4 Replies)
Discussion started by: ks_reddy
4 Replies

5. Linux

clarification for cmd excution

Hi Code: nohup sar -u 10 $COUNT | awk 'NR>2 {$2=$4=$7="";}1' | tr -s ' ' '\t' 2>/dev/null >sar.out & i have execute above line from shell script .. but sar.out is created with ZERO bite size.. Help on that what is the problem here (1 Reply)
Discussion started by: Mani_apr08
1 Replies

6. Programming

Expext Script excution problem..

Hey Guys , iam trying to match to string in TCL scripts but it not working . Its only working true condition to print yes.. When condtion fail then even it prints YES Any suggestion this ?? if {} { puts"yes" ... (3 Replies)
Discussion started by: hackerdilli
3 Replies

7. Shell Programming and Scripting

Need to read two files in parallel

Following is the requirement In FileA I have the content as follows. 1,2,3 111,222,333 1000,2000,3000 In FileB I have the content as follows. 4,5,6 444,555,666 4000,5000,6000 I need to read FileA and FileB parallely and create the FileC as follows. 1,2,3,4,5,6... (1 Reply)
Discussion started by: kmanivan82
1 Replies

8. Shell Programming and Scripting

Customized command excution

Hi, The below commands will be inside a KSH script and SOLARIS machine. Command-1 //Executes in 10s,5s,..5mins,etc TIMER = Look for Command-1 execution status - IF finished in 25secs move to next command in the script IF NOT kill above command and move to next command. Command-2 Any... (9 Replies)
Discussion started by: penqueen
9 Replies

9. Shell Programming and Scripting

Comparing list of files in parallel

Hi everyone. I have a list of files like: file001 file002 file003 . . . . file385 file386 file387 There are more files than above, but I hope you understand what I'm trying to do here. Is there a way I can create a loop to compare: file001 with file385 file002 with file386 (9 Replies)
Discussion started by: craigsky
9 Replies

10. UNIX for Beginners Questions & Answers

How to paste multiple files in parallel?

Hi all, I am trying to paste thousands of files together into a matrix. Each file has only 1 column and all the files have the same number of rows (~27k rows). I tried paste * > output as well as some other for loops but the output only contains the columns from the 1st and last files. The... (8 Replies)
Discussion started by: notimenocall
8 Replies
wxSizerItem(3erl)					     Erlang Module Definition						 wxSizerItem(3erl)

NAME
wxSizerItem - See external documentation: wxSizerItem. DESCRIPTION
See external documentation: wxSizerItem . DATA TYPES
wxSizerItem() : An object reference, The representation is internal and can be changed without notice. It can't be used for comparsion stored on disc or distributed for use on other nodes. EXPORTS
new() -> wxSizerItem() See external documentation . new(Window::wxWindow() (see module wxWindow) | wxSizer() (see module wxSizer), Flags::wxSizerFlags() (see module wxSizerFlags)) -> wxSizerItem() See external documentation . new(Width::integer(), Height::integer(), Flags::wxSizerFlags() (see module wxSizerFlags)) -> wxSizerItem() See external documentation . new(Window::wxWindow() (see module wxWindow) | wxSizer() (see module wxSizer), Proportion::integer(), Flag::integer(), Border::integer(), UserData::wx() (see module wx)) -> wxSizerItem() See external documentation . new(Width::integer(), Height::integer(), Proportion::integer(), Flag::integer(), Border::integer(), UserData::wx() (see module wx)) -> wxSizerItem() See external documentation . calcMin(This::wxSizerItem()) -> {W::integer(), H::integer()} See external documentation . deleteWindows(This::wxSizerItem()) -> ok See external documentation . detachSizer(This::wxSizerItem()) -> ok See external documentation . getBorder(This::wxSizerItem()) -> integer() See external documentation . getFlag(This::wxSizerItem()) -> integer() See external documentation . getMinSize(This::wxSizerItem()) -> {W::integer(), H::integer()} See external documentation . getPosition(This::wxSizerItem()) -> {X::integer(), Y::integer()} See external documentation . getProportion(This::wxSizerItem()) -> integer() See external documentation . getRatio(This::wxSizerItem()) -> float() See external documentation . getRect(This::wxSizerItem()) -> {X::integer(), Y::integer(), W::integer(), H::integer()} See external documentation . getSize(This::wxSizerItem()) -> {W::integer(), H::integer()} See external documentation . getSizer(This::wxSizerItem()) -> wxSizer() (see module wxSizer) See external documentation . getSpacer(This::wxSizerItem()) -> {W::integer(), H::integer()} See external documentation . getUserData(This::wxSizerItem()) -> wx() (see module wx) See external documentation . getWindow(This::wxSizerItem()) -> wxWindow() (see module wxWindow) See external documentation . isSizer(This::wxSizerItem()) -> bool() See external documentation . isShown(This::wxSizerItem()) -> bool() See external documentation . isSpacer(This::wxSizerItem()) -> bool() See external documentation . isWindow(This::wxSizerItem()) -> bool() See external documentation . setBorder(This::wxSizerItem(), Border::integer()) -> ok See external documentation . setDimension(This::wxSizerItem(), Pos::{X::integer(), Y::integer()}, Size::{W::integer(), H::integer()}) -> ok See external documentation . setFlag(This::wxSizerItem(), Flag::integer()) -> ok See external documentation . setInitSize(This::wxSizerItem(), X::integer(), Y::integer()) -> ok See external documentation . setMinSize(This::wxSizerItem(), Size::{W::integer(), H::integer()}) -> ok See external documentation . setMinSize(This::wxSizerItem(), X::integer(), Y::integer()) -> ok See external documentation . setProportion(This::wxSizerItem(), Proportion::integer()) -> ok See external documentation . setRatio(This::wxSizerItem(), X::float() | term()) -> ok See external documentation . Alternatives: setRatio(This::wxSizerItem(), Ratio::float()) -> ok setRatio(This::wxSizerItem(), Size::{W::integer(),H::integer()}) -> ok setRatio(This::wxSizerItem(), Width::integer(), Height::integer()) -> ok See external documentation . setSizer(This::wxSizerItem(), Sizer::wxSizer() (see module wxSizer)) -> ok See external documentation . setSpacer(This::wxSizerItem(), Size::{W::integer(), H::integer()}) -> ok See external documentation . setSpacer(This::wxSizerItem(), Width::integer(), Height::integer()) -> ok See external documentation . setWindow(This::wxSizerItem(), Window::wxWindow() (see module wxWindow)) -> ok See external documentation . show(This::wxSizerItem(), Show::bool()) -> ok See external documentation . destroy(This::wxSizerItem()) -> ok Destroys this object, do not use object again AUTHORS
<> wxErlang 0.98.9 wxSizerItem(3erl)
All times are GMT -4. The time now is 03:17 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy