Parllel Processing Post: 302128671

Sponsored Content

Top Forums Shell Programming and Scripting Parllel Processing Post 302128671 by nagrcm on Thursday 26th of July 2007 02:10:52 AM

07-26-2007

Registered User

Parllel Processing

In simple words of the scenario.

I have some set of files that are to loaded into a table in parallel

Loading part would be taken care of the Abintio graph. I must develop a script which instructs this graph about how to process the files like wat files to process and how to process by passing on some parameters.

This is the task...

let me explain u much better with an example.

Let have that i have the following files EMP.dat, DEPT.dat, BONUS.dat

I have splited all these files into two. EMP001.dat, EMP002.dat, DEPT001.dat, DEPT002.dat, BONUS001.dat, BONUS002.dat

Now take TO Parameters $Load_batch and $set_time

IF $load_batch ='N' the script should provide the information of the files to be processed (which are EMP.dat, DEPT.dat, BONUS.dat)

IF $load_batch = "y" The script should provide the information of batches (first batch files EMP001.dat, DEPT001.dat, BONUS001.dat; Second Batch EMP002.dat, DEPT002.dat, BONUS002.dat)

$set_time is a parameter defines the time between these batches...

Any ideas on this.

?

nagrcm

View Public Profile for nagrcm

Find all posts by nagrcm

7 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to make parallel processing rather than serial processing ??

Hello everybody, I have a little problem with one of my program. I made a plugin for collectd (a stats collector for my servers) but I have a problem to make it run in parallel. My program gathers stats from logs, so it needs to run in background waiting for any new lines added in the log...

2. Shell Programming and Scripting

File processing

bash-2.05$ vi file_read.sh "file_read.sh" 9 lines, 97 characters #!/bin/ksh print "Enter the group name" read gname varA= grep "$gname::" group print $varA This is the scenario: I am trying to get a userid and groupname from user and trying to add the userid to the given group....

3. Shell Programming and Scripting

help with file processing

Hi , I have a memory file like this with two columns: @C010 AA @C011 AA @C012 FE @C013 FF @C014 F7 @C015 FF first is memory add, second is the data. I wan to convert into a serial sequence starting from '00000' all the way to 'FFFFF' with those fields from the above file...

4. Shell Programming and Scripting

Help in file processing

Hi experts, please help for my script..I'm confused how to do this. i have 3 .csv file(excel file) The 5 .csv files are A.csv,B.csv,C.csv A.csv has column1 column2 a 1 b 4 c 2 d ...

5. Shell Programming and Scripting

processing with awk

I have many lines like the following in a file(there are also other kinds of lines) Host: 72.52.104.74 (tserv1.fmt2.he.net) Ports: 22/open/tcp//tcpwrapped///, 53/open/tcp//domain//PowerDNS 3.3/, 179/open/tcp//tcpwrapped/// Ignored State: closed (997) Seq Index: 207 IP ID Seq: All...

6. Programming

Data processing

Hello guys! I have some issue in how to processing some data. I have some files with 3 columns. The 1st column is a name of my sample. The 2nd column is a numerical sequence (very big sequence) starting from "1". And the 3rd column is a feature of each line, represented for a number (completely...

7. Programming

awk processing / Shell Script Processing to remove columns text file

Hello, I extracted a list of files in a directory with the command ls . However this is not my computer, so the ls functionality has been revamped so that it gives the filesizes in front like this : This is the output of ls command : I stored the output in a file filelist 1.1M...

LEARN ABOUT LINUX

dh_installdeb

DH_INSTALLDEB(1)						     Debhelper							  DH_INSTALLDEB(1)

NAME

       dh_installdeb - install files into the DEBIAN directory

SYNOPSIS

       dh_installdeb [debhelperoptions]

DESCRIPTION

       dh_installdeb is a debhelper program that is responsible for installing files into the DEBIAN directories in package build directories with
       the correct permissions.

FILES

       package.postinst
       package.preinst
       package.postrm
       package.prerm
	   These maintainer scripts are installed into the DEBIAN directory.

	   Inside the scripts, the token #DEBHELPER# is replaced with shell script snippets generated by other debhelper commands.

       package.triggers
       package.shlibs
	   These control files are installed into the DEBIAN directory.

       package.conffiles
	   This control file will be installed into the DEBIAN directory.

	   In v3 compatibility mode and higher, all files in the etc/ directory in a package will automatically be flagged as conffiles by this
	   program, so there is no need to list them manually here.

       package.maintscript
	   Lines in this file correspond to dpkg-maintscript-helper(1) commands and parameters.  Any shell metacharacters will be escaped, so
	   arbitrary shell code cannot be inserted here.  For example, a line such as "mv_conffile /etc/oldconffile /etc/newconffile" will insert
	   maintainer script snippets into all maintainer scripts sufficient to move that conffile.

	   A versioned Pre-Dependency on dpkg is needed to use dpkg-maintscript-helper(1). An appropriate Pre-Dependency is set in
	   ${misc:Pre-Depends} ; you should make sure to put that token into an appropriate place in your debian/control file.

SEE ALSO

       debhelper(7)

       This program is a part of debhelper.

AUTHOR

       Joey Hess <joeyh@debian.org>

8.9.0ubuntu2.1							    2012-06-12							  DH_INSTALLDEB(1)