Parllel Processing Post: 302128671

Sponsored Content

Top Forums Shell Programming and Scripting Parllel Processing Post 302128671 by nagrcm on Thursday 26th of July 2007 02:10:52 AM

07-26-2007

Registered User

Parllel Processing

In simple words of the scenario.

I have some set of files that are to loaded into a table in parallel

Loading part would be taken care of the Abintio graph. I must develop a script which instructs this graph about how to process the files like wat files to process and how to process by passing on some parameters.

This is the task...

let me explain u much better with an example.

Let have that i have the following files EMP.dat, DEPT.dat, BONUS.dat

I have splited all these files into two. EMP001.dat, EMP002.dat, DEPT001.dat, DEPT002.dat, BONUS001.dat, BONUS002.dat

Now take TO Parameters $Load_batch and $set_time

IF $load_batch ='N' the script should provide the information of the files to be processed (which are EMP.dat, DEPT.dat, BONUS.dat)

IF $load_batch = "y" The script should provide the information of batches (first batch files EMP001.dat, DEPT001.dat, BONUS001.dat; Second Batch EMP002.dat, DEPT002.dat, BONUS002.dat)

$set_time is a parameter defines the time between these batches...

Any ideas on this.

?

nagrcm

View Public Profile for nagrcm

Find all posts by nagrcm

7 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to make parallel processing rather than serial processing ??

Hello everybody, I have a little problem with one of my program. I made a plugin for collectd (a stats collector for my servers) but I have a problem to make it run in parallel. My program gathers stats from logs, so it needs to run in background waiting for any new lines added in the log...

2. Shell Programming and Scripting

File processing

bash-2.05$ vi file_read.sh "file_read.sh" 9 lines, 97 characters #!/bin/ksh print "Enter the group name" read gname varA= grep "$gname::" group print $varA This is the scenario: I am trying to get a userid and groupname from user and trying to add the userid to the given group....

3. Shell Programming and Scripting

help with file processing

Hi , I have a memory file like this with two columns: @C010 AA @C011 AA @C012 FE @C013 FF @C014 F7 @C015 FF first is memory add, second is the data. I wan to convert into a serial sequence starting from '00000' all the way to 'FFFFF' with those fields from the above file...

4. Shell Programming and Scripting

Help in file processing

Hi experts, please help for my script..I'm confused how to do this. i have 3 .csv file(excel file) The 5 .csv files are A.csv,B.csv,C.csv A.csv has column1 column2 a 1 b 4 c 2 d ...

5. Shell Programming and Scripting

processing with awk

I have many lines like the following in a file(there are also other kinds of lines) Host: 72.52.104.74 (tserv1.fmt2.he.net) Ports: 22/open/tcp//tcpwrapped///, 53/open/tcp//domain//PowerDNS 3.3/, 179/open/tcp//tcpwrapped/// Ignored State: closed (997) Seq Index: 207 IP ID Seq: All...

6. Programming

Data processing

Hello guys! I have some issue in how to processing some data. I have some files with 3 columns. The 1st column is a name of my sample. The 2nd column is a numerical sequence (very big sequence) starting from "1". And the 3rd column is a feature of each line, represented for a number (completely...

7. Programming

awk processing / Shell Script Processing to remove columns text file

Hello, I extracted a list of files in a directory with the command ls . However this is not my computer, so the ls functionality has been revamped so that it gives the filesizes in front like this : This is the output of ls command : I stored the output in a file filelist 1.1M...

LEARN ABOUT DEBIAN

stapgraph

STAP(1) 						      General Commands Manual							   STAP(1)

NAME

       stapgraph - systemtap grapher

SYNOPSIS

       stapgraph [ OPTIONS ] FILENAME [ ARGUMENTS ]
       stapgraph [ OPTIONS ] - [ ARGUMENTS ]
       stapgraph [ OPTIONS ] -e SCRIPT [ ARGUMENTS ]

DESCRIPTION

       The stapgraph program is a simple frontend to stap that produces graphs from the output of stap scripts.

       This manual corresponds to version 1.7.

OPTIONS

       stapgraph accepts all options accepted by stap.	They are passed through to the stap process running the script or probe points.

USER INTERFACE

       The script that produces results can be started from a dialog within the program, in addition to being launched using the program's command
       line arguments. This dialog allows arguments to be passed to stap and to the script.

       When running, a mouse left click on a graph pauses the display of data, even though it is still collected  by  the  program.  Dragging  the
       mouse scrolls the graph backwards and forwards in time; clicking the displayed pause button restarts the display of new data.

       The scroll wheel can be used to zoom in and out on the graph.

SCRIPT OUTPUT SYNTAX

       Scripts can output  Comma Separated Values (CSV) which will be interpreted by stapgraph.  The first value is the X coordinate of the graph,
       usually a time value.  stapgraph assumes that this value is constantly increasing.

       A different syntax allows the script to specify many details of the graph's appearance.

       %DataSet: DATA-SET-NAME	SCALE  COLOR  GRAPHSTYLE

       The data set name identifies a data set that will be emitted by the script.  "SCALE" is the overall vertical scale of the  graph  for  this
       data  set.  "COLOR"  is a red-green-blue hex color. "GRAPHSTYLE" is one of bar - a traditional bar graph dot - plotted points discreet - An
       event style timeline that records that an event happened at a certain time.

       %Title: TITLE
	      A title for the whole graph.

       %XAxisTitle: TITLE
	      The legend for the X axis

       %YAxisTitle: TITLE
	      The legend for the Y axis.

       %YMax: MAX VALUE
	      The maximum value displayed on the graph.

GRAPH DATA

	DATA SET  TIME	VALUE Each data point that is to be plotted is tagged with the data set name, followed by the time and value of the point.

SEE ALSO

       stap(3stap), staprun(3stap),

BUGS

       Use the Bugzilla link off of the project web page or our mailing list.  http://sourceware.org/systemtap/,<systemtap@sourceware.org>.

Red Hat 							    2013-05-14								   STAP(1)