Parllel Processing


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Parllel Processing
# 1  
Old 07-26-2007
Java Parllel Processing

In simple words of the scenario.

I have some set of files that are to loaded into a table in parallel

Loading part would be taken care of the Abintio graph. I must develop a script which instructs this graph about how to process the files like wat files to process and how to process by passing on some parameters.

This is the task...

let me explain u much better with an example.

Let have that i have the following files EMP.dat, DEPT.dat, BONUS.dat

I have splited all these files into two. EMP001.dat, EMP002.dat, DEPT001.dat, DEPT002.dat, BONUS001.dat, BONUS002.dat

Now take TO Parameters $Load_batch and $set_time

IF $load_batch ='N' the script should provide the information of the files to be processed (which are EMP.dat, DEPT.dat, BONUS.dat)

IF $load_batch = "y" The script should provide the information of batches (first batch files EMP001.dat, DEPT001.dat, BONUS001.dat; Second Batch EMP002.dat, DEPT002.dat, BONUS002.dat)

$set_time is a parameter defines the time between these batches...

Any ideas on this.

?
# 2  
Old 07-30-2007
#!/bin/sh
CURDIR=/home/user/folder

ls -1 *.dat > $CURDIR/dat_fiels
for i in `cat $CURDIR/dat_fiels`
do
echo $i | cut -d"." -f1 | grep "[0-9]" 1> /dev/null
if [ $? -eq 0 ]
then
echo "splitted file $i"
else
echo "normat file $i"
fi
done


o/p will be :

normat file BONUS.dat
splitted file DEPT001.dat
splitted file DEPT002.dat
splitted file DEPT003.dat
normat file DEPT.dat
splitted file EMP001.dat
splitted file EMP002.dat
splitted file EMP003.dat
normat file EMP.dat
Login or Register to Ask a Question

Previous Thread | Next Thread

7 More Discussions You Might Find Interesting

1. Programming

awk processing / Shell Script Processing to remove columns text file

Hello, I extracted a list of files in a directory with the command ls . However this is not my computer, so the ls functionality has been revamped so that it gives the filesizes in front like this : This is the output of ls command : I stored the output in a file filelist 1.1M... (5 Replies)
Discussion started by: ajayram
5 Replies

2. Programming

Data processing

Hello guys! I have some issue in how to processing some data. I have some files with 3 columns. The 1st column is a name of my sample. The 2nd column is a numerical sequence (very big sequence) starting from "1". And the 3rd column is a feature of each line, represented for a number (completely... (2 Replies)
Discussion started by: bfantinatti
2 Replies

3. Shell Programming and Scripting

processing with awk

I have many lines like the following in a file(there are also other kinds of lines) Host: 72.52.104.74 (tserv1.fmt2.he.net) Ports: 22/open/tcp//tcpwrapped///, 53/open/tcp//domain//PowerDNS 3.3/, 179/open/tcp//tcpwrapped/// Ignored State: closed (997) Seq Index: 207 IP ID Seq: All... (9 Replies)
Discussion started by: esolvepolito
9 Replies

4. Shell Programming and Scripting

Help in file processing

Hi experts, please help for my script..I'm confused how to do this. i have 3 .csv file(excel file) The 5 .csv files are A.csv,B.csv,C.csv A.csv has column1 column2 a 1 b 4 c 2 d ... (6 Replies)
Discussion started by: selvam
6 Replies

5. Shell Programming and Scripting

help with file processing

Hi , I have a memory file like this with two columns: @C010 AA @C011 AA @C012 FE @C013 FF @C014 F7 @C015 FF first is memory add, second is the data. I wan to convert into a serial sequence starting from '00000' all the way to 'FFFFF' with those fields from the above file... (15 Replies)
Discussion started by: return_user
15 Replies

6. Shell Programming and Scripting

File processing

bash-2.05$ vi file_read.sh "file_read.sh" 9 lines, 97 characters #!/bin/ksh print "Enter the group name" read gname varA= grep "$gname::" group print $varA This is the scenario: I am trying to get a userid and groupname from user and trying to add the userid to the given group.... (11 Replies)
Discussion started by: mnathan
11 Replies

7. Shell Programming and Scripting

How to make parallel processing rather than serial processing ??

Hello everybody, I have a little problem with one of my program. I made a plugin for collectd (a stats collector for my servers) but I have a problem to make it run in parallel. My program gathers stats from logs, so it needs to run in background waiting for any new lines added in the log... (0 Replies)
Discussion started by: Samb95
0 Replies
Login or Register to Ask a Question