Visit Our UNIX and Linux User Community


Newbie question


 
Thread Tools Search this Thread
Top Forums Programming Newbie question
# 1  
Old 06-06-2008
Newbie question

Dear all,

I have a question related to parallel programing and if you can give me some hints on how to deal with it, it would be really great.

I would like to run a small application on a supercompter of 128 CPUs. Unfortunately, on this machine only jobs which require 32 CPUs are allowed to run. My applications are, however, scalling only up to 16 CPUs. If I require 32 CPUs, the job will not run as I do not have enough data to fill the CPUs.

I think that maybe it is possible to submit, instead of a job on 32 CPUs, 2 jobs - each on 16 CPUs. This would mean that a master script (in MPI, probably) should ask at the begining for 32 CPUs, and then divide this partition into two smaller ones (of 16 CPUs), on each of them starting a different application (from a different directory).

As I have no experience with parallel programing I do not know to do this. Is it possible to give some hints on how to do it. (I do not expect someone to do the work in my place, but I need some hints to get start with, as possible keyword to use, etc).

Many thanks for any answer.

With all the best wishes,
Eduard
# 2  
Old 06-09-2008
Disclaimer: It's been a while since I have used parallel machines so I cannot remember details, but since nobody else has answered yet I'll have a go.

Now, your supercomputer might be set up differently, but every one I have ever used
uses a job submission system like PBS or NQS. You write a script with commands for the particular job submission system you are using (N.B. MPI is the parallel communications library, not a scripting language). You need to check whether this system allows you to subdivide a 32-node partition and run on each subdvision.

If this is impossible, and you cannot convince the admin to allow 16-node partitions, then you probably will have to change your program logic to run on 32-nodes, which means learning a little bit about parallel programming. Start by finding out where the magic number 16 comes from in your program, how the data is decomposed over the nodes, and then try to figure out how to generalise this for 32 (or any number) nodes.

Previous Thread | Next Thread
Test Your Knowledge in Computers #358
Difficulty: Medium
ibmawk is a fork of mawk, allowing applications to embed multiple parallel instances of awk interpreters.
True or False?

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

newbie question

Hi all, I am sure this is very simple but I cant quite get it. I am trying to search textfile1.txt for a string then take the results of the search and append the result to textfile3.txt So far I have used $ find file1.txt -exec grep "string i am looking for" '{}' \; -print this... (2 Replies)
Discussion started by: radgator
2 Replies

2. Shell Programming and Scripting

perl newbie . &&..programming newbie (question 2)

Hello everyone, I am having to do a lot of perl scripting these days and I am learning a lot. I have this problem I want to move files from a folder and all its sub folders to one parent folder, they are all .gz files.. there is folder1\folder2\*.gz and there are about 50 folders... (1 Reply)
Discussion started by: xytiz
1 Replies

3. UNIX for Dummies Questions & Answers

UNIX newbie NEWBIE question!

Hello everyone, Just started UNIX today! In our school we use solaris. I just want to know how do I setup Solaris 10 not the GUI one, the one where you have to type the commands like ECHO, ls, pwd, etc... I have windows xp and I also have vmware. I hope I am not missing anything! :p (4 Replies)
Discussion started by: Hanamachi
4 Replies

4. UNIX for Dummies Questions & Answers

Newbie question?

What is the best way to learn UNIX on the web, with out buying books? any link would be much help. Thank you in advance, L (1 Reply)
Discussion started by: lsoria1
1 Replies

5. UNIX for Dummies Questions & Answers

newbie question

I am taking a db classes toward oracle 10g. I am taking unix as well . I need to know what is the best option for os . should I use linux fedora. or get a sun box and start learning from there. Thanks (6 Replies)
Discussion started by: xzyan
6 Replies

6. UNIX for Dummies Questions & Answers

Very new newbie question

sorry if im not asking inthe right spot but, how do you turn the beeping off every time you hit a key onthe keyboard. I tried the click -n but it told me it didnt recognize click any help would be greatly appreciated ( the beeping is not going over well in the surrounding cubicles) thank you... (4 Replies)
Discussion started by: Split100
4 Replies

7. Shell Programming and Scripting

Newbie question

Hello, I have text file while looks this test1 test2 test3 test4 test5 test6 and if I want to parse it and make new file which would like this test1 test2 test3 test4 test5 test6 How can I do this in korn shell script Thanks (9 Replies)
Discussion started by: peeyush_23
9 Replies

8. UNIX for Dummies Questions & Answers

Newbie Question

Hi, I have a file, that is delimited by :: and the purpose of this file is none of your business. ;) There are about 65000 lines in this file, and there are lines that I would like to remove. About 45000 of them. Is there some sort of commands that I can run, to remove word(s) from this... (4 Replies)
Discussion started by: th3gh05t
4 Replies

9. UNIX Desktop Questions & Answers

Newbie Question

I am fairly new to *nix/linux and I have just installed SuSe 8.2. I am wondering what skills would be good to learn. I know that I will need to learn how to write scripts, but what scripting languages should I learn. I greatly appreciate any and all comments. ... (5 Replies)
Discussion started by: Ntalektual
5 Replies

10. UNIX for Dummies Questions & Answers

newbie question

hi im thinking of getting unix but i have no idea where to start I know that its an OS similar to linux but what hardware does in run on? i've heard of solaris but im not quit sure what it is thankxs (3 Replies)
Discussion started by: ninja
3 Replies

Featured Tech Videos