Sponsored Content
Special Forums UNIX and Linux Applications High Performance Computing Massively parallel on single core? Post 302395111 by Neo on Monday 15th of February 2010 03:56:54 AM
Old 02-15-2010
Seems overly academic.....

If practice, most people who have a requirement to run 100,000 parallel applications, they would turn do some distributed processing package, for example cluster management software.

Hardware and existing distributed processing software is cheaper (and more practical) than attempting to design a single-core solution (the title of this thread).

In general, you should design your HPC application as a distributed architecture and make the centralized approach a special case of a distributed architecture.
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

parallel processing

Hi I want to run two shell script files parallely. These two scripts are interacting with the database. can any body help on this Pls Regards Audippa naidu.M (3 Replies)
Discussion started by: audippa
3 Replies

2. UNIX for Dummies Questions & Answers

difference between Dual-core & Core-to-duo

Can anybody tell What is the exact difference between a Dual-core processor and a Core-to-duo processor ?Advance thanks to all my friends. (1 Reply)
Discussion started by: Ajith kumar.G
1 Replies

3. Programming

how to know the application run on which core, and run how many times on this core

I have a dual core pc, I write a application with two child process. I know I can add sched_get_cpu to know the process run on which core, but, it just when the sched_get_cpu is called, it will tell me the result, my quesion is how to know the child proceess spend how many times on one core. (2 Replies)
Discussion started by: yanglei_fage
2 Replies

4. Shell Programming and Scripting

Replace single quote with two single quotes in perl

Hi I want to replace single quote with two single quotes in a perl string. If the string is <It's Simpson's book> It should become <It''s Simpson''s book> (3 Replies)
Discussion started by: DushyantG
3 Replies

5. Shell Programming and Scripting

Multiple lines in a single column to be merged as a single line for a record

Hi, I have a requirement with, No~Dt~Notes 1~2011/08/1~"aaa bbb ccc ddd eee fff ggg hhh" Single column alone got splitted into multiple lines. I require the output as No~Dt~Notes 1~2011/08/1~"aaa<>bbb<>ccc<>ddd<>eee<>fff<>ggg<>hhh" mean to say those new lines to be... (1 Reply)
Discussion started by: Bhuvaneswari
1 Replies

6. Shell Programming and Scripting

For loop in parallel

Hello, My script shell is: for i in $(seq $nb_lignes) do //command java done Please, how can i execute all iteration in parallel ? Thank you so much. (9 Replies)
Discussion started by: chercheur857
9 Replies

7. Shell Programming and Scripting

How to replace file massively?

Hi Gurus, I need to change a large amout of file name's. for example: current file name: file_ABCDE_sufix.txt I need to change them as file_FGHIGHKE_sufix.txt. Is there any way I can change them with script. Thanks in advance (1 Reply)
Discussion started by: ken6503
1 Replies

8. Shell Programming and Scripting

Paste 2 single column files to a single file

Hi, I have 2 csv/txt files with single columns. I am trying to merge them using paste, but its not working.. output3.csv: flowerbomb everlon-jewelry sofft steve-madden dolce-gabbana-watchoutput2.csv: http://www1.abc.com/cms/slp/2/Flowerbomb http://www1.abc.com/cms/slp/2/Everlon-Jewelry... (5 Replies)
Discussion started by: ajayakunuri
5 Replies

9. Shell Programming and Scripting

Python GNU parallel single command on multiple cores

Hello, I have a 4 core machine. Here is my initial script cd /work/ python script.py input.txt output.txt 1 2 3 This script runs for 1.5hrs. So I read across the web and figured out that you can use GNU parallel to submit multiple jobs using parallel. But I am not sure if I can run... (4 Replies)
Discussion started by: jacobs.smith
4 Replies
dgscat(1)						      PT-Scotch user's manual							 dgscat(1)

NAME
dggath, dgscat, gscat - convert distributed source graphs to or from centralized ones SYNOPSIS
dggath [options] [igfile] [ogfile] dgscat [options] [igfile] [ogfile] gscat [options] [igfile] [ogfile] DESCRIPTION
The dggath program gathers distributed graphs into centralized graphs. It reads a set of files igfile representing fragments of a distrib- uted source graph, and writes them back on the form of a single centralized source graph ogfile. The dgscat program scatters centralized source graphs into distributed graphs. It reads a centralized source graph igfile and writes it back on the form of a set of files ogfile representing fragments of the corresponding distributed source graph. The gscat program does exactly the same as dgscat, but does not require to be run in a parallel environment. Since gscat processes the input centralized graph file as a text stream, it does not need to load the full graph in memory before building the distributed graph fragment files. It is therefore much less resource consuming, but does not allow for the checking of graph consistency, as it has no global vision of the graph structure. When file names are not specified, data is read from standard input and written to standard output. Standard streams can also be explicitly represented by a dash '-'. When the proper libraries have been included at compile time, dggath and dgscat can directly handle compressed graphs, both as input and output. A stream is treated as compressed whenever its name is postfixed with a compressed file extension, such as in 'brol.grf.bz2' or '-.gz'. The compression formats which can be supported are the bzip2 format ('.bz2'), the gzip format ('.gz'), and the lzma format ('.lzma', on input only). dggath and dgscat base on implementations of the MPI interface to spread work across the processing elements. It is therefore not likely to be run directly, but instead through some launcher command such as mpirun. DISTRIBUTED FILE NAMES
In order to tell whether programs should read from, or write to, a single file located on only one processor, or to multiple instances of the same file on all of the processors, or else to distinct files on each of the processors, a special grammar has been designed, which is based on the '%' escape character. Four such escape sequences are defined, which are interpreted independently on every processor, prior to file opening. By default, when a filename is provided, it is assumed that the file is to be opened on only one of the processors, called the root processor, which is usually process 0 of the communicator within which the program is run. The index of the root processor can be changed by means of the -r option. Using any of the first three escape sequences below will instruct programs to open in parallel a file of name equal to the interpreted filename, on every processor on which they are run. %p Replaced by the number of processes in the global communicator in which the program is run. Leads to parallel opening. %r Replaced on each process running the program by the rank of this process in the global communicator. Leads to parallel opening. %- Discarded, but leads to parallel opening. This sequence is mainly used to instruct programs to open on every processor a file of identical name. The opened files can be, according whether the given path leads to a shared directory or to directories that are local to each processor, either to the opening of multiple instances of the same file, or to the opening of distinct files which may each have a different content, respectively (but in this latter case it is much recommended to identify files by means of the '%r' sequence). %% Replaced by a single '%' character. File names using this escape sequence are not considered for parallel opening, unless one or several of the three other escape sequences are also present. For instance, filename 'brol' will lead to the opening of file 'brol' on the root processor only, filename '%-brol' (or even 'br%-ol') will lead to the parallel opening of files called 'brol' on every processor, and filename 'brol%p-%r' will lead to the opening of files 'brol2-0' and 'brol2-1', respectively, on each of the two processors on which the program were to run. OPTIONS
-c For dggath and dgscat only. Check the consistency of the input source graph after loading it into memory. -h Display some help. -rpnum Set root process for centralized files (default is 0). -V Display program version and copyright. EXAMPLE
Run dgscat on 5 processing elements to scatter centralized graph file brol.grf into 5 gzipped file fragments brol5-0.dgr.gz to brol5-4.dgr.gz. $ mpirun -np 5 dgscat brol.grf brol%p-%r.dgr.gz SEE ALSO
dgmap(1), dgord(1), dgtst(1), gmk_hy(1). PT-Scotch user's manual. AUTHOR
Francois Pellegrini <francois.pellegrini@labri.fr> August 03, 2010 dgscat(1)
All times are GMT -4. The time now is 10:31 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy