A thread is a lightweight process. The implementation of threads and processes differs from one operating system to another, but in most cases, a thread is contained inside a process. Multiple threads can exist within the same process and share resources such as memory, while different processes do not share these resources. -- excerpt from http://en.wikipedia.org/wiki/Thread_(computing)
Here is a sample use of GNU parallel that counts file contents with wc:
% ./s1
Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution : Debian GNU/Linux 5.0.8 (lenny)
bash GNU bash 3.2.39
parallel GNU parallel 20111122
-----
Structure of directories:
d1
|-- a.txt
|-- b.txt
|-- binary-1.exe
`-- c.txt
d2
|-- frog-town.jpg
|-- x.txt
`-- y.txt
0 directories, 7 files
-----
Results of parallel processes:
job 1, process 27495, wc = 4 16 70 d1/a.txt
job 2, process 27515, wc = 16 16 123 d1/b.txt
job 3, process 27535, wc = 26 265 1464 d1/c.txt
job 4, process 27555, wc = 4 16 70 d2/x.txt
job 5, process 27575, wc = 16 16 123 d2/y.txt
Each one of the tasks was run as a separate process. The calling sequence for parallel is complex, so some experimentation might be useful. I have not tried it, but I think parallel claims to be able to utilize different computers for tasks.
hello,
I have wrote a multi thread application to run under uclinux.
the problem is that threads does not share data. using the ps command it shows a single process for each thread.
I test the application under Ubuntu 8.04 and Open Suse 10.3 with 2.6 kernel and there were no problems and also... (8 Replies)
I have a unix directory where a million of small text files getting accumulated every week.
As of now there is a shell batch program in place which merges all the files in this directory into a single file and ftp to other system.
Previously the volume of the files would be around 1 lakh... (2 Replies)
awk "/May 23, 2012 /,0" /var/tmp/datafile
the above command pulls out information in the datafile. the information it pulls is from the date specified to the end of the file.
now, how can i make this faster if the datafile is huge? even if it wasn't huge, i feel there's a better/faster way to... (8 Replies)
Hi,
I have a large number of input files with two columns of numbers.
For example:
83 1453
99 3255
99 8482
99 7372
83 175
I only wish to retain lines where the numbers fullfil two requirements. E.g:
=83
1000<=<=2000
To do this I use the following... (10 Replies)
Hi,
I have a problem where I need to make this input:
nameRow1a,text1a,text2a,floatValue1a,FloatValue2a,...,floatValue140a
nameRow1b,text1b,text2b,floatValue1b,FloatValue2b,...,floatValue140b
look like this output:
nameRow1a,text1b,text2a,(floatValue1a - floatValue1b),(floatValue2a -... (4 Replies)
I have the below command which is referring a large file and it is taking 3 hours to run. Can something be done to make this command faster.
awk -F ',' '{OFS=","}{ if ($13 == "9999") print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12 }' ${NLAP_TEMP}/hist1.out|sort -T ${NLAP_TEMP} |uniq>... (13 Replies)
I have nginx web server logs with all requests that were made and I'm filtering them by date and time.
Each line has the following structure:
127.0.0.1 - xyz.com GET 123.ts HTTP/1.1 (200) 0.000 s 3182 CoreMedia/1.0.0.15F79 (iPhone; U; CPU OS 11_4 like Mac OS X; pt_br)
These text files are... (21 Replies)
Discussion started by: brenoasrm
21 Replies
LEARN ABOUT DEBIAN
srun_cr
SRUN_CR(1) slurm components SRUN_CR(1)NAME
srun_cr - run parallel jobs with checkpoint/restart support
SYNOPSIS
srun_cr [OPTIONS...]
DESCRIPTION
The design of srun_cr is inspired by mpiexec_cr from MVAPICH2 and cr_restart form BLCR. It is a wrapper around the srun command to enable
batch job checkpoint/restart support when used with SLURM's checkpoint/blcr plugin.
OPTIONS
The srun_cr execute line options are identical to those of the srun command. See "man srun" for details.
DETAILS
After initialization, srun_cr registers a thread context callback function. Then it forks a process and executes "cr_run --omit srun" with
its arguments. cr_run is employed to exclude the srun process from being dumped upon checkpoint. All catchable signals except SIGCHLD
sent to srun_cr will be forwarded to the child srun process. SIGCHLD will be captured to mimic the exit status of srun when it exits.
Then srun_cr loops waiting for termination of tasks being launched from srun.
The step launch logic of SLURM is augmented to check if srun is running under srun_cr. If true, the environment variable
SURN_SRUN_CR_SOCKET should be present, the value of which is the address of a Unix domain socket created and listened to be srun_cr. After
launching the tasks, srun tires to connect to the socket and sends the job ID, step ID and the nodes allocated to the step to srun_cr.
Upon checkpoint, srun_cr checks to see if the tasks have been launched. If not srun_cr first forwards the checkpoint request to the tasks
by calling the SLURM API slurm_checkpoint_tasks() before dumping its process context.
Upon restart, srun_cr checks to see if the tasks have been previously launched and checkpointed. If true, the environment variable
SLURM_RESTART_DIR is set to the directory of the checkpoint image files of the tasks. Then srun is forked and executed again. The envi-
ronment variable will be used by the srun command to restart execution of the tasks from the previous checkpoint.
COPYING
Copyright (C) 2009 National University of Defense Technology, China. Produced at National University of Defense Technology, China (cf,
DISCLAIMER). CODE-OCEC-09-009. All rights reserved.
This file is part of SLURM, a resource management program. For details, see <http://www.schedmd.com/slurmdocs/>.
SLURM is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free
Software Foundation; either version 2 of the License, or (at your option) any later version.
SLURM is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
SEE ALSO srun(1)srun_cr 2.0 March 2009 SRUN_CR(1)