Sponsored Content
Top Forums UNIX for Advanced & Expert Users Thread synchronisation problem... Post 302320510 by Gnaag on Thursday 28th of May 2009 07:30:35 AM
Old 05-28-2009
Thread synchronisation problem...

Hello, hope my english will be sufficient to be clear enough...

I'm in progress on some script that should copy one big source file (200GB average) on one sata drive, to multiple (30+) sata drives.
Hardware is not the problem but copy performance is... If i launch all copy process at the same time, source device is overrun by read IO and performance is (very) bad. If i launch one copy after one copy, copy performance is ok but i have multiply copy time by 30...

Let's summary the story :

i thnak to some script that read via ''dd'' in block mode to a ramdrive, ''1MB per 1MB piece'', then this script launch n writing dd (n = number of destination devices) to destination devices from ramdrive.
Idea is reading from a ramdrive with 30+ thread at the same time do not cause catastrophic perfomrance drop as it does with mecanical source device.

BUT...

I'm just facing a simple but big problem :
in the writing part of my script, i'm launching several ''write dd'' at the same time :
dd if=/mnt/ramdrive/buf.dat of=$destination/stillwriting.tmp bs=$BS seek=$POINTEUR_BLOC oflag=append status=noxfer &
"&" is my only known way to launch parralel process in a script, the problem is that the script continue to run and especially begins to put following 1MB block to ramdrive BEFORE end of all writing process. This result in corrupted files. I'm quite sure of that diagnostic because if i insert some sleep commande in my writing loop, error do not occur (but performance is lower and it's kind of very dirty scripting...).

Would you know a way to pause running of my script until all writing process are finished ?


Thanks a lot for your help from Paris !
 

8 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

shared memory synchronisation

hello everybody i want to do synchronisation to access a shared memory bu i don't know too much how well i know that i should use semaphore have you any example of synchronisation of a shared memory by use of semaphore because i haven't find any thanks (0 Replies)
Discussion started by: student00
0 Replies

2. Shell Programming and Scripting

sh : URGENT synchronisation insmod in script

Hello, By now in linux 2.4, I have a sh script wich start 2 modules as follow : /sbin/insmod module1.o /sbin/insmod module2.o I added an application in user space named "user_app" which communicate with module1 with a /proc. I now tape the commands myself during code execution on a... (1 Reply)
Discussion started by: crip01
1 Replies

3. Post Here to Contact Site Administrators and Moderators

posting thread is problem

hi, i am unable to post new thread. pls revert back what i need to do. i already posted 3 threads.... I am getting message like " Message is too short" even though my message is more than 100 characters. (1 Reply)
Discussion started by: spc432
1 Replies

4. UNIX for Dummies Questions & Answers

Need help on file synchronisation in unix

i want to do file synchronisation...its a client-server model..have to do system call 'ls -l' in both client and server ..the server has to keeep track of client files and have to keep a back up of client files..and tracking must be done based on time stamp.. suppose if client contains a file... (2 Replies)
Discussion started by: shilpam,edappal
2 Replies

5. UNIX for Dummies Questions & Answers

thread pool problem

hello everyone. I want to implement a thread pool, with 10 threads most. Inside main,I call a function (lets say it foo) wich creates (if it is needed) or uses an existing thread from the pool and sends it to do a job.My problem is that I dont know how to pass the argument from the main to the... (2 Replies)
Discussion started by: garag11
2 Replies

6. Shell Programming and Scripting

rsync script for synchronisation and backup

hello, i need to modified my synch/back scripts.... i want that this script only syncro folders in destinationfolder. f.e. when in destination are two folders 1) admin 2) users but in SOURCE are three: 1) admin 2) users 3) antivirus the script should only increnmential sync the... (0 Replies)
Discussion started by: onkeldave
0 Replies

7. Shell Programming and Scripting

Synchronisation of 2 arrays

If the element found in array2 doesn't exist in array1 i want i to be deleted. This code doesn't work correctly. What's wrong? Is there a simpler solution? for (( i=0; i<=${#array2}; i++ )) do for (( v=0; v<=${#array1}; v++ )) do if }" = "{$array1}" ] then count=1 break... (3 Replies)
Discussion started by: iums1
3 Replies

8. Programming

Thread synchronisation

hi, i have to do a program with following condition. please help me to write the program. conditions-i have to create a thread with handle called first and it should call the member function(may do anything lik print anything) of a class called thread1 and for example let take that this first... (5 Replies)
Discussion started by: senthil.march
5 Replies
nv_sata(7D)							      Devices							       nv_sata(7D)

NAME
nv_sata - Nvidia ck804/mcp55 SATA controller driver SYNOPSIS
sata@unit-address DESCRIPTION
The nv_sata driver is a SATA HBA driver that supports Nvidia ck804 and mcp55 SATA HBA controllers. Note that while these Nvidia con- trollers support standard SATA features including SATA-II drives, NCQ, hotplug and ATAPI drives, the driver currently does not support NCQ features. CONFIGURATION
The nv_sata module contains no user configurable parameters. FILES
/kernel/drv/nv_sata 32-bit ELF kernel module (x86). /kernel/drv/amd64/nv_sata 64-bit ELF kernel module (x86). ATTRIBUTES
See attributes(5) for descriptions of the following attribute: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Architecture |x86 | +-----------------------------+-----------------------------+ |Availability |SUNWnvsata | +-----------------------------+-----------------------------+ SEE ALSO
cfgadm(1M), cfgadm_sata(1M), prtconf(1M), sata(7D), sd(7D) Writing Device Drivers SunOS 5.11 22 Jul 2008 nv_sata(7D)
All times are GMT -4. The time now is 06:11 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy