Sponsored Content
Full Discussion: Copy files in Parallel
Top Forums UNIX for Dummies Questions & Answers Copy files in Parallel Post 302389153 by Corona688 on Friday 22nd of January 2010 11:07:49 PM
Old 01-23-2010
Running 500 threads won't make your disk copy any faster unless your disk has 500 heads.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

split process files in parallel and rejoin

Hi I need to split a huge file into multiple smaller files using split command. After that i need to process each file in the back ground with sql loader .Sql loader is a utlity to load CSV files into oracle . Check the status of each of these sqlloaders and then after sucessfull... (6 Replies)
Discussion started by: xiamin
6 Replies

2. UNIX for Advanced & Expert Users

implementation of copy command in parallel

hey i have to implement copy command in parallel in c language. i dont know how to create a new directory in destination. if anything u know related to this help me (1 Reply)
Discussion started by: rajsekhar28
1 Replies

3. Shell Programming and Scripting

parallel excution for 2000 files.

Hi, I have a function abc(). i am calling the function 9 times. it is working fine and completed the script execution 10 hours. input files: CUSTOMER_INFO_1111_12345.csv CUSTOMER_INFO_1222_12345.csv CUSTOMER_INFO_1322_12345.csv CUSTOMER_INFO_1333_12345.csv CUSTOMER_INFO_1151_12345.csv... (4 Replies)
Discussion started by: onesuri
4 Replies

4. Shell Programming and Scripting

scp or rsync multiple files in parallel from a remote host

Hi. I'm trying to speed up an rsync command by running it in parallel. There's no real option for this other than if the files are in multiple directories (which they're not). And even then there's no way of knowing if rsync has succeeded as the process is running in the background .. and... (4 Replies)
Discussion started by: Big_Jeffrey
4 Replies

5. Shell Programming and Scripting

Need to read two files in parallel

Following is the requirement In FileA I have the content as follows. 1,2,3 111,222,333 1000,2000,3000 In FileB I have the content as follows. 4,5,6 444,555,666 4000,5000,6000 I need to read FileA and FileB parallely and create the FileC as follows. 1,2,3,4,5,6... (1 Reply)
Discussion started by: kmanivan82
1 Replies

6. Shell Programming and Scripting

Comparing list of files in parallel

Hi everyone. I have a list of files like: file001 file002 file003 . . . . file385 file386 file387 There are more files than above, but I hope you understand what I'm trying to do here. Is there a way I can create a loop to compare: file001 with file385 file002 with file386 (9 Replies)
Discussion started by: craigsky
9 Replies

7. Shell Programming and Scripting

Parallel move keeping folder structure along with files in it

The below will move all the files in the directory dir to the destination using parallel and create a log, however will not keep them in the directory. I have tried mkdir -p but that does not seem to work or at least I can not seem to get it (as it deletes others files when I use it). What is the... (2 Replies)
Discussion started by: cmccabe
2 Replies

8. Shell Programming and Scripting

Copy file from different ports in parallel

Hello folks, Can you please help me to solve the below concern. I have a source server with 2 ports and have to copy the files from both the port to destination server simultaneously in my shell script. How can I achieve that? Source : x.x.x.x port -22 X.x.x.x port -2222 ... (7 Replies)
Discussion started by: sadique.manzar
7 Replies

9. Shell Programming and Scripting

Alignment tool to join text files in 2 directories to create a parallel corpus

I have two directories called English and Hindi. Each directory contains the same number of files with the only difference being that in the case of the English Directory the tag is .english and in the Hindi one the tag is .Hindi The file may contain either a single text or more than one text... (7 Replies)
Discussion started by: gimley
7 Replies

10. UNIX for Beginners Questions & Answers

How to paste multiple files in parallel?

Hi all, I am trying to paste thousands of files together into a matrix. Each file has only 1 column and all the files have the same number of rows (~27k rows). I tried paste * > output as well as some other for loops but the output only contains the columns from the 1st and last files. The... (8 Replies)
Discussion started by: notimenocall
8 Replies
DB5.1_HOTBACKUP(1)					      General Commands Manual						DB5.1_HOTBACKUP(1)

NAME
db5.1_hotbackup - Create "hot backup" or "hot failover" snapshots SYNOPSIS
db5.1_hotbackup [-cDuVv] [-d data_dir ...] [-h home] [-l log_dir] [-P password] -b backup_dir DESCRIPTION
The db5.1_hotbackup utility creates "hot backup" or "hot failover" snapshots of Berkeley DB database environments. The db5.1_hotbackup utility performs the following steps: 1. If the -c option is specified, checkpoint the source home database environment, and remove any unnecessary log files. 2. If the target directory for the backup does not exist, it is created with mode read-write-execute for the owner. If the target directory for the backup does exist and the -u option was specified, all log files in the target directory are removed; if the -u option was not specified, all files in the target directory are removed. 3. If the -u option was not specified, copy application-specific files found in the database environment home directory, or any directory specified using the -d option, into the target directory for the backup. 4. Copy all log files found in the directory specified by the -l option (or in the database environment home directory, if no -l option was specified), into the target directory for the backup. 5. Perform catastrophic recovery on the hot backup. 6. Remove any unnecessary log files from the hot backup. The db5.1_hotbackup utility does not resolve pending transactions that are in the prepared state. Applications that use DB_TXN->prepare should specify DB_RECOVER_FATAL when opening the environment, and run DB_ENV->txn_recover to resolve any pending transactions, when failing over to the hot backup. OPTIONS
-b Specify the target directory for the backup. -c Before performing the snapshot, checkpoint the source database environment and remove any log files that are no longer required in that environment. To avoid making catastrophic failure impossible, log file removal must be integrated with log file archival. -d Specify one or more source directories that contain databases; if none is specified, the database environment home directory will be searched for database files. As database files are copied into a single backup directory, files named the same, stored in different source directories, could overwrite each other when copied into the backup directory. -h Specify the source directory for the backup, that is, the database environment home directory. -l Specify a source directory that contains log files; if none is specified, the database environment home directory will be searched for log files. -P Specify an environment password. Although Berkeley DB utilities overwrite password strings as soon as possible, be aware there may be a window of vulnerability on systems where unprivileged users can see command-line arguments or where utilities are not able to overwrite the memory containing the command-line arguments. -u Update a pre-existing hot backup snapshot by copying in new log files. If the -u option is specified, no databases will be copied into the target directory. -V Write the library version number to the standard output, and exit. -v Run in verbose mode, listing operations as they are done. -D Use the data directories listed in the DB_CONFIG configuration file in the source directory. This option has three effects: First, if they do not already exist, the specified data directories will be created relative to the target directory (with mode read-write- execute owner). Second, all files in the source data directories will be copied to the target data directories. If the DB_CONFIG file specifies one or more absolute pathnames, files in those source directories will be copied to the top-level target directory. Third, the DB_CONFIG configuration file will be copied from the +source directory to the target directory, and subsequently used for configuration if recovery is run in the target directory. Care should be taken with the -D option and data directories which are named relative to the source directory but are not subdirectories (that is, the name includes the element "..") Specifically, the constructed target directory names must be meaningful and distinct from the source directory names, otherwise running recovery in the target directory might corrupt the source data files. It is an error to use absolute pathnames for data directories or the log directory in this mode, as the DB_CONFIG configuration file copied into the target directory would then point at the source directories and running recovery would corrupt the source data files. The db5.1_hotbackup utility uses a Berkeley DB environment (as described for the -h option, the environment variable DB_HOME, or because the utility was run in a directory containing a Berkeley DB environment). In order to avoid environment corruption when using a Berkeley DB environment, db5.1_hotbackup should always be given the chance to detach from the environment and exit gracefully. To cause db5.1_hot- backup to release all environment resources and exit cleanly, send it an interrupt signal (SIGINT). The db5.1_hotbackup utility exits 0 on success, and >0 if an error occurs. ENVIRONMENT
DB_HOME If the -h option is not specified and the environment variable DB_HOME is set, it is used as the path of the database home, as described in DB_ENV->open. AUTHORS
Oracle Corporation. This manual page was created based on the HTML documentation for db_hotbackup from Sleepycat, by Thijs Kinkhorst <thijs@kinkhorst.com>, for the Debian system (but may be used by others). 28 January 2005 DB5.1_HOTBACKUP(1)
All times are GMT -4. The time now is 09:41 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy