Sponsored Content
Operating Systems Solaris Best way to copy 4Tb of data from one filesystem to another Post 302998027 by rbatte1 on Wednesday 24th of May 2017 09:50:23 AM
Old 05-24-2017
The answer will always be 'It depends.'
  • What is the data like (how much can it compress)?
  • Are the file attributes set to compress them? This will cause delays in itself.
  • Is the target on the same server?
    • For a local copy, there could be disk contention.
    • What is you disk infrastructure like? Do you have multiple controllers, SAN disk (how many paths etc.) or simple disks with logical volume mirrors?
    • How much data are you copying?
    • How many files are there? A file creation takes several IO operations beyond the volume of data so many small files can copy slower than few bigger ones.
    • How much memory do you have? For a copy to the same server with lots of free memory, you might be able to cache the files to reduce IO contention something like this:
      find /path/to/source -type f -exec cat {} \; /dev/null
  • For a remote copy, what is your network like - and that could be endless........

What can you tell us about the server(s) and data? It will probably be trial and error.

If this is a regular process, consider rsync which will only copy differences. It might still be slower if you have milliions over small files though. If there is lots of data change, perhaps you should consider snapshots, but then you haven't told us what OS you have, so i don't know if that's available to you in any useful way.





Robin
 

10 More Discussions You Might Find Interesting

1. Solaris

Filesystem Copy

Hi, Fairly new to the Unix worldand was wonderig without using tar and cp is there another way to copy files from one filesystem to a mount point but preserving all data, time stamp etc.. Cheers, :D (2 Replies)
Discussion started by: Monkey007
2 Replies

2. UNIX for Dummies Questions & Answers

copy, clear and keep to new data only

I have an Ingres database logfile that grows constantly, iircp.log. It is always "attached" to the Ingres process that uses it, and I do not want to screw up the data. I have been copying it to another directory and then using vi on the original to reduce the size 34000 lines at a time. What I want... (1 Reply)
Discussion started by: sarge
1 Replies

3. UNIX for Dummies Questions & Answers

Copy unix filesystem to windows

Hi all Is there a way to copy a unix filesystem (folders, subfolder and files) to windows (AD) and at the same time maintaine the unix permission (user/group) when copied to windows (AD) filesystem? I want the same permission in windows as I had in unix in one copy job. Is this... (8 Replies)
Discussion started by: tomjen
8 Replies

4. Shell Programming and Scripting

Will moving data from one filesystem to another affect current software installation

Here we have concern. We have a IBM software installed in a server S1 in the location : /opt/IBM. In this server S1, we have /opt file system in local disk. We don't have any option to increase the file system there. We have created a separate /opt/IBM_NEW file system with 10GB in S1 server.... (1 Reply)
Discussion started by: mehimadri
1 Replies

5. Solaris

copy data from one old filesystem to newfilesystem

Hi gurus I configured raid 5 volume n ive created a filesystem and mounted it to a directory also...everythin is ready..the purpose of doing it is to move my data from an old filesystem pin02 to the newly created filesystem pin02_new...plz tel me the steps to move data without any... (10 Replies)
Discussion started by: madanmeer
10 Replies

6. Hardware

How to connect a 4TB G-Raid hard drive to a laptop?

I am having trouble connecting my 4TB G-Raid Hard drive to my Compaq Hp laptop can anyone tell me how or what I need in order to connect the 4T and getting it working. (10 Replies)
Discussion started by: Jake Wolf
10 Replies

7. Solaris

Copy data over a TB

Hi All, We are not able to grow a UFS filesystem since its size will be going over a TB and it wasn't created using -T with newfs. Hence we have decided to take the backup of all the files on another filesystem and recreate it using -T with newfs. Please recommend the most reliable... (2 Replies)
Discussion started by: vishalaswani
2 Replies

8. Shell Programming and Scripting

Copy and paste data

I need to copy from specified lines and paste the data into several other lines. XX123450008 xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x XX123451895 xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x ...... XX123452012 xx.x xx.x xx.x xx.x xx.x xx.x xx.x... (13 Replies)
Discussion started by: ncwxpanther
13 Replies

9. Solaris

Solaris Filesystem usage discrepancy after "copy"

I have a query someone may be able to shed some light on... We have a Solaris 10 OS Sun V490 server Sparc. I have a SAN attached EMC Clarrion LUN which we have app data stored on. Pseudo name=emcpower0a CLARiiON ID=CK200070300470 Due to storage requirements - I need to migrate this... (4 Replies)
Discussion started by: ru4n1
4 Replies

10. UNIX for Advanced & Expert Users

Errors received while restoring my data from ext3 filesystem

Hi All, I have dual boot on my laptop - Win8 And linuxMint. From linuxmint i had copied all my data found on the NTFS partitions (on my laptop) to an external hard drive (formatted with ext3). i used rsync for this. Now after my hard disk crashed, am restoring the data back from ext3... (2 Replies)
Discussion started by: coolatt
2 Replies
LVCONVERT(8)						      System Manager's Manual						      LVCONVERT(8)

NAME
lvconvert - convert a logical volume from linear to mirror or snapshot SYNOPSIS
lvconvert -m|--mirrors Mirrors [--mirrorlog {disk|core}] [--corelog] [-R|--regionsize MirrorLogRegionSize] [-A|--alloc AllocationPolicy] [-b|--background] [-i|--interval Seconds] [-h|-?|--help] [-v|--verbose] [--version] LogicalVolume[Path] [PhysicalVolume[Path]...] lvconvert -s|--snapshot [-c|--chunksize ChunkSize] [-h|-?|--help] [-v|--verbose] [-Z|--zero y|n] [--version] OriginalLogicalVolume[Path] SnapshotLogicalVolume[Path] DESCRIPTION
lvconvert will change a linear logical volume to a mirror logical volume or to a snapshot of linear volume and vice versa. It is also used to add and remove disk logs from mirror devices. OPTIONS
See lvm for common options. Exactly one of --mirrors or --snapshot arguments required. -m, --mirrors Mirrors Specifies the degree of the mirror you wish to create. For example, "-m 1" would convert the original logical volume to a mirror volume with 2-sides; that is, a linear volume plus one copy. --mirrorlog {disk|core} Specifies the type of log to use. The default is disk, which is persistent and requires a small amount of storage space, usually on a separate device from the data being mirrored. Core may be useful for short-lived mirrors: It means the mirror is regenerated by copying the data from the first device again every time the device is activated - perhaps, for example, after every reboot. --corelog The optional argument "--corelog" is the same as specifying "--mirrorlog core". -R, --regionsize MirrorLogRegionSize A mirror is divided into regions of this size (in MB), and the mirror log uses this granularity to track which regions are in sync. -b, --background Run the daemon in the background. -i, --interval Seconds Report progress as a percentage at regular intervals. -s, --snapshot Create a snapshot from existing logical volume using another existing logical volume as its origin. -c, --chunksize ChunkSize Power of 2 chunk size for the snapshot logical volume between 4k and 512k. -Z, --zero y|n Controls zeroing of the first KB of data in the snapshot. If the volume is read-only the snapshot will not be zeroed. Examples "lvconvert -m1 vg00/lvol1" converts the linear logical volume "vg00/lvol1" to a two-way mirror logical volume. "lvconvert --mirrorlog core vg00/lvol1" converts a mirror with a disk log to a mirror with an in-memory log. "lvconvert --mirrorlog disk vg00/lvol1" converts a mirror with an in-memory log to a mirror with a disk log. "lvconvert -m0 vg00/lvol1" converts a mirror logical volume to a linear logical volume. "lvconvert -s vg00/lvol1 vg00/lvol2" converts logical volume "vg00/lvol2" to snapshot of original volume "vg00/lvol1" SEE ALSO
lvm(8), vgcreate(8), lvremove(8), lvrename(8), lvextend(8), lvreduce(8), lvdisplay(8), lvscan(8) Red Hat, Inc LVM TOOLS 2.02.44-cvs (02-17-09) LVCONVERT(8)
All times are GMT -4. The time now is 09:25 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy