12-23-2011
Copy data over a TB
Hi vishalaswani, there is a big difference in the copy performance of a filesystem if you have a lot of little files or a few big files, I think that if you are in the first case, you can use the command you explain, but if you are in the second case, perhaps you can try rsync, it will be faster, the command will be, for instance:
rsync -Havx ORIGINAL_FOLDER DESTINATION_FOLDER
Where DESTINATION_FOLDER can be a local folder or something like server:/folder, whit this parameters you will have a mirror copy, including the links. You can use -e rsh in order to avoid the default ssh option. Of course, if the folder structure is really big, you can launch a "foreach" command whit a rsync by folder, so you can copy several folders at the same time which can not be done with ufsdump. I'm copying around 4,5TB all nights from my NFS main server to a slow disk machines which a script based in this idea.
Also, you can use the same command with tar instead of ufsdump, it's a little bit faster, but remember the E option if you use long names.
10 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
I have a log file that I would like to copy from my windows pc to my unix pc for further processuon. Is there any command that can help me perform this task?. I already have a cron to fire up the process, but nothing seems to be coming up.
I am trying to use ftp but nothing is coming forth.... (8 Replies)
Discussion started by: odogbolu98
8 Replies
2. UNIX for Dummies Questions & Answers
We keep getting production files into an input directory.
These files will be processed three times a day.
8:00AM
1:30PM
5:30PM
file1.20041005_05303423.dat
File2.200041005_14232313.dat
Once the files are processed, they are archived immediately to prod/archive directory
... (1 Reply)
Discussion started by: zomboo
1 Replies
3. UNIX for Dummies Questions & Answers
I have an Ingres database logfile that grows constantly, iircp.log. It is always "attached" to the Ingres process that uses it, and I do not want to screw up the data. I have been copying it to another directory and then using vi on the original to reduce the size 34000 lines at a time. What I want... (1 Reply)
Discussion started by: sarge
1 Replies
4. AIX
I need to take data from one database to another on the same machine.
My first attempt has been using restore, but I am getting an error.
Here is the command I am trying:
db2 restore database prod into test
I get the following error:
The container is already in use.
What would... (2 Replies)
Discussion started by: jyoung
2 Replies
5. UNIX for Dummies Questions & Answers
Dear sir/madam
Could you tell me how to copy or get data from tape to any folder in unix?
Thanks, (2 Replies)
Discussion started by: seyha_moth
2 Replies
6. Solaris
I 've few data sets in my zfs pool which has been exported to the non global zones and i want to copy data on those datasets/file systems to my datasets in new pool mounted on global zone, how can i do that ? (2 Replies)
Discussion started by: fugitive
2 Replies
7. Solaris
Hi gurus
I configured raid 5 volume n ive created a filesystem and mounted it to a directory also...everythin is ready..the purpose of doing it is to move my data from an old filesystem pin02 to the newly created filesystem pin02_new...plz tel me the steps to move data without any... (10 Replies)
Discussion started by: madanmeer
10 Replies
8. UNIX for Dummies Questions & Answers
Hi All,
HP-UX dev4 B.11.11 U 9000/800 3251073457
I need to copy huge data from windows text file to vi editor. when I tried copy huge data, the format of data is not preserverd and appered to scatterd through the vi, something like give below. Please let me know, how can I correct this?
... (18 Replies)
Discussion started by: alok.behria
18 Replies
9. Shell Programming and Scripting
I need to copy from specified lines and paste the data into several other lines.
XX123450008 xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x
XX123451895 xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x xx.x
......
XX123452012 xx.x xx.x xx.x xx.x xx.x xx.x xx.x... (13 Replies)
Discussion started by: ncwxpanther
13 Replies
10. Solaris
Dear Experts,
I would like to know what's the best method for copy data around 3 mio (spread in a hundred folders, size each file around 1kb) between 2 servers?
I already tried using Rsync and tar command. But using these command is too long.
Please advice.
Thanks
Edy (11 Replies)
Discussion started by: edydsuranta
11 Replies
MKFS(8) System Manager's Manual MKFS(8)
NAME
mkfs, mkext - archive or update a file system
SYNOPSIS
disk/mkfs [-aprv] [-n name] [-s source] [-u users] [-z n] proto ...
disk/mkext [-d name] [-u] [-h] [-v] file ...
DESCRIPTION
Mkfs copies files from the file tree source (default /) to a kfs file system (see kfs(4)). The kfs service is mounted on /n/kfs, and
/adm/users is copied to /n/kfs/adm/users. The proto files are read, and any files specified in them that are out of date are copied to
/n/kfs.
Each line of the proto file specifies a file to copy. Indentation is significant, with each level of indentation corresponding to a level
in the file tree. Fields within a line are separated by white space. The first field is the last path element in the destination file
tree. The second field specifies the permissions. The third field is the owner of the file, and the fourth is the group owning the file.
The fifth field is the name of the file from which to copy; this file is read from the current name space, not the source file tree. All
fields except the first are optional.
Names beginning with a are expanded as environment variables. If the first file specified in a directory is all of the files in that
directory are copied. If the first file is all of the files are copied, and all subdirectories are recursively copied.
Mkfs copies only those files that are out of date. Such a file is first copied into a temporary file in the appropriate destination direc-
tory and then moved to the destination file. Files in the kfs file system that are not specified in the proto file are not updated and not
removed.
The options to mkfs are:
a Instead of writing to a kfs file system, write an archive file to standard output, suitable for mkext. All files in proto, not just
those out of date, are archived.
n name Use kfs.name as the name of the kfs service (default kfs).
p Update the permissions of a file even if it is up to date.
r Copy all files.
s source
Copy from files rooted at the tree source.
u users
Copy file users into /adm/users in the new system.
v Print the names of all of the files as they are copied.
z n Copy files assuming kfs block n (default 1024) bytes long. If a block contains only 0-valued bytes, it is not copied.
Mkext unpacks archive files made by the -a option of mkfs. The -d option specifies a directory (default /n/kfs) to serve as the root of
the unpacked file system. The -u option, to be used only when initializing a new fs(4) file system, sets the owners of the files created
to correspond to those in the archive and restores the modification times of the files. (This is only permitted at the initial load of the
files into a file system.) Each file on the command line is unpacked in one pass through the archive. If the file is a directory, all
files and subdirectories of that directory are also unpacked. When a file is unpacked, the entire path is created if it does not exist.
If no files are specified, the entire archive is unpacked; in this case, missing intermediate directories are not created. The -v option
prints the names and sizes of files as they are extracted; -h prints headers for the files on standard output instead of unpacking the
files.
EXAMPLES
Make an archive to establish a new file system:
disk/mkfs -a -u files/adm.users -s dist proto > arch
Unpack that archive onto a new file system:
srv il!newfs
mount -c /srv/il!newfs /n/newfs
disk/mkext -u -d /n/newfs < arch
FILES
/lib/proto
directory of prototype files.
/lib/proto/portproto
generic prototype file.
SOURCE
/sys/src/cmd/disk/mkfs.c
/sys/src/cmd/disk/mkext.c
SEE ALSO
prep(8), kfscmd(8), wren(3), pip(8)
MKFS(8)