Sponsored Content
Operating Systems Solaris How to safely copy full filesystems with large files (10Gb files) Post 302457247 by Corona688 on Monday 27th of September 2010 03:09:31 PM
Old 09-27-2010
Any sparse files may also change size when copied(blocks not stored in the input become stored blocks full of zeroes in the output), though the content remains identical.
 

9 More Discussions You Might Find Interesting

1. Answers to Frequently Asked Questions

32 or 64 bit filesystems/files/OS's/CPU's

Determining if an OS is using 32 or 64 bits This may be our number one question. Sadly I have not been able to find a definitive answer for Linux. If you have a Linux solution, please post it in our Linux forum. I will edit this post to include it. 32 / 64 bit (Solaris) hp-ux ... (0 Replies)
Discussion started by: Perderabo
0 Replies

2. UNIX for Dummies Questions & Answers

Finding hidden files under mounted filesystems

I have never heard of this before but someone at work here says there is a command to find files that are under currently mounted filesystems. Does anyone know what this command is and is it available on HP-UX? (3 Replies)
Discussion started by: keelba
3 Replies

3. Shell Programming and Scripting

Divide large data files into smaller files

Hello everyone! I have 2 types of files in the following format: 1) *.fa >1234 ...some text... >2345 ...some text... >3456 ...some text... . . . . 2) *.info >1234 (7 Replies)
Discussion started by: ad23
7 Replies

4. Shell Programming and Scripting

How to safely rm/mv files/directory

Hi all, Am writing a script that does a rm/mv if a file exist, however, in one scenario, one of the variables which is supposed to a variable for a directory is undefined/blank so instead of the variable resolving to /tmp/logfile.dmp, it resolves instead to / so the rm translates to a rm /... (2 Replies)
Discussion started by: newbie_01
2 Replies

5. Red Hat

Advice regarding filesystems handling large number of files

Hi All, I have a CentOS operating system installed. I work with really huge number of files which are not only huge in number but some of them really huge in size. Minimum number of files could be 1 million to 2 million in one directory itself. Some of the files are even several Gigabytes in... (2 Replies)
Discussion started by: shoaibjameel123
2 Replies

6. Shell Programming and Scripting

Copy down remote files and rename them to include the server name with full path

I need to pull down a good bit of files for another support team for an upgrade project. I have a server.list with all of the server names. I need to do two parts: FIRST: I have this example, but it does not list the server name in front of each line. #! /bin/bash for server in $(<... (10 Replies)
Discussion started by: asnatlas
10 Replies

7. Red Hat

Can all files under /tmp be safely removed

I wanted to know whether all files under /tmp can be safely removed. I guess that /tmp may also have temporary files for applications currently being worked on, so at the most those applications may just shut down. I hope that my question is clear whether all files under /tmp can be safely... (5 Replies)
Discussion started by: RHCE
5 Replies

8. Shell Programming and Scripting

Rsync - how to copy hidden folder or hidden files when using full path

Hello. I use this command : rsync -av --include=".*" --dry-run "$A_FULL_PATH_S" "$A_FULL_PATH_D"The data comes from the output of a find command. And no full source directories are in use, only some files. Source example... (2 Replies)
Discussion started by: jcdole
2 Replies

9. Shell Programming and Scripting

Safely Remove Files with Special Chars

Hey Guys, I'm swamped writing code for the forums: Could someone write a script or command line to safely delete files with special chars in filenames from a directory: Example: -rw-r--r-- 1 root root 148 Apr 30 23:00 ?xA?? -rw-r--r-- 1 root root 148... (8 Replies)
Discussion started by: Neo
8 Replies
E2IMAGE(8)						      System Manager's Manual							E2IMAGE(8)

NAME
e2image - Save critical ext2/ext3/ext4 filesystem metadata to a file SYNOPSIS
e2image [ -rsI ] device image-file DESCRIPTION
The e2image program will save critical ext2, ext3, or ext4 filesystem metadata located on device to a file specified by image-file. The image file may be examined by dumpe2fs and debugfs, by using the -i option to those programs. This can assist an expert in recovering cat- astrophically corrupted filesystems. In the future, e2fsck will be enhanced to be able to use the image file to help recover a badly dam- aged filesystem. If image-file is -, then the output of e2image will be sent to standard output, so that the output can be piped to another program, such as gzip(1). (Note that this is currently only supported when creating a raw image file using the -r option, since the process of creating a normal image file, or QCOW2 image currently requires random access to the file, which cannot be done using a pipe. This restriction will hopefully be lifted in a future version of e2image.) It is a very good idea to create image files for all of filesystems on a system and save the partition layout (which can be generated using the fdisk -l command) at regular intervals --- at boot time, and/or every week or so. The image file should be stored on some filesystem other than the filesystem whose data it contains, to ensure that this data is accessible in the case where the filesystem has been badly damaged. To save disk space, e2image creates the image file as a sparse file, or in QCOW2 format. Hence, if the sparse image file needs to be copied to another location, it should either be compressed first or copied using the --sparse=always option to the GNU version of cp. This does not apply to the QCOW2 image, which is not sparse. The size of an ext2 image file depends primarily on the size of the filesystems and how many inodes are in use. For a typical 10 gigabyte filesystem, with 200,000 inodes in use out of 1.2 million inodes, the image file will be approximately 35 megabytes; a 4 gigabyte filesys- tem with 15,000 inodes in use out of 550,000 inodes will result in a 3 megabyte image file. Image files tend to be quite compressible; an image file taking up 32 megabytes of space on disk will generally compress down to 3 or 4 megabytes. RESTORING FILESYSTEM METADATA USING AN IMAGE FILE
The -I option will cause e2image to install the metadata stored in the image file back to the device. It can be used to restore the filesystem metadata back to the device in emergency situations. WARNING!!!! The -I option should only be used as a desperation measure when other alternatives have failed. If the filesystem has changed since the image file was created, data will be lost. In general, you should make a full image backup of the filesystem first, in case you wish to try other recovery strategies afterwards. RAW IMAGE FILES
The -r option will create a raw image file instead of a normal image file. A raw image file differs from a normal image file in two ways. First, the filesystem metadata is placed in the proper position so that e2fsck, dumpe2fs, debugfs, etc. can be run directly on the raw image file. In order to minimize the amount of disk space consumed by a raw image file, the file is created as a sparse file. (Beware of copying or compressing/decompressing this file with utilities that don't understand how to create sparse files; the file will become as large as the filesystem itself!) Secondly, the raw image file also includes indirect blocks and directory blocks, which the standard image file does not have, although this may change in the future. Raw image files are sometimes used when sending filesystems to the maintainer as part of bug reports to e2fsprogs. When used in this capacity, the recommended command is as follows (replace hda1 with the appropriate device): e2image -r /dev/hda1 - | bzip2 > hda1.e2i.bz2 This will only send the metadata information, without any data blocks. However, the filenames in the directory blocks can still reveal information about the contents of the filesystem that the bug reporter may wish to keep confidential. To address this concern, the -s option can be specified. This will cause e2image to scramble directory entries and zero out any unused portions of the directory blocks before writing the image file. However, the -s option will prevent analysis of problems related to hash-tree indexed directories. Note that this will work even if you substitute "/dev/hda1" for another raw disk image, or QCOW2 image previously created by e2image. QCOW2 IMAGE FILES The -Q option will create a QCOW2 image file instead of a normal, or raw image file. A QCOW2 image contains all the information the raw image does, however unlike the raw image it is not sparse. The QCOW2 image minimize the amount of disk space by storing data in special format with pack data closely together, hence avoiding holes while still minimizing size. In order to send filesystem to the maintainer as a part of bug report to e2fsprogs, use following commands (replace hda1 with the appropri- ate device): e2image -Q /dev/hda1 hda1.qcow2 bzip2 -z hda1.qcow2 This will only send the metadata information, without any data blocks. However, the filenames in the directory blocks can still reveal information about the contents of the filesystem that the bug reporter may wish to keep confidential. To address this concern, the -s option can be specified. This will cause e2image to scramble directory entries and zero out any unused portions of the directory blocks before writing the image file. However, the -s option will prevent analysis of problems related to hash-tree indexed directories. Note that QCOW2 image created by e2image is regular QCOW2 image and can be processed by tools aware of QCOW2 format such as for example qemu-img. AUTHOR
e2image was written by Theodore Ts'o (tytso@mit.edu). AVAILABILITY
e2image is part of the e2fsprogs package and is available from http://e2fsprogs.sourceforge.net. SEE ALSO
dumpe2fs(8), debugfs(8) E2fsprogs version 1.42.5 July 2012 E2IMAGE(8)
All times are GMT -4. The time now is 10:07 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy