Sponsored Content
Top Forums Shell Programming and Scripting Extract contents of tar ball without extracting files Post 302601539 by jim mcnamara on Friday 24th of February 2012 12:47:08 AM
Old 02-24-2012
You have to store data somewhere. We have two choices - memory or mass storage (tape, disk, etc). You just eliminated one, so we are left with memory. That means a program has to receive the data

When tar outputs, not just data comes out, also metadata. So you get to filter it out, I don't know how to do that a priori.

Code:
tar xvf somefile.tar  /full/path/to/file/to/extract.dat | ( your program lives in here )

Your program has to read from stdin, pitch metadata, and then keep real data. I wonder what you do with it then if you don't put some of it to disk or the tty.

The only point I would make is this is an odd request as tar is inherently slower than disk I/O, so why would you do all this? No spare disk space?
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extracting .tar files.

Hey guys complete n00b here so I'll try my best at explaining. I'm creating a backup and restore utility and decided to use tar. I create a backup folder in each user's account and when backing up (say word processing files), I use the following: tar cvf /home/user/backup/wpbackup.tar... (2 Replies)
Discussion started by: EwanD
2 Replies

2. UNIX for Advanced & Expert Users

Size of a tarball without untarring - Catch parent tar ball has sub tars

hi, I am in a weird situation. I have a parent tarball which contains 2 sub tarballs. The structure is such : Parent.tar.gz ---- > child1.tar.gz and child2.tar.gz I need to get the size of the parent tarball without untaring it I know that the command is gunzip -c parent.tar.gz | wc -c ... (1 Reply)
Discussion started by: mnanavati
1 Replies

3. Shell Programming and Scripting

Extract files from tar ball without directory structure

Hi, I have tar filw which has multiple directories which contain files. When i extract using tar -xf the directory structure also get extracted. I require only files and not directory structures as there will be overhead of moving the files again. So i searched here and got a solution but... (4 Replies)
Discussion started by: chetan.c
4 Replies

4. UNIX for Advanced & Expert Users

Extracting directories only from tar ball

Hi Somebody must have done this before, but I can't seem to find any answer on my problem. On HP-UX 11i v3 I have a relatively large tar ball (~120 GB), and I want to create the directory structure only from the archive. There is no option to make a new archive with only the directory... (3 Replies)
Discussion started by: hpvm_adm
3 Replies

5. UNIX for Dummies Questions & Answers

Listing contents of .tar.gz files

Hi All, I would like to know couple of ways to list the content available in tar and gzipped file without extracting. i.e., I would like to display the contents of test.tar.gz without extracting. Note :: please suggest a command other that tar -ztvf (9 Replies)
Discussion started by: Girish19
9 Replies

6. Shell Programming and Scripting

Extract a tar ball into multiple directories

#cat a BAC064DAL BAC063DAL BAC056PHX BAC066DAL BAC062PHX BAC062DAL BAC060DAL BAC058PHX BAC054PHX BAC051PHX # for i in `cat a` > do > tar xvf $a/$a*.tar* > done tar: /*.tar*: Cannot open: No such file or directory tar: Error is not recoverable: exiting now tar: /*.tar*: Cannot... (3 Replies)
Discussion started by: kenshinhimura
3 Replies

7. Shell Programming and Scripting

Need Tar Ball command to exclude directories

Hi, uname -a SunOS mymac 5.11 11.2 sun4u sparc SUNW,SPARC-Enterprise I need to tar a folder /tmp/moht but do not want these three folders to be included in the tar file -> savejpg, bmpsave and imgsave I tried --exclude, -path, -not options but it says bad option Can you help me with... (3 Replies)
Discussion started by: mohtashims
3 Replies

8. Shell Programming and Scripting

Untar only folder structure from a tar ball

I have a tar file hello.tar which is 95 GB. hello.tar has many files and folders including some tar files as well. I wish to create a new tar ball which should maintain only the folder structure of hello.tar and the tar ball within the hello.tar So basically the idea is to untar... (2 Replies)
Discussion started by: mohtashims
2 Replies

9. Shell Programming and Scripting

Problem creating a tar ball in different directories

Hi all. I'm hitting a problem creating a tar archive in one directory from files located in a different directory. It fails when I replace the absolute paths with variables in the script but works if I just run tar on the cmdln. E.g. #!/bin/ksh BASE=$PWD STAGE=$BASE/stage LOG=$BASE/log... (4 Replies)
Discussion started by: user052009
4 Replies

10. Shell Programming and Scripting

Create Tar ball

Hi Team, Following unix command is throwing error. Can anyone please help me to fix the issue? tar -cvf /aa/bb/cc/tarball1.tar /x/y/z1/abc.ksh /x/y/z2/pqr.txt /x/y/z3/lmn.tmp Error message thrown: tar: Removing leading `/' from member names OS: uname -a Linux xyz... (1 Reply)
Discussion started by: kmanivan82
1 Replies
dds2tar(1)						      General Commands Manual							dds2tar(1)

NAME
dds2tar - tool for fast tape access SYNOPSIS
dds2tar [ -f device ] [ -t indexfile ] [options] string ... DESCRIPTION
dds2tar uses an index to find the files with record seek (a fast operation of DAT devices). Since the file structure of the tape archives is used to extract the files, the archive has to be created by tar, compressed only by (the transparent signal processor of) the device. So you can step through the archive very quickly and extract files. The index may be created using dds2index or tar-vRt and is normally stored as a file on your hard disk. A tar archive is a sequence of blocks (e.g. 10240 bytes by default), each containing the same number (20 by default) of records, 512 byte each. dds2tar reads the tape and writes the tar records of the specified files (that means the header record and the data records of each selected file) to stdout. You may pipe the dds2tar output to the stdin of tar -xvvf - to restore the files to your disk. (See EXAMPLES below.) Before a file is extracted, the records of parent directories of the file are also written to stdout. The index of the archive should contain enough information to compute the number of the block containing the header of each selected file. dds2index will give such a table, tar -Rvt e.g. will not (only record numbers are listed). A patch for GNU tar-1.12 is available, adding the option --record-file. This patch is not included in the version of tar that ships with Debian. Alternatively there are some tricks to get the missing information. The strings are regular expressions to select the files. The matching algorithm is the one from GNU tar. If the option -l is given, the matched file names are printed to stdout (You may not pipe this list of pathnames to tar!). The default device is /dev/nst0, which may be overridden with the environment variable TAPE, which in turn may be overridden with the -f device option. The device must be a SCSI tape device. OPTIONS
-f devicefile Device of the tape archive. Must be a SCSI tape device. -t indexfile Specifies the index file (default is stdin). -s # Set the number of the first tape block of the archive. This option is useful only if the index file contains the verbose output of tar -Rvt. Any information about the first block inside the index file will be overridden by this option. If no information is available, the archive has to be the first file of the tape. If you have positioned your tape at the first block of the archive, you can use dds2tar `mt-dds` -t index ... | tar -f - ... to complete the information of the output of tar -Rvt stored in the index file. -b # Set the blocksize of the archive (tar -b #). This option is useful only if the index file contains the verbose output of tar (or if you have problems with the size of the internal buffer of dds2tar). Any information about the blocksize inside the index file will be overridden by this option. If no information is available, the default blocksize of tar is used. -z The index file should be read and stored in compressed mode. OPTIONS you didn't really need --z, --no-compress Don't filter the archive file through gzip. -q, --quick Don't extract the parent directories of the selected objects from tape. --body Write only the first selected file to stdout. This is useful if you want to read a file or extract an archive which is part of the current archive. -v,--verbose verbose mode. --hash-mode Print a hash sign for each MB. -V,--version Print only the Version Number to stderr. -l Don't access the tape but print the file names to stdout. You may not pipe this list of pathnames into tar. --extract The stdout is closed and opened by a pipe to the command tar -fxb - 1 . You may find this option convenient, I like to pipe the output to tar by hand. EXAMPLES
Example of getting the index from the default tape /dev/nst0 and storing it in file archive.idx: dds2index -t archive.idx Alternatively you can use a patched version of tar to create an index file. With the patch you can direct the errors and warning to stdout and the index information including information about the blocksize and the number of the first block to a file: tar -t --record-file archive.idx If the archive is the first file of the tape and the blocksize is the default of 20, you can use the verbose output of tar (-Rv) as an index file. tar -t -v -R | tee archive.idx If the archive is not the first file of the tape, you can store all the necessary information inside the index file with the use of mt-dds and tar : mt asf ... mt-dds tell > archive.idx tar -tvR >>archive.idx Example of using dds2tar to extract the gnu library (all files containing the string "glibc" in filename) from the default tape /dev/nst0, using the previously stored index file archive.idx: dds2tar -t archive.idx '*glibc*' | tar xvvf - To see in advance what would happen in the previous command without actually writing anything to your disk, you may use: dds2tar -t archive.idx '*glibc*' | tar tvvf - Example of checking the matches. You may try: dds2tar -t archive.idx -l '*glibc*' BACKGROUND INFORMATION
tapes A tape device handles all I/O (read, write, seek) in units of tape records. The bigger a tape record, the more effective usually is the access (and the less gaps are on QIC-tapes). However, normally a program will only read or write complete tape records. Normal tape drives allow to seek only relative to the current position. However, some newer SCSI-2 tapes, i.e. DAT, conforming to the DDS standard, keep track of the absolute position on the tape by inserting the tape record number inside each track. This number can be read while the fast seek is performed. The tar(1) program uses a slightly different terminology. It calls tape blocks what normally is called tape records. In the following sections we use the tar terminology to avoid confusion. tar The unit inside a tar archive is a tar record with a fixed length of 512 bytes. Every file, directory or soft link will occupy at least one tar record of information about pathname, permission information and so on called header record. The data of each file is stored in addi- tional tar records directly after the header record of that file. tar reports the tar record number of every header record in the archive with its -R option. tar counts the records continuously, starting with 0 (if invoked as tar -tR) or with 1 (if invoked as tar -cR). tar handles multiple records as a tar block, mainly to make the access of tapes (or disks) more efficient (and save tape space of QIC- tapes). tar only writes and reads full blocks to or from an archive. The -b option of tar controls, how many records are in one block. The default number of records per block is 20. This number is usually called the tar block size. However, this term is a little bit con- fusing, since it does not mean the number of bytes in a block. Thus a perhaps better name would be the tar blocking factor. tar on tapes tar writes or reads its archive to or from tape in units of tar blocks. As stated above, only a complete tape block may be transferred to/from tape. To extract a specific tar block from tape, one has to read an entire tape block into a buffer and extract the specified tar record from the buffer manually. If you would like to read a tar record with a given number, you have to know the number of the first tape block of the archive and the tar block size to compute the number of the tape block witch contains the tar record to read. If the tar ar- chive is the first file on the tape, the tape block number is the equal to the tar block number. Example: A file with the tar record number 1234 (records start with 0) may be found in a tape tar archive, written with a blocking factor of 20. It may be found in the tar block with the number blk = (int) 1234/20 = (int) 61.7 = 61 which is also the tape block number. The requested file is within this tar block at the record offset rec = 1234-(61*20) = 14 in 512 byte units. If a current archive is not the first archive on the tape, then the number of tape blocks of all previous archives has to be added to the block number computed above, to get the current tape block number. The number of previous tape records should be obtained from DDS devices when the tape is positioned at the beginning of the current archive (use mt-dds without arguments for example). Example: Assuming the archive in the above example to be the second file on a tape, and the archive starts at tape block 20222. Then we will find our file with tar record number 1234 in the tape block tblk = 20222 + (int) 1234/20 = 20283 on the tape. The record offset inside the tape block will be the same as above. WARNING
This program can only read records (tar is calling them tape blocks) up to 32 kbytes due to the limitations of the Linux device driver. The extracted archive is written to stdout with a block size of 512 bytes. ENVIRONMENT
The environment variable TAPE overrides the default tape device /dev/nst0. The variable DDS2TAR cat be used to give some options, e.g. --compress, -z, -s # , -b #. SEE ALSO
dds2index(1), mt(1), mt-dds(1), tar(1) HISTORY
This program was created to use the fast seek operation of my DAT streamer. The tapes are called dds (digital data storage). Since the program will write a tar archive to stdout, I called this program dds2tar. If I created the index file, I'm now able to restore a file of 1MB within one minute even if the tape contains more than 2GB of data. Thanks to Andreas (Andreas_Bagge@h2.maus.de), who has written a nice manual page for the overloaded version 1.1.3 of the program dds2tar (I added too much features ... ) His manual page for dds2tar-1.1.3 gave me the idea how to split the program dds2tar into the peaces dds2tar, dds2index and mt-dds. Additionally his manual page was the starting point for this page. Since the version 2.2 has a very robust algorithm to read the index file and the ability of pattern matching, a lot of options where obso- lete and has been deleted. I tried to make dds2tar as simple I can. AUTHOR
J"org Weule (weule@cs.uni-duesseldorf.de), Phone +49 211 751409. This software is available at ftp.uni-duesseldorf.de:/pub/unix/apollo and sunsite.unc.edu:/pub/Linux/system/Backup 2.3 dds2tar(1)
All times are GMT -4. The time now is 07:12 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy