|
|
Sponsored Content
Top Forums
Shell Programming and Scripting
Grouping files into tars
Post 302301289 by muay_tb on Thursday 26th of March 2009 12:08:49 PM
03-26-2009
Grouping files into tars
Hi all,
I have a problem where i have several files in a directory which I SCP from a server to my local machine and i would like to periodically tar/gzip them based on their naming convention.
Here is the scenario:
I SCP files (which all end with the same ending) periodically across to a '/stored' location:
stat_t_123
stat_p_123
stat_c_123
stat_t_999
stat_p_999
stat_c_999
As you can see they all end in '123' or '999' (as an example).
So what I'm looking for is a way to query the '/stored' directory and look for all files ending with '123' or '999' and then grouping them together into a single tar/gzip'd file.
the tarred files will then remain in the '/stored' location.
So you would end up with:
/stored/files123.tar.gz
/stored/files999.tar.gz
I know i can get all files which are not tars or gzips like this:
`ls /stored | grep -v ".tar" | grep -v ".tar.gz"`
but how would i achieve the grouping?
any helps guys?
I have a problem where i have several files in a directory which I SCP from a server to my local machine and i would like to periodically tar/gzip them based on their naming convention.
Here is the scenario:
I SCP files (which all end with the same ending) periodically across to a '/stored' location:
stat_t_123
stat_p_123
stat_c_123
stat_t_999
stat_p_999
stat_c_999
As you can see they all end in '123' or '999' (as an example).
So what I'm looking for is a way to query the '/stored' directory and look for all files ending with '123' or '999' and then grouping them together into a single tar/gzip'd file.
the tarred files will then remain in the '/stored' location.
So you would end up with:
/stored/files123.tar.gz
/stored/files999.tar.gz
I know i can get all files which are not tars or gzips like this:
`ls /stored | grep -v ".tar" | grep -v ".tar.gz"`
but how would i achieve the grouping?
any helps guys?
|
|
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
how would I write a command line that creates a new file named stuff.txt in the current working directory which contains the number of directories in the current working directory, followed by the number of empty files in the current working directory, followed by the name of the working directory? (3 Replies)
Discussion started by: jorogon0099
3 Replies
2. Shell Programming and Scripting
Hello Friends,
I have .tar files which exists under different directories after the below code is run:
find . -name "*" -type f -print | grep .tar > tmp.txt
cat tmp.txt
./dir1/subdir1/subdir2/database-db1_28112009.tar
./dir2/subdir3/database-db2_28112009.tar... (2 Replies)
Discussion started by: EAGL€
2 Replies
3. UNIX for Advanced & Expert Users
hi,
I am in a weird situation. I have a parent tarball which contains 2 sub tarballs.
The structure is such :
Parent.tar.gz ---- > child1.tar.gz and child2.tar.gz
I need to get the size of the parent tarball without untaring it
I know that the command is gunzip -c parent.tar.gz | wc -c ... (1 Reply)
Discussion started by: mnanavati
1 Replies
4. Shell Programming and Scripting
I have a file, which is really large but i shortened it:
A3059GVS 1 A 01 Plate_1 40 25.37016 14.6298
A3059GVS 2 A 01 Plate_2 40 26.642002 13.3583
A3059GVS 3 A 02 Plate_1 40 25.381462 ... (4 Replies)
Discussion started by: mykey242
4 Replies
5. Shell Programming and Scripting
I have a list of fils stored insortedLst, and want to select certain fields to group specific files together:
Example of the files would be as below:
n02-z30-dsr65-ndelt0.25-varp0.002-16x12drw-run1.log
n02-z30-dsr65-ndelt0.25-varp0.002-16x12drw-run2.log... (2 Replies)
Discussion started by: kristinu
2 Replies
6. Shell Programming and Scripting
I have my log file as below
00:18:02 - Nothing normal; Garbage Collection kicked off & running from last 3 min...
00:19:02 - Nothing normal; Garbage Collection kicked off & running from last 4 min...
00:19:02 - Nothing normal; Garbage Collection kicked off & running from last 4 min...... (11 Replies)
Discussion started by: manas_ranjan
11 Replies
7. Shell Programming and Scripting
Hi all,
I am using following command:
perl program.pl input.txt output.txt CUTOFF 3 > groups_3.txt
containing program.pl, two files (input.txt, output.txt) and getting output in groups_3.txt:
But, I wish to have 30 files corresponding to each CUTOFF ranging from 0 to 30 using the same... (1 Reply)
Discussion started by: bioinfo
1 Replies
8. Shell Programming and Scripting
awk 'FNR==NR {a; next} $NF in a' genes.txt refseq_exons.txt > output.txt
I can not figure out how to group the same name in $4 together.
Basically, all the SKI together in separate rows and all the TGFB2. Thank you :).
chr1 2160133 2161174 SKI
chr1 218518675 218520389 TGFB2... (1 Reply)
Discussion started by: cmccabe
1 Replies
9. Shell Programming and Scripting
Hi All,
I want to read the input file and store the output in the Output file. I pasted the sample Input and Output file below. Help me with this.
Input file
=================================
ITEM1 AAAAA 1
ITEM1 BBBBB 1
ITEM1 CCCCC 1
ITEM2 AAAAA 5
ITEM2 CCCCC 4... (1 Reply)
Discussion started by: humaemo
1 Replies
10. Shell Programming and Scripting
I have this Requirement where i have to group the files,
I have a folder say "temp" where many files resides...files are like this;
010020001_S-ABC-Sort-DEFAW_YYYYMMDD_HHMMSS.txt
010020004_S-PQR-Sort-DRTON_YYYYMMDD_HHMMSS.txt
010020009_S-JKL-Sort_MNOLO_YYYYMMDD_HHMMSS.txt... (8 Replies)
Discussion started by: gnnsprapa
8 Replies
LEARN ABOUT CENTOS
virt-tar-out
virt-tar-out(1) Virtualization Support virt-tar-out(1) NAME
virt-tar-out - Pack a virtual machine disk image directory into a tarball. SYNOPSIS
virt-tar-out -a disk.img /dir files.tar virt-tar-out -d domain /dir files.tar virt-tar-out -d domain /dir - | gzip --best > files.tar.gz DESCRIPTION
"virt-tar-out" packs a virtual machine disk image directory into a tarball. The first parameter is the absolute path of the virtual machine directory. The second parameter is the tar file to write. Use "-" to write to standard output. EXAMPLES
Download the home directories from a guest: virt-tar-out -d MyGuest /home - | gzip --best > homes.tar.gz JUST A SHELL SCRIPT WRAPPER AROUND GUESTFISH
This command is just a simple shell script wrapper around the guestfish(1) "tar-out" command. For anything more complex than a trivial copy, you are probably better off using guestfish directly. OPTIONS
Since the shell script just passes options straight to guestfish, read guestfish(1) to see the full list of options. SEE ALSO
guestfish(1), virt-cat(1), virt-copy-in(1), virt-copy-out(1), virt-edit(1), virt-tar-in(1), http://libguestfs.org/. AUTHORS
Richard W.M. Jones ("rjones at redhat dot com") COPYRIGHT
Copyright (C) 2011-2012 Red Hat Inc. LICENSE
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. BUGS
To get a list of bugs against libguestfs, use this link: https://bugzilla.redhat.com/buglist.cgi?component=libguestfs&product=Virtualization+Tools To report a new bug against libguestfs, use this link: https://bugzilla.redhat.com/enter_bug.cgi?component=libguestfs&product=Virtualization+Tools When reporting a bug, please supply: o The version of libguestfs. o Where you got libguestfs (eg. which Linux distro, compiled from source, etc) o Describe the bug accurately and give a way to reproduce it. o Run libguestfs-test-tool(1) and paste the complete, unedited output into the bug report. libguestfs-1.22.6 2013-08-24 virt-tar-out(1)