03-26-2009
Grouping files into tars
Hi all,
I have a problem where i have several files in a directory which I SCP from a server to my local machine and i would like to periodically tar/gzip them based on their naming convention.
Here is the scenario:
I SCP files (which all end with the same ending) periodically across to a '/stored' location:
stat_t_123
stat_p_123
stat_c_123
stat_t_999
stat_p_999
stat_c_999
As you can see they all end in '123' or '999' (as an example).
So what I'm looking for is a way to query the '/stored' directory and look for all files ending with '123' or '999' and then grouping them together into a single tar/gzip'd file.
the tarred files will then remain in the '/stored' location.
So you would end up with:
/stored/files123.tar.gz
/stored/files999.tar.gz
I know i can get all files which are not tars or gzips like this:
`ls /stored | grep -v ".tar" | grep -v ".tar.gz"`
but how would i achieve the grouping?
any helps guys?
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
how would I write a command line that creates a new file named stuff.txt in the current working directory which contains the number of directories in the current working directory, followed by the number of empty files in the current working directory, followed by the name of the working directory? (3 Replies)
Discussion started by: jorogon0099
3 Replies
2. Shell Programming and Scripting
Hello Friends,
I have .tar files which exists under different directories after the below code is run:
find . -name "*" -type f -print | grep .tar > tmp.txt
cat tmp.txt
./dir1/subdir1/subdir2/database-db1_28112009.tar
./dir2/subdir3/database-db2_28112009.tar... (2 Replies)
Discussion started by: EAGL€
2 Replies
3. UNIX for Advanced & Expert Users
hi,
I am in a weird situation. I have a parent tarball which contains 2 sub tarballs.
The structure is such :
Parent.tar.gz ---- > child1.tar.gz and child2.tar.gz
I need to get the size of the parent tarball without untaring it
I know that the command is gunzip -c parent.tar.gz | wc -c ... (1 Reply)
Discussion started by: mnanavati
1 Replies
4. Shell Programming and Scripting
I have a file, which is really large but i shortened it:
A3059GVS 1 A 01 Plate_1 40 25.37016 14.6298
A3059GVS 2 A 01 Plate_2 40 26.642002 13.3583
A3059GVS 3 A 02 Plate_1 40 25.381462 ... (4 Replies)
Discussion started by: mykey242
4 Replies
5. Shell Programming and Scripting
I have a list of fils stored insortedLst, and want to select certain fields to group specific files together:
Example of the files would be as below:
n02-z30-dsr65-ndelt0.25-varp0.002-16x12drw-run1.log
n02-z30-dsr65-ndelt0.25-varp0.002-16x12drw-run2.log... (2 Replies)
Discussion started by: kristinu
2 Replies
6. Shell Programming and Scripting
I have my log file as below
00:18:02 - Nothing normal; Garbage Collection kicked off & running from last 3 min...
00:19:02 - Nothing normal; Garbage Collection kicked off & running from last 4 min...
00:19:02 - Nothing normal; Garbage Collection kicked off & running from last 4 min...... (11 Replies)
Discussion started by: manas_ranjan
11 Replies
7. Shell Programming and Scripting
Hi all,
I am using following command:
perl program.pl input.txt output.txt CUTOFF 3 > groups_3.txt
containing program.pl, two files (input.txt, output.txt) and getting output in groups_3.txt:
But, I wish to have 30 files corresponding to each CUTOFF ranging from 0 to 30 using the same... (1 Reply)
Discussion started by: bioinfo
1 Replies
8. Shell Programming and Scripting
awk 'FNR==NR {a; next} $NF in a' genes.txt refseq_exons.txt > output.txt
I can not figure out how to group the same name in $4 together.
Basically, all the SKI together in separate rows and all the TGFB2. Thank you :).
chr1 2160133 2161174 SKI
chr1 218518675 218520389 TGFB2... (1 Reply)
Discussion started by: cmccabe
1 Replies
9. Shell Programming and Scripting
Hi All,
I want to read the input file and store the output in the Output file. I pasted the sample Input and Output file below. Help me with this.
Input file
=================================
ITEM1 AAAAA 1
ITEM1 BBBBB 1
ITEM1 CCCCC 1
ITEM2 AAAAA 5
ITEM2 CCCCC 4... (1 Reply)
Discussion started by: humaemo
1 Replies
10. Shell Programming and Scripting
I have this Requirement where i have to group the files,
I have a folder say "temp" where many files resides...files are like this;
010020001_S-ABC-Sort-DEFAW_YYYYMMDD_HHMMSS.txt
010020004_S-PQR-Sort-DRTON_YYYYMMDD_HHMMSS.txt
010020009_S-JKL-Sort_MNOLO_YYYYMMDD_HHMMSS.txt... (8 Replies)
Discussion started by: gnnsprapa
8 Replies
LEARN ABOUT CENTOS
git-tar-tree
GIT-TAR-TREE(1) Git Manual GIT-TAR-TREE(1)
NAME
git-tar-tree - Create a tar archive of the files in the named tree object
SYNOPSIS
git tar-tree [--remote=<repo>] <tree-ish> [ <base> ]
DESCRIPTION
THIS COMMAND IS DEPRECATED. Use git archive with --format=tar option instead (and move the <base> argument to --prefix=base/).
Creates a tar archive containing the tree structure for the named tree. When <base> is specified it is added as a leading path to the files
in the generated tar archive.
git tar-tree behaves differently when given a tree ID versus when given a commit ID or tag ID. In the first case the current time is used
as modification time of each file in the archive. In the latter case the commit time as recorded in the referenced commit object is used
instead. Additionally the commit ID is stored in a global extended pax header. It can be extracted using git get-tar-commit-id.
OPTIONS
<tree-ish>
The tree or commit to produce tar archive for. If it is the object name of a commit object.
<base>
Leading path to the files in the resulting tar archive.
--remote=<repo>
Instead of making a tar archive from local repository, retrieve a tar archive from a remote repository.
CONFIGURATION
tar.umask
This variable can be used to restrict the permission bits of tar archive entries. The default is 0002, which turns off the world write
bit. The special value "user" indicates that the archiving user's umask will be used instead. See umask(2) for details.
EXAMPLES
git tar-tree HEAD junk | (cd /var/tmp/ && tar xf -)
Create a tar archive that contains the contents of the latest commit on the current branch, and extracts it in /var/tmp/junk directory.
git tar-tree v1.4.0 git-1.4.0 | gzip >git-1.4.0.tar.gz
Create a tarball for v1.4.0 release.
git tar-tree v1.4.0^{tree} git-1.4.0 | gzip >git-1.4.0.tar.gz
Create a tarball for v1.4.0 release, but without a global extended pax header.
git tar-tree --remote=example.com:git.git v1.4.0 >git-1.4.0.tar
Get a tarball v1.4.0 from example.com.
git tar-tree HEAD:Documentation/ git-docs > git-1.4.0-docs.tar
Put everything in the current head's Documentation/ directory into git-1.4.0-docs.tar, with the prefix git-docs/.
GIT
Part of the git(1) suite
Git 1.8.3.1 06/10/2014 GIT-TAR-TREE(1)