Fast processing(mv command) of 1 million+ files using find, mv and xargs Post: 302789043

9 More Discussions You Might Find Interesting

1. AIX

command usage on find with xargs and tar

my task : tar up large bunch of files(about 10,000 files) in the current directories that created more than 30 days ago but it come with following error find ./ -ctime +30 | xargs tar rvf test1.tar tar: test1.tar: A file or directory in the path name does not exist.

2. UNIX for Dummies Questions & Answers

use of xargs and prune piping with find command.

Can anyone interpret and tell me the way the below command works? find * -name "*${msgType}" -mtime +${archiveDays} -prune -type f -print 2>/dev/null | xargs rm -f 2> /dev/null Please tell me the usage of prune and xargs in the above command? Looking forward your reply. Thanks in...

3. Shell Programming and Scripting

find with xargs to rm found files

I believe what is happening is rm is executing in the script on every directory and on failure of the first it stops although returns status 0. find $HOME -name /directory/filename | xargs -l rm This is the code I use but file remains. I am using sun solaris system which has way limited...

4. Solaris

Need to know command to delete more than 3 million files from /var/spool/clientmqueue

Hi I need to delete more than 3 million files from /var/spool/clientmqueue. When I give the following command to delete the files, I get the error # pwd /var/spool/clientmqueue # rm -f * /usr/bin/rm: arg list too long Please tell me how can I delete the files

5. Shell Programming and Scripting

help using find/xargs to apply mp3gain to files

I need to apply mp3gain (album mode) to all mp3 files in a given directory. Each album is in its own directory under /media/data/music/albums for example: /media/data/music/albums/foo /media/data/music/albums/bar /media/data/music/albums/more What needs to happen is: cd...

6. Shell Programming and Scripting

find numeric duplicates from 300 million lines....

these are numeric ids.. 222932017099186177 222932014385467392 222932017371820032 222932017409556480 I have text file having 300 millions of line as shown above. I want to find duplicates from this file. Please suggest the quicker way.. sort | uniq -d will...

7. UNIX for Dummies Questions & Answers

Deleting a million of files ..

Hi, Which way is faster rm -rf /path/ or find / -name -exec rm {} \; and why?

8. Shell Programming and Scripting

Parallel processing for functions in xargs

I have a script (ksh) which tries to run a function in parallel for performance gains. I am also trying to limit the number of parallel child processes to avoid overloading the system by using a variable to count triggered processes and waiting for completion e.g. do_something () { ... } ...

9. UNIX for Advanced & Expert Users

Zip million files taking 12 hours or more

Hi I have task to zip files based on modified time but they are in millions and it is taking lot of time more than 12 hours and also eating up high cpu is there any other / better way to handle it quickly with less cpu consumptionfind . ! -name \"*.gz\" -mtime +7 -type f | grep -v '/.*/' |...

LEARN ABOUT DEBIAN

dpkg-name

dpkg-name(1)							  dpkg utilities						      dpkg-name(1)

NAME

       dpkg-name - rename Debian packages to full package names

SYNOPSIS

       dpkg-name [option...]  [--] file...

DESCRIPTION

       This manual page documents the dpkg-name program which provides an easy way to rename Debian packages into their full package names. A full
       package name consists of package_version_architecture.package-type as specified in the control file of the package. The version part of the
       filename  consists  of the upstream version information optionally followed by a hyphen and the revision information. The package-type part
       comes from that field if present or fallbacks to deb.

OPTIONS

       -a, --no-architecture
	      The destination filename will not have the architecture information.

       -k, --symlink
	      Create a symlink, instead of moving.

       -o, --overwrite
	      Existing files will be overwritten if they have the same name as the destination filename.

       -s, --subdir [dir]
	      Files will be moved into a subdirectory. If the directory given as argument exists the files will be moved into that directory  oth-
	      erwise the name of the target directory is extracted from the section field in the control part of the package. The target directory
	      will be `unstable/binary-architecture/section'. If the section is not found in the control, then `no-section'  is  assumed,  and	in
	      this case, as well as for sections `non-free' and `contrib' the target directory is `section/binary-architecture'. The section field
	      isn't required so a lot of packages will find their way to the `no-section' area. Use this option with care, it's messy.

       -c, --create-dir
	      This option can used together with the -s option. If a target directory isn't found it will  be  created	automatically.	 Use  this
	      option with care.

       -?, --help
	      Show the usage message and exit.

       -v, --version
	      Show the version and exit.

EXAMPLES

       dpkg-name bar-foo.deb
	      The  file  `bar-foo.deb' will be renamed to bar-foo_1.0-2_i386.deb or something similar (depending on whatever information is in the
	      control part of `bar-foo.deb').

       find /root/debian/ -name '*.deb' | xargs -n 1 dpkg-name -a
	      All files with the extension `deb' in the directory /root/debian and its subdirectory's will be renamed  by  dpkg-name  if  required
	      into names with no architecture information.

       find -name '*.deb' | xargs -n 1 dpkg-name -a -o -s -c
	      Don't  do  this.	Your archive will be messed up completely because a lot of packages don't come with section information.  Don't do
	      this.

       dpkg-deb --build debian-tmp && dpkg-name -o -s .. debian-tmp.deb
	      This can be used when building new packages.

BUGS

       Some packages don't follow the name structure package_version_architecture.deb.	Packages renamed by dpkg-name will follow this	structure.
       Generally  this	will have no impact on how packages are installed by dselect(1)/dpkg(1), but other installation tools might depend on this
       naming structure.

SEE ALSO

       deb(5), deb-control(5), dpkg(1), dpkg-deb(1), find(1), xargs(1).

Debian Project							    2012-04-15							      dpkg-name(1)