Copying files automatically Post: 302430728

Sponsored Content

Top Forums UNIX for Dummies Questions & Answers Copying files automatically Post 302430728 by MrEddy on Friday 18th of June 2010 01:12:22 PM

06-18-2010

Registered User

Copying files automatically

Ok here is my issue. I have processes which will get screwed up and not run because certain log directories wind up filled too much and a directory will have 100,000 files in it. The only way to fix this is to move files out of this directory until the number of files is small enough that the system will clean it up. No more than 1000 or so files can be there.

This is a long process as I can't just do cp * because I get argument too long. I've been trying to do a

Code:

find * | head -1000 | xargs cp

type of command to simply spew out 1000 files and copy or move them to another folder. Then I could simply !! that until they are all moved. Or loop it somehow. But, I cannot get this to work I do not know what i'm doing wrong here with xargs.

Code:

find * -exec cp {} /tmp/testcopy \;

This does work except that there is no limiting the files there. So i'll just run into the same arguement list too long if the directory is crazy big. I can't limit them by filename or anything like that either. All the names are just crazy numbers with no real meaning. So the only way I can think to limit them si with tail or head. Something like that.

Any ideas would be welcome. I also can't put scripts on boxes or anything. So whatever command I run has to be something I can simply punch into the prompt and execute.

MrEddy

View Public Profile for MrEddy

Find all posts by MrEddy

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How can I automatically find important files???

how can I automatically check if important files exist in a directory and if not, automatically put the important files where they are needed say, I want to put .bashrc and a dozen other important files like it into every user's directory, how can I do this??? how do I check every user's...

2. UNIX for Dummies Questions & Answers

Automatically copying files from server to local drive?

I would like automate the process of copying some logs files from a server to my local hard drive at a set time each week/day. I don't really know anything about creating and scheduling jobs. Is this something I could setup relatively easily? I would like to automatically copy all the logs...

3. UNIX for Advanced & Expert Users

copying of files by userB, dir & files owned by userA

I am userB and have a dir /temp1 This dir is owned by me. How do I recursively copy files from another users's dir userA? I need to preserve the original user who created files, original group information, original create date, mod date etc. I tried cp -pr /home/userA/* . ...

4. Shell Programming and Scripting

Shell script to automatically download files

I am new to shell scripting and need to write a program to copy files that are posted as links on a specific url. I want all the links copied with the same file name and the one posted on the webpage containing the url onto a specific directory. That is the first part. The second part of the script...

5. Shell Programming and Scripting

Files copying - [ Listed files alone. ] - Shell script

Hi All, I am doing this for svn patch making. I got the list of files to make the patch. I have the list in a file with path of all the files. To Do From Directory : /myproject/MainDir To Directory : /myproject/data List of files need to copy is in the file: /myproject/filesList.txt ...

6. Shell Programming and Scripting

Copy files by automatically creating directories

I am in a situation where I have some hundreds of thousands of files in a directory (name of directory: big_directory). Now, working on this directory is extremely slow. I now plan to do this: Copy 1000 files in each of the directories by creating the directories automatically. All files have...

7. OS X (Apple)

Moving Files Automatically

I'm looking to find/create a script so that when a download is complete, I can run the script in order for it to automatically move a file such as... 'example.avi' into my videos folder I'm a novice when it comes to scripting, any advice/help would be greatly appreciated! Thanks Andrew

8. Shell Programming and Scripting

Delete log files automatically.

Here is the script I want to run to deleted log files after a certain time: touch /usr/WebSphere/AppServer/profiles/AppSrv01/apps/RSA/logs find /usr/WebSphere/AppServer/profiles/AppSrv01/apps/RSA/logs -atime +120 - exec rm -rf {}\; Exerytime I run it, it throws me the error: find: paths must...

9. Shell Programming and Scripting

Convert a lot of files in subdirectories automatically

Hi, I have a huge structure of directories and subdirectories contsining some data. The lowest folders contain a file "image.png" which need to be converted to "folder.jpg". But how can I do that for all these files automatically? That's what I alredy have find /path -type f -name...

LEARN ABOUT DEBIAN

dh_movefiles

DH_MOVEFILES(1) 						     Debhelper							   DH_MOVEFILES(1)

NAME

       dh_movefiles - move files out of debian/tmp into subpackages

SYNOPSIS

       dh_movefiles [debhelperoptions] [--sourcedir=dir] [-Xitem] file...]

DESCRIPTION

       dh_movefiles is a debhelper program that is responsible for moving files out of debian/tmp or some other directory and into other package
       build directories. This may be useful if your package has a Makefile that installs everything into debian/tmp, and you need to break that
       up into subpackages.

       Note: dh_install is a much better program, and you are recommended to use it instead of dh_movefiles.

FILES

       debian/package.files
	   Lists the files to be moved into a package, separated by whitespace. The filenames listed should be relative to debian/tmp/. You can
	   also list directory names, and the whole directory will be moved.

OPTIONS

       --sourcedir=dir
	   Instead of moving files out of debian/tmp (the default), this option makes it move files out of some other directory. Since the entire
	   contents of the sourcedir is moved, specifying something like --sourcedir=/ is very unsafe, so to prevent mistakes, the sourcedir must
	   be a relative filename; it cannot begin with a `/'.

       -Xitem, --exclude=item
	   Exclude files that contain item anywhere in their filename from being installed.

       file ...
	   Lists files to move. The filenames listed should be relative to debian/tmp/. You can also list directory names, and the whole directory
	   will be moved. It is an error to list files here unless you use -p, -i, or -a to tell dh_movefiles which subpackage to put them in.

NOTES

       Note that files are always moved out of debian/tmp by default (even if you have instructed debhelper to use a compatibility level higher
       than one, which does not otherwise use debian/tmp for anything at all). The idea behind this is that the package that is being built can be
       told to install into debian/tmp, and then files can be moved by dh_movefiles from that directory. Any files or directories that remain are
       ignored, and get deleted by dh_clean later.

SEE ALSO

       debhelper(7)

       This program is a part of debhelper.

AUTHOR

       Joey Hess <joeyh@debian.org>

9.20120909							    2012-05-19							   DH_MOVEFILES(1)