Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Copying files
Top Forums Shell Programming and Scripting Copying files Post 302893883 by Corona688 on Friday 21st of March 2014 12:21:44 PM
Old 03-21-2014
Quote:
Originally Posted by hburnswell
The files will be used as a one time read of the txt files. There will not be ongoing reads/writes.
That is not the problem, though:
Quote:
Originally Posted by Corona688
It's a very bad idea to put 90,000 files in one directory. Creating or deleting files inside that folder will become very slow.
This could happen during the move, because moving a file counts. Anything which alters the output of 'ls' is altering the directory entry, which is what slows down. Moving the first few thousand files will happen lickety-split, but it will quickly bog down. It could take hours or days to move them all. That's why those files have been split into a tree of folders, to avoid this. All for an operation you say you're going to be doing what -- once? What will you do with them after -- leave them sitting forever, or delete them? Deleting that massive pile of files will take a similarly awful amount of time.

Strangely, "ongoing reads/writes" usually aren't a problem. Creating one file isn't so bad... Opening one file isn't so bad... Which is how people end up with folders full of 90,000 files -- something logging or recording for years and years without cease in the same place. Someday they need to delete them all, and find themselves in big trouble. We regularly get threads like this.

So, let me repeat my question:

Quote:
What is your actual goal here? I don't mean "moving the files". Why are you moving the files? If we can find a way to accomplish your ask without dumping 90,000 files in one directory you'll be much better off.
Please tell us why you think you need 90,000 files in one place. What could you do with them all in one place, that you can't do now?
Last edited by Corona688; 03-21-2014 at 01:39 PM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Copying files

I like to know the command structure of copying files/directories from a unix box using telnet session to a windows box. (4 Replies)
Discussion started by: alpheusm
4 Replies

2. Solaris

Copying Files and

I am new user to solaris and installed solaris operating system on full Harddisk 120Gb. I am unable to copy music files to desktop and /home directory. One thing happened while registering is- i entered login-root and its password. The message prompted your system is crashed. Is it because of... (1 Reply)
Discussion started by: patilmukundraj
1 Replies

3. Shell Programming and Scripting

copying files

hi I want to copy all files from the current directory and move to .archive file. Moreover,I want to add .bak to each file name, that will be copied. How can I do that? (4 Replies)
Discussion started by: tjay83
4 Replies

4. Solaris

Copying Files

Hi, I understand that to copy files across server, the feasible way will be using scp command. Am I right? What if the two servers are not connected to a network? If by using a cross cable to link up both the server, what will be the best (fastest) way to copy files across? scp as well? ... (3 Replies)
Discussion started by: user50210
3 Replies

5. UNIX for Advanced & Expert Users

copying of files by userB, dir & files owned by userA

I am userB and have a dir /temp1 This dir is owned by me. How do I recursively copy files from another users's dir userA? I need to preserve the original user who created files, original group information, original create date, mod date etc. I tried cp -pr /home/userA/* . ... (2 Replies)
Discussion started by: Hangman2
2 Replies

6. Shell Programming and Scripting

Copying Files

Hi All, I'm trying to list some files from my log directory and files are like this log.20110302_20.gz log.20110302_21.gz log.20110302_22.gz log.20110302_23.gz log.20110303_00.gz log.20110303_01.gz log.20110303_02.gz ............ log.20110311_22.gz log.20110311_23.gz... (2 Replies)
Discussion started by: thelakbe
2 Replies

7. Shell Programming and Scripting

Files copying - [ Listed files alone. ] - Shell script

Hi All, I am doing this for svn patch making. I got the list of files to make the patch. I have the list in a file with path of all the files. To Do From Directory : /myproject/MainDir To Directory : /myproject/data List of files need to copy is in the file: /myproject/filesList.txt ... (4 Replies)
Discussion started by: linuxadmin
4 Replies

8. Shell Programming and Scripting

Copying files after result

Hi, I have a shell script #!/bin/sh date echo 'HI PROD' echo $Please ENTER THE INPUT 1 for old files 2 for new file read i if ; then cd /apps/acetp3_logs/prod3/O* pwd echo $PLEASE ENTER THE STRING TO SEARCH (PLEASE ENTER THE STRING INSIDE QUOTES ' ') read j echo... (6 Replies)
Discussion started by: thelakbe
6 Replies

9. Shell Programming and Scripting

Copying files excluding some files

Hi, I have a folder which contains files in this format. abc-bin.000001 abc-bin.000002 abc-bin.000003 abc-bin.000004 abc-bin.000005 abc-bin.000006 abc-bin.000007 abc-bin.000008 abc-bin.000009 abc-bin.000010 abc-bin.index I want to copy all the files between abc-bin.000004... (6 Replies)
Discussion started by: arijitsaha
6 Replies

10. Shell Programming and Scripting

Copying files

I'm trying to do this exact same thing, so far I have created this to move files i've named my script CP.sh #!/bin/bash cd /root/my-documents/NewDir/ for f in *.doc do cp -v $f root/my-documents/NewDir $f{%.doc} done When i go to run this in the console i type, bin/sh/ CP.sh but it... (7 Replies)
Discussion started by: MKTM_93_SIMP
7 Replies

LEARN ABOUT FREEBSD

dh_movefiles

DH_MOVEFILES(1) 						     Debhelper							   DH_MOVEFILES(1)

NAME

       dh_movefiles - move files out of debian/tmp into subpackages

SYNOPSIS

       dh_movefiles [debhelperoptions] [--sourcedir=dir] [-Xitem] [file...]

DESCRIPTION

       dh_movefiles is a debhelper program that is responsible for moving files out of debian/tmp or some other directory and into other package
       build directories. This may be useful if your package has a Makefile that installs everything into debian/tmp, and you need to break that
       up into subpackages.

       Note: dh_install is a much better program, and you are recommended to use it instead of dh_movefiles.

FILES

       debian/package.files
	   Lists the files to be moved into a package, separated by whitespace. The filenames listed should be relative to debian/tmp/. You can
	   also list directory names, and the whole directory will be moved.

OPTIONS

       --sourcedir=dir
	   Instead of moving files out of debian/tmp (the default), this option makes it move files out of some other directory. Since the entire
	   contents of the sourcedir is moved, specifying something like --sourcedir=/ is very unsafe, so to prevent mistakes, the sourcedir must
	   be a relative filename; it cannot begin with a `/'.

       -Xitem, --exclude=item
	   Exclude files that contain item anywhere in their filename from being installed.

       file ...
	   Lists files to move. The filenames listed should be relative to debian/tmp/. You can also list directory names, and the whole directory
	   will be moved. It is an error to list files here unless you use -p, -i, or -a to tell dh_movefiles which subpackage to put them in.

NOTES

       Note that files are always moved out of debian/tmp by default (even if you have instructed debhelper to use a compatibility level higher
       than one, which does not otherwise use debian/tmp for anything at all). The idea behind this is that the package that is being built can be
       told to install into debian/tmp, and then files can be moved by dh_movefiles from that directory. Any files or directories that remain are
       ignored, and get deleted by dh_clean later.

SEE ALSO

       debhelper(7)

       This program is a part of debhelper.

AUTHOR

       Joey Hess <joeyh@debian.org>

11.1.6ubuntu2							    2018-05-10							   DH_MOVEFILES(1)
All times are GMT -4. The time now is 05:48 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy