Sponsored Content
Top Forums Shell Programming and Scripting Loop to move files in different directories Post 302460290 by DGPickett on Wednesday 6th of October 2010 11:20:29 AM
Old 10-06-2010
BTW: I hardly ever cd, even in scripts, as I found it hindered my productivity and led to errors. The problem with cd is then, commands are not reusable. The cd is really a typing shorthand. Effectively, it is in competition with command recall. Absolute paths are not much of a burden if they are not typed. UNIX ksh life was more consistent, error-free, better if I used X cut/paste, command recall & editing: set -o vi/viraw w/export HISTSIZE=32767, even archiving .sh_history occasionally for my tools that recall history. I even wrote a vi wrapper so my xterm scroll history (also set real big) was not overwritten and the return is always zero (so results are not discarded). Since I never leave $HOME, I can use relative paths for all common things, some through my own sym-links, and absolute for things less frequent. All my history is rerunnable. If it needs cd, I do: (cd ... ; .... )

---------- Post updated at 11:20 AM ---------- Previous update was at 11:12 AM ----------

My solution moves one dir at a time, not one file, which might be a bit faster and lower overhead.

It also ensures the target is present. It does not check to see if the source files are present, but that is not worth scripting or easy to script cheaply.

If you get a huge dir, when someone allows too many files to expand the directory inode, the speed difference is very substantial. With my plan, you are going to traverse that directory:
  1. once in find to find the archive/,
  2. once in ksh to find the source files for the mv command line, and then
  3. once (for every file?) inside mv to find the archive\
  4. once for every file inside mv, but just far enough through the directory to find that file and overwrite that part of the directory.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

grep'ing for specific directories, and using the output to move files

Hello, this is probably another really simple tasks for most of you gurus, however I am trying to make a script which takes an input, greps a specific file for that input, prints back to screen the results (which are directory names) and then be able to use the directory names to move files.... (1 Reply)
Discussion started by: JayC89
1 Replies

2. Shell Programming and Scripting

How to loop through directories to touch files

Hi, Please help me on this. Suppose i have the following directory structure. /app/data /app/data/eng /app/data/med /app/data/bsc each of the directories data,data/eng,data/med,data/bsc holds files with date extension like a.20081230 b.20081230 and so on I need a script to loop... (9 Replies)
Discussion started by: sussane
9 Replies

3. UNIX for Dummies Questions & Answers

want to move files in a dir into different directories based on the filename

I want to move the files in a dir to different dirs based on their file names. Ex: i have 4 different files with name - CTS_NONE_10476031_MRL_PFT20081215a.txt CTS_NONE_10633009_MRL_PFT20091020a.txt CTS_NONE_10345673_MRL_PFT20081215a.txt CTS_NONE_10872456_MRL_PFT20091020a.txt and the 1st... (2 Replies)
Discussion started by: Sriranga
2 Replies

4. Shell Programming and Scripting

want to move files in a dir into different directories based on the filename

I want to move the files in a dir to different dirs based on their file names. Ex: i have 4 different files with name - CTS_NONE_10476031_MRL_PFT20081215a.txt CTS_NONE_10633009_MRL_PFT20091020a.txt CTS_NONE_10345673_MRL_PFT20081215a.txt CTS_NONE_10872456_MRL_PFT20091020a.txt and the 1st... (4 Replies)
Discussion started by: Sriranga
4 Replies

5. Shell Programming and Scripting

Help with command to Move files by X number to seperate directories

Hello, I need help finding a script that will allow me to move files from one directory to another directory 10k files at a time. I have a directory that has 100 K files in it. I need to have those 100k files broken apart to separate directories each with 10k files in them. Here is the... (8 Replies)
Discussion started by: Geo_Bean
8 Replies

6. OS X (Apple)

Batch file to move video files and retain sub-directories

I have just purchased my first ever Apple computer - and am therefore new to UNIX also. I would like to create a simple "batch file" (apologies if this is the wrong terminology) to do the following: When I plug my camera into the MAC it automatically downloads photos and videos into a new... (1 Reply)
Discussion started by: mm0mss
1 Replies

7. Shell Programming and Scripting

Recursively move directories along with files/specific files

I would like to transfer all files ending with .log from /tmp and to /tmp/archive (using find ) The directory structure looks like :- /tmp a.log b.log c.log /abcd d.log e.log When I tried the following command , it movies all the log files... (8 Replies)
Discussion started by: frintocf
8 Replies

8. UNIX for Dummies Questions & Answers

Move multipe files to corresponding directories

Hi, In a parent directory there are several files in the form IDENTIFIER1x IDENTIFIER1.yyy IDENTIFIER1_Z, etc IDENTIFIER2x IDENTIFIER2.yyy IDENTIFIER2_Z, etc IDENTIFIER3x IDENTIFIER3.yyy, IDENTIFIER3_Z, etcIn the same parent directory there are corresponding directories named... (7 Replies)
Discussion started by: spirospap
7 Replies

9. Shell Programming and Scripting

Need BASH Script Help to Move Files While Creating Directories

I've got this script to loop through all folders and move files that are more than 2 years old. I'm using the install command because it creates the necessary directories on the destination path and then I remove the source. I'd like to change the script to use the mv command since it is much... (4 Replies)
Discussion started by: consultant
4 Replies

10. UNIX for Beginners Questions & Answers

Move several files into specific directories with a loop

Hello, I'm a first time poster looking for help in scripting a task in my daily routine. I am new in unix but i am attracted to its use as a mac user. Bear with me... I have several files (20) that I manually drag via the mouse into several named directories over a network. I've used rsync... (14 Replies)
Discussion started by: SonnyClark
14 Replies
MAKEHISTORY(8)						      System Manager's Manual						    MAKEHISTORY(8)

NAME
makehistory - tools to recover Usenet history database. SYNOPSIS
makehistory [ -A oldtmp ] [ -a active ] [ -b ] [ -f filename ] [ -i ] [ -n ] [ -o ] [ -r ] [ -s size ] [ -T tmpdir ] [ -u [ -v ] ] DESCRIPTION
Makehistory rebuilds the history(5) text file and the associated dbz(3) database. The default name of the text file is /var/lib/news/his- tory; to specify a different name, use the ``-f'' flag. Makehistory scans the active(5) file to determine which newsgroup directories within the spool directory, /var/spool/news, should be scanned. (If a group is removed, but its spool directory still exists, makehistory will ignore it.) The program reads each file found and writes a history line for it. After the text file is written, makehistory will build the dbz database. OPTIONS
-A If the ``-A'' flag is used then the argument given is the pathname makehistory can use to store a copy of the history file as it's being built. It will be appended to, so existing data will not be lost (and so should be valid history entries). -a If the ``-a'' flag is given then the argument is the active file to use rather than the default one of /var/lib/news/active. -b If the ``-b'' flag is used, then makehistory will remove any articles that do not have valid Message-ID headers in them. -f If the ``-f'' flag is used, then the database files are named file.dir and file.pag. If the ``-f'' flag is not used, then a tempo- rary link to the name history.n is made and the database files are written as history.n.pag and history.n.dir. -o If the ``-o'' flag is used, then the link is not made and any existing history files are overwritten. If the old database exists, makehistory will use it to determine the size of the new database. -i To ignore the old database use the ``-i'' flag. Using the ``-o'' flag implies the ``-i'' flag. -s The program will also ignore any old database if the ``-s'' flag is used to specify the approximate number of entries in the new database. Accurately specifying the size is an optimization that will create a more efficient database. (The size should be the estimated eventual size of the file, typically the size of the old file.) For more information, see the discussion of dbzfresh and dbzsize in dbz(3). -u If the ``-u'' flag is given, then makehistory assumes that innd is running. It will pause the server while scanning, and then send ``addhist'' commands (see ctlinnd(8)) to the server for any article that is not found in the dbz database. The command ``makehis- tory -bu'' is useful after a system crash, to delete any mangled articles and bring the article database back into a more consistent state. -v If the ``-v'' flag is used with the ``-u'' flag, then makehistory will put a copy of all added lines on its standard output. -n To scan the spool directory without rebuilding the dbz files, use the ``-n'' flag. If used with ``-u'', the server will not be paused while scanning. -r To just build the dbz files from an existing text file, use the ``-r'' flag. The ``-i'' or ``-s'' flags can be useful if there are no valid dbz files to use. -T Makehistory needs to create a temporary file that contains one line for each article it finds, which can become very large. This file is created in the /var/spool/news/in.coming/tmp directory. The ``TMPDIR'' environment variable may be used to specify a dif- ferent directory. Alternatively, the ``-T'' flag may be used to specify a temporary directory. In addition, the sort(1) that is invoked during the build writes large temporary files (often to /var/tmp but see your system manpages). If the ``-T'' flag is used, then the flag and its value will be passed to sort. On most systems this will change the temporary directory that sort uses. if used, this flag and its value will be passed on to the sort(1) command that is invoked during the build. EXAMPLES
A typical way to use this program is with the following /bin/sh commands: ctlinnd throttle "Rebuilding history file" cd /var/lib/news if makehistory -n -f history.n ; then : else echo Error creating history file! exit 1 fi # The following line can be used to retain expired history # It is not necessary for the history file to be sorted. # awk 'NF==2 { print; }' <history >>history.n # View history file for mistakes. if makehistory -r -s `wc -l <history` -f history.n; then mv history.n history mv history.n.dir history.dir mv history.n.pag history.pag fi ctlinnd go '' BUGS AND LIMITATIONS
Makehistory does not handle symbolic links. If the news spool area is split across multiple partitions, the following commands should probably be run before the database is regenerated: cd /var/spool/news find . -type l -name '[1-9]*' -print | xargs -t rm Make sure to run the command on all the appropriate partitions! HISTORY
Written by Rich $alz <rsalz@uunet.uu.net> for InterNetNews. This is revision 1.3, dated 1996/11/26. SEE ALSO
active(5), ctlinnd(8), dbz(3), filechan(8), history(5), innd(8), newsfeeds(5), makeactive(8), newsrequeue(8). MAKEHISTORY(8)
All times are GMT -4. The time now is 04:39 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy