Move files from one directory to another in chunks
All,
I have an application that is not working properly and the company is 'in the process' of fixing it. In the meantime, I want to write a bash script work-around. However, what I thought was going to be simple is seemingly not.
Need:
- Move files from one directory to another in chunks.
If I move all the files at once the system resources are heavily consumed and the application suffers. The source directory ($dir1) will contain any number of files (100's or 1000's) at any given time. So, I'd like to do something like:
I know I can accomplish what I need with a brute force creation of many files to read from but there must be a more sophisticated way to do it.
Is there way to drop the list of files into an array and do something like:
I need to find a usable number of files to move at a time (could be 100, 5000, 10000) while not knowing how many files could be in $dir1 at any given time.
Any guidance is greatly appreciated.
TIA,
Herb
Last edited by rbatte1; 08-22-2017 at 06:26 AM..
Reason: Added CODE tags
Is this possible? Let me know If I need specify further on what I am trying to do- I just want to spare you the boring details of my personal file management.
Thanks in advance-
Brian- (2 Replies)
Hi All,
I am coding for a requirement where I need to move files (filename.yymmdd) from one directory(A) to another(B) based on 2 date fields in a paramtere file. (Paramfile.txt)
For e.g: In Paramfile.txt,
BUS_DT =20120612
SUB_DT =20120602
In this case, i need to move all the files... (14 Replies)
Hi,
need to zip all files in a directory and move to another directory after the zip..
i am using this one but didnt help me...
zip -r my_proj_`date +%Y%m%d%H%MS`.zip /path/my_proj
mv in_proj_`date +%Y%m%d%H%M%S`.zip /path/source/
i am trying to zip all the files in my_proj... (0 Replies)
I have a directory e2e_ms_xfer/cent01
this contains the multiple files some of which will be named below with unique date time stamps
e2e_ms_edd_nom_CCYYMMDD_HHMM.csv
What I want to do is in a loop
1) Get the oldest file
2) Rename
3) Move it up one level from e2e_ms_xfer/cent01 to... (1 Reply)
I have a directory e2e_ms_xfer/cent01
this contains the multiple files some of which will be named below with unique date time stamps
e2e_ms_edd_nom_CCYYMMDD_HHMM.csv
What I want to do is in a loop
1) Get the oldest file
2) Rename
3) Move it up one level from e2e_ms_xfer/cent01 to... (1 Reply)
Hi all,
i have a folder, with tons of files containing as following,
on /my/folder/jobs/
some_name_2016-01-17-22-38-58_some name_0_0.zip.done
some_name_2016-01-17-22-40-30_some name_0_0.zip.done
some_name_2016-01-17-22-48-50_some name_0_0.zip.done
and these can be lots of similar files,... (6 Replies)
I want to move all the files in a given directory up one level.
For example:
Dir1
Subdir1
I want to move all the files in Subdir1 up to Dir1 (then I want to ultimately delete Subdir1)
Thanks,
Ted (10 Replies)
Hi All,
Daily i am doing the house keeping in one of my server and manually moving the files which were older than 90 days and moving to destination folder.
using the find command . Could you please assist me how to put the automation using the shell script .
... (11 Replies)
Hi All,
We have main directory called "head"
under this we have several sub directories and under these directories we have sub directories.
My requirement is I have to find the SQL files which are having the string "procedure" under "head" directory and sub directories as well.
And create... (14 Replies)
Discussion started by: ROCK_PLSQL
14 Replies
LEARN ABOUT DEBIAN
raidfile-config
RAIDFILE-CONFIG(8) Box Backup RAIDFILE-CONFIG(8)NAME
raidfile-config - Configure Box Backup's RAID files
SYNOPSIS
raidfile-config config-dir blocksize dir1 [dir2 [dir3]]
DESCRIPTION
raidfile-config creates a raidfile.conf file for Box Backup. This file holds information about the directories used to store backups in.
Box Backup supports userland RAID, in a restricted RAID5 configuration, where 3 and only 3 'drives' are supported. You can read more about
RAID5 (and other RAID-levels) here[1].
Parameters
The parameters are as follows:
config-dir
The directory path where configuration files are located. Usually this is /etc/box. raidfile.conf will be written in this directory.
blocksize
The block size used for file storage in the system, in bytes. Using a multiple of the file system block size is a good strategy.
Depending on the size of the files you will be backing up, this multiple varies. Of course it also depends on the native block size of
your file system.
dir1
The first directory in the built-in RAID array.
dir2
The second directory in the built-in RAID array. If you are not using the built-in RAID functionality, this field should be ignored.
You should not use the built-in RAID if you have a hardware RAID solution or if you're using another type of software RAID (like md on
Linux).
dir3
The third directory in the built-in RAID array. The same notes that apply to dir2 also apply to dir3.
Note that there are currently no way to add multiple disk sets to the raidfile.conf file using command line tools, etc. See
raidfile.conf(5) for details on adding more disks.
BUGS
If you find a bug in Box Backup, and you want to let us know about it, join the mailing list[2], and send a description of the problem
there.
To report a bug, give us at least the following information:
o The version of Box Backup you are running
o The platform you are running on (hardware and OS), for both client and server.
o If possible attach your config files (bbstored.conf, bbackupd.conf) to the bug report.
o Also attach any log file output that helps shed light on the problem you are seeing.
o And last but certainly not least, a description of what you are seeing, in as much detail as possible.
FILES
raidfile-config generates the raidfile.conf(5) file.
SEE ALSO bbstored-config(8), bbstored.conf(5), raidfile.conf(5)AUTHORS
Ben Summers
Per Thomsen
James O'Gorman
NOTES
1. here
http://en.wikipedia.org/wiki/Redundant_array_of_independent_disks#RAID_5
2. mailing list
http://lists.warhead.org.uk/mailman/listinfo/boxbackup
Box Backup 0.11 10/28/2011 RAIDFILE-CONFIG(8)