Sponsored Content
Full Discussion: moving large number of files
Top Forums Shell Programming and Scripting moving large number of files Post 302127840 by radoulov on Thursday 19th of July 2007 07:07:49 PM
Old 07-19-2007
Quote:
Originally Posted by matrixmadhan
I doubt this command, because OP had said 35000 files and more.

What if the number of command line arguments to ' mv ' exceeds the permissible limit ?

Its quite possible ! Smilie
Didn't think about that Smilie

So, with recent zsh:

Code:
setopt nomatch
while set -- *.dat([1]);do
mv -- *.dat([1,1000]) test
done

Adjust 1000 for your limit.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to Compare a large number of files.

I have a large Filesystem on an AIX server and another one on a Red Hat box. I have syncd the two filesystems using rsysnc. What Im looking for is a script that would compare to the two filesystems to make sure the bits match up and the number of files match up. its around 2.8 million... (5 Replies)
Discussion started by: zippdawg2001
5 Replies

2. Shell Programming and Scripting

Moving extremely large number of files to destination

Hello Techies, m here with a small issue. Have read out all the relavent threads on unix.com but it was not so good sol. for me. It's a simple & known issue. I want to move lots of files to dest folder. Actually I want to pick up 1 year older files, but that is even taking lots of... (16 Replies)
Discussion started by: kedar.mehta
16 Replies

3. Shell Programming and Scripting

Need help combining large number of text files

Hi, i have more than 1000 data files(.txt) like this first file format: 178.83 554.545 179.21 80.392 second file: 178.83 990.909 179.21 90.196 etc. I want to combine them to the following format: 178.83,554.545,990.909,... 179.21,80.392,90.196,... (7 Replies)
Discussion started by: mr_monocyte
7 Replies

4. UNIX for Dummies Questions & Answers

questing regarding tar large number of files

I want to tar large number of files about 150k. i am using the find command as below to create a file with all file names. & then trying to use the tar -I command as below. # find . -type f -name "gpi*" > include-file # tar -I include-file -cvf newfile.tar This i got from one of the posts... (2 Replies)
Discussion started by: crux123
2 Replies

5. Shell Programming and Scripting

Concatenation of a large number of files

Hellow i have a large number of files that i want to concatenate to one. these files start with the word 'VOICE_' for example VOICE_0000000000 VOICE_1223o23u0 VOICE_934934927349 I use the following code: cat /ODS/prepaid/CDR_FLOW/MEDIATION/VOICE_* >> /ODS/prepaid/CDR_FLOW/WORK/VOICE ... (10 Replies)
Discussion started by: chriss_58
10 Replies

6. UNIX for Dummies Questions & Answers

Delete large number of files

Hi. I need to delete a large number of files listed in a txt file. There are over 90000 files in the list. Some of the directory names and some of the file names do have spaces in them. In the file, each line is a full path to a file: /path/to/the files/file1 /path/to/some other/files/file 2... (4 Replies)
Discussion started by: inakajin
4 Replies

7. Shell Programming and Scripting

Moving and renaming large ammount of files

Hey, I'm kinda new to the shell scripting and I don't wanna mess things up yet :) Looking for a solution to the following: I need to move all the files like "filename.failed.dateandtime" to another directory also renaming them "filename.ready". I can't figure how to do this with multiple files... (4 Replies)
Discussion started by: sg3
4 Replies

8. UNIX for Dummies Questions & Answers

Rename a large number of files in subdirectories

Hi, I have a large number of subdirectories (>200), and in each of these directories there is a file with a name like "opp1234.dat". I'd like to know how I could change the names of these files to say "out.dat" in all these subdirectories in one go. Thanks! (5 Replies)
Discussion started by: lost.identity
5 Replies

9. Shell Programming and Scripting

Sftp large number of files

Want to sftp large number of files ... approx 150 files will come to server every minute. (AIX box) Also need make sure file has been sftped successfully... Please let me know : 1. What is the best / faster way to transfer files? 2. should I use batch option -b so that connectivity will be... (3 Replies)
Discussion started by: vegasluxor
3 Replies

10. Shell Programming and Scripting

Removing large number of temp files

Hi All, I am having a situation now to delete a huge number of temp files created during run times approx. 16700+ files. We have never imagined that we will get this this much big list of files during run time. It worked fine for lesser no of files in the list. But when list is huge we are... (7 Replies)
Discussion started by: mad man
7 Replies
cpio(1) 						      General Commands Manual							   cpio(1)

Name
       cpio - copy file archives in and out

Syntax
       cpio -i [ -C ] [keys] [patterns]

       cpio -o [keys]

       cpio -p [keys] directory

Description
       The  command  is  a filter designed to let you copy files to or from an archive.  The command differs from the command in that lets you ar-
       chive any kind of file, while is limited to program object files.

Options
       -i   Copies files that match the specified pattern.  If the pattern is not specified, copies in all files.  Extracts files from	the  stan-
	    dard  input,  which  is  assumed to be the product of a previous cpio -o, and places them into the user's current directory tree.  For
	    files with the same name, the newer file replaces the older file unless the -u option is used.

	    Only files with names that match patterns are selected.  The patterns are specified using the notation for names described in In  pat-
	    terns,  the  slash	for  directories (/) is included in searches using meta-characters.  For example, suppose the archive contains the
	    file and the pathname information in the archive indicates that the directory below contains the file This command copies  both  files
	    into the user's current directory:
	    cpio -i *p < /dev/rmt0l

	    Multiple  patterns	may be specified and if no patterns are specified, the default for patterns is * (that is, select all files).  The
	    extracted files are conditionally created and copied into the current directory tree based upon the options described below.  The com-
	    mand has three function keys, each with its own set of options.

       -C   Specifies  compatability mode for reading archives created with the old style header format.  Use this option to copy in archives gen-
	    erated by pre-V4.2 This option is valid only for copy-in (-i) operations.

       -o   Copies out the specified files.  Reads the standard input to obtain a list of path names and copies those files onto the standard out-
	    put together with path name and status information.

       -p   Copies  files  into  the specified destination directory, which must already exist.  Reads the standard input to obtain a list of path
	    names of files that are conditionally created.  This list of files is copied into  the  destination  directory  tree  based  upon  the
	    options used.  For files with the same name, the newer file replaces the older file unless the -u option is used.

Keys
       6    Processes a file with the UNIX System Sixth Edition format.

       a    Retains  original  access  times  of  input  files, and can be used with -o and -p.  Normally, the read(s) used in the copy update the
	    copied file's access time.

       B    Determines input/output is to be blocked 5,120 bytes to the record.  This option is meaningful only with data directed to or from or

       b    Swaps both bytes and halfwords.

       c    Creates header information in ASCII format and can be used with -i and -o.

       d    Creates subdirectories, as needed, below the specified destination directory.

       f    Copies all files except those that match the specified pattern.

       k    Enables symbolic link handling and is used with the -i, -o, and -p options.

       l    Creates links wherever possible.

       m    Retains modification time for each copied file.  This option does not work on directories or symbolic links that are being copied; the
	    directory is always reset to show the access time when the copy was made.

       r    Interactively renames files.  If you respond with a null line, the file is skipped (not copied).  Use only with the -i option.

       s    Swaps bytes while copying files in.

       S    Swaps half words while copying files in.

       t    Prints a table of contents of the input (no files are created).

       u    Copies files unconditionally.  (Otherwise, an older file will not replace a newer file with the same name).

       v    Displays  detailed	(verbose)  information as it copies and/or creates file.  When used with the t option, the table of contents looks
	    like the output of an ls -l command.  For further information, see

Examples
       This example shows how to copy the contents of the user's current directory into an archive.
       ls | cpio -o  > /dev/rmt0l
       This example shows how to duplicate a directory hierarchy.
       mkdir ~phares/newdir
       cd ~phares/olddir
       find . -print | cpio -pdl ~phares/newdir
       This example shows how to copy all files and directories with names containing the characters "chapter" in user smith's home directory  and
       underlying directories.
       find ~smith -name '*chapter*' -print | cpio -o > /dev/rmt0h
       This example shows the results of using the r option with the -i function key.
       ls | cpio -ir > ~smith/newdir
       Rename <file1>
       newnamefile1
       Rename <file2>
       <RETURN>
       Skipped
       Rename <file3>
       newnamefile3
       In  some  cases,  the  -cpio  option of the command can be used more effectively than pipes and redirects using For instance, the following
       example
       find . -print | cpio -oB > /dev/rmt0l
       can be handled more efficiently by:
       find . -cpio /dev/rmt0l
       To copy the contents of a directory (with symbolic link handling enabled) to the tape drive, type:
       ls | cpio -ok > /dev/rmt0h
       To restore the archived files back into a directory, type:
       cpio -ik < /dev/rmt0h
       The following example moves files, including symbolic links, from an old directory to a new directory:
       mkdir ~craig/newdir
       cd ~craig/olddir
       ls | cpio -pdk ~craig/newdir

Restrictions
       Pathnames are restricted to 128 characters.

       When there are too many unique linked files, the program runs out of memory and cannot trace them.  In this case,  linking  information	is
       lost.

       Only the superuser can copy special files.

See Also
       ar(1), find(1), cpio(5)

																	   cpio(1)
All times are GMT -4. The time now is 10:50 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy