Sponsored Content
Full Discussion: moving large number of files
Top Forums Shell Programming and Scripting moving large number of files Post 302127778 by kahuna on Thursday 19th of July 2007 12:43:07 PM
Old 07-19-2007
I don't know that this would be faster, but it should execute "mv" fewer times
(You'd have to figure out an appropriate number for -s)

Code:
xargs -s 1000 echo  < list | sed 's+^+mv +;s+$+ test/+' |sh -

If you have perl and the test directory is in the same partition, you might try
Code:
perl -nle 'rename "$_", "test/$_";' list

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to Compare a large number of files.

I have a large Filesystem on an AIX server and another one on a Red Hat box. I have syncd the two filesystems using rsysnc. What Im looking for is a script that would compare to the two filesystems to make sure the bits match up and the number of files match up. its around 2.8 million... (5 Replies)
Discussion started by: zippdawg2001
5 Replies

2. Shell Programming and Scripting

Moving extremely large number of files to destination

Hello Techies, m here with a small issue. Have read out all the relavent threads on unix.com but it was not so good sol. for me. It's a simple & known issue. I want to move lots of files to dest folder. Actually I want to pick up 1 year older files, but that is even taking lots of... (16 Replies)
Discussion started by: kedar.mehta
16 Replies

3. Shell Programming and Scripting

Need help combining large number of text files

Hi, i have more than 1000 data files(.txt) like this first file format: 178.83 554.545 179.21 80.392 second file: 178.83 990.909 179.21 90.196 etc. I want to combine them to the following format: 178.83,554.545,990.909,... 179.21,80.392,90.196,... (7 Replies)
Discussion started by: mr_monocyte
7 Replies

4. UNIX for Dummies Questions & Answers

questing regarding tar large number of files

I want to tar large number of files about 150k. i am using the find command as below to create a file with all file names. & then trying to use the tar -I command as below. # find . -type f -name "gpi*" > include-file # tar -I include-file -cvf newfile.tar This i got from one of the posts... (2 Replies)
Discussion started by: crux123
2 Replies

5. Shell Programming and Scripting

Concatenation of a large number of files

Hellow i have a large number of files that i want to concatenate to one. these files start with the word 'VOICE_' for example VOICE_0000000000 VOICE_1223o23u0 VOICE_934934927349 I use the following code: cat /ODS/prepaid/CDR_FLOW/MEDIATION/VOICE_* >> /ODS/prepaid/CDR_FLOW/WORK/VOICE ... (10 Replies)
Discussion started by: chriss_58
10 Replies

6. UNIX for Dummies Questions & Answers

Delete large number of files

Hi. I need to delete a large number of files listed in a txt file. There are over 90000 files in the list. Some of the directory names and some of the file names do have spaces in them. In the file, each line is a full path to a file: /path/to/the files/file1 /path/to/some other/files/file 2... (4 Replies)
Discussion started by: inakajin
4 Replies

7. Shell Programming and Scripting

Moving and renaming large ammount of files

Hey, I'm kinda new to the shell scripting and I don't wanna mess things up yet :) Looking for a solution to the following: I need to move all the files like "filename.failed.dateandtime" to another directory also renaming them "filename.ready". I can't figure how to do this with multiple files... (4 Replies)
Discussion started by: sg3
4 Replies

8. UNIX for Dummies Questions & Answers

Rename a large number of files in subdirectories

Hi, I have a large number of subdirectories (>200), and in each of these directories there is a file with a name like "opp1234.dat". I'd like to know how I could change the names of these files to say "out.dat" in all these subdirectories in one go. Thanks! (5 Replies)
Discussion started by: lost.identity
5 Replies

9. Shell Programming and Scripting

Sftp large number of files

Want to sftp large number of files ... approx 150 files will come to server every minute. (AIX box) Also need make sure file has been sftped successfully... Please let me know : 1. What is the best / faster way to transfer files? 2. should I use batch option -b so that connectivity will be... (3 Replies)
Discussion started by: vegasluxor
3 Replies

10. Shell Programming and Scripting

Removing large number of temp files

Hi All, I am having a situation now to delete a huge number of temp files created during run times approx. 16700+ files. We have never imagined that we will get this this much big list of files during run time. It worked fine for lesser no of files in the list. But when list is huge we are... (7 Replies)
Discussion started by: mad man
7 Replies
App::Cache(3pm) 					User Contributed Perl Documentation					   App::Cache(3pm)

NAME
App::Cache - Easy application-level caching SYNOPSIS
# in your class: my $cache = App::Cache->new({ ttl => 60*60 }); $cache->delete('test'); my $data = $cache->get('test'); my $code = $cache->get_code("code", sub { $self->calculate() }); my $html = $cache->get_url("http://www.google.com/"); $cache->set('test', 'one'); $cache->set('test', { foo => 'bar' }); my $scratch = $cache->scratch; $cache->clear; DESCRIPTION
The App::Cache module lets an application cache data locally. There are a few times an application would need to cache data: when it is retrieving information from the network or when it has to complete a large calculation. For example, the Parse::BACKPAN::Packages module downloads a file off the net and parses it, creating a data structure. Only then can it actually provide any useful information for the programmer. Parse::BACKPAN::Packages uses App::Cache to cache both the file download and data structures, providing much faster use when the data is cached. This module stores data in the home directory of the user, in a dot directory. For example, the Parse::BACKPAN::Packages cache is actually stored underneath "~/.parse_backpan_packages/cache/". This is so that permisssions are not a problem - it is a per-user, per-application cache. METHODS
new The constructor creates an App::Cache object. It takes three optional parameters: o ttl contains the number of seconds in which a cache entry expires. The default is 30 minutes. my $cache = App::Cache->new({ ttl => 30*60 }); o application sets the application name. If you are calling new() from a class, the application is automagically set to the calling class, so you should rarely need to pass it in: my $cache = App::Cache->new({ application => 'Your::Module' }); o directory sets the directory to be used for the cache. Normally this is just set for you and will be based on the application name and be created in the users home directory. Sometimes for testing, it can be useful to set this. my $cache = App::Cache->new({ directory => '/tmp/your/cache/dir' }); o enabled can be set to 0 for testing, in which case you will always get cache misses: my $cache = App::Cache->new({ enabled => 0 }); clear Clears the cache: $cache->clear; delete Deletes an entry in the cache: $cache->delete('test'); get Gets an entry from the cache. Returns undef if the entry does not exist or if it has expired: my $data = $cache->get('test'); get_code This is a convenience method. Gets an entry from the cache, but if the entry does not exist, set the entry to the value of the code reference passed: my $code = $cache->get_code("code", sub { $self->calculate() }); get_url This is a convenience method. Gets the content of a URL from the cache, but if the entry does not exist, set the entry to the content of the URL passed: my $html = $cache->get_url("http://www.google.com/"); scratch Returns a directory in the cache that the application may use for scratch files: my $scratch = $cache->scratch; set Set an entry in the cache. Note that an entry value may be an arbitrary Perl data structure: $cache->set('test', 'one'); $cache->set('test', { foo => 'bar' }); directory Returns the full path to the cache directory. Primarily useful for when you are writing tests that use App::Cache and want to clean up after yourself. If you are doing that you may want to explicitly set the 'application' constructor parameter to avoid later cleaning up a cache dir that was already in use. my $dir = $cache->directory; AUTHOR
Leon Brocard <acme@astray.com> COPYRIGHT
Copyright (C) 2005-7, Leon Brocard LICENSE
This module is free software; you can redistribute it or modify it under the same terms as Perl itself. perl v5.12.3 2009-12-08 App::Cache(3pm)
All times are GMT -4. The time now is 08:22 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy