Sponsored Content
Top Forums Shell Programming and Scripting Process 2 lists at the same time and move files. Post 302960107 by RudiC on Tuesday 10th of November 2015 04:59:27 PM
Old 11-10-2015
You seem to be assuming that all those files with all possible value combinations exist. Should that not be the case, running cp for each and ignoring the error message is a huge waste of resources. And, although it will most probably be cached/buffered by the system, you are reading list1 about a hundred times. How about
- reading list1 and list2 once into memory
- running cp only for existing files?
Try
Code:
cd /archive/$year$month/storage
ls | awk '
FNR==1          {FC++}
FC<=2           {A[FC,FNR] = $0
                 C[FC] = FNR
                 next
                }
                {for (i=1; i<=C[1]; i++)
                   for (j=1; j<=C[2]; j++)
                     if ($0 ~ A[1,i]"[NIT]......"A[2,j]"#") print "cp", $0, "/foundfiles"
                }
' SUBSEP="\t" path/to/list1 path/to/list2 - | sh

Please adapt the match regex to your needs!

---------- Post updated at 22:59 ---------- Previous update was at 22:50 ----------

You could further improve performance by excluding unapt files with a pattern to ls. And, you could break out of the loops when a file was found.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Archiving and move files in the same time

Hi All, I have tried so many command but none work like i wanted. I would like archive which i assume it will move the files and archive it somewhere. for example: if i have a folder and files: /home/blah/test /home/blah/hello /home/blah/foo/bar i would like to archive folder... (6 Replies)
Discussion started by: c00kie88
6 Replies

2. Shell Programming and Scripting

Shell script to move certain files on scheduled time

Hi Friends, I want a shell script which will move certain .jar files from a specified location (say /publish/content) to (/publish/archive) on every saturday morning 6 am. One more thing to add is that before moving files it must check free space at (/publish/archive), if it is more than 60 %... (7 Replies)
Discussion started by: abhishek27
7 Replies

3. Shell Programming and Scripting

Move files one at the time and wait until the previous file is handled

I'm a novice at unix and need it more and more to do my work. I seem running into problems getting this script "attempt" to work: I need to copy all files in a directory, which is containing 22000 files, into a directory one level up. There a tool monitors the content of the dir and processes... (2 Replies)
Discussion started by: compasscard
2 Replies

4. Shell Programming and Scripting

Shell Script to Create non-duplicate lists from two lists

File_A contains Strings: a b c d File_B contains Strings: a c z Need to have script written in either sh or ksh. Derive resultant files (File_New_A and File_New_B) from lists File_A and File_B where string elements in File_New_A and File_New_B are listed below. Resultant... (7 Replies)
Discussion started by: mlv_99
7 Replies

5. UNIX for Dummies Questions & Answers

move files between file systems with privileges, time stamp

Hi I have to move files between file systems but files in new file system must have the same attributes as in old one (privileges, time stamp etc). Which tool is best : - ufsdump / ufsrestore - tar - cpio - pax - dd - mv Or maybe there is sth else, you suggest to use. Thx for help (5 Replies)
Discussion started by: presul
5 Replies

6. UNIX for Dummies Questions & Answers

How to move files based on filetype and time created?

Hi, I'm trying to improve my Unix skills and I'm wondering what is the best way to move some files based on filetype and attributes like time created? For instance, lets suppose I have a directory with many different files in it and I'd like to move all the jpgs that were created between May... (6 Replies)
Discussion started by: LuckyTommy
6 Replies

7. Shell Programming and Scripting

Process files in a directory and move them

I have a directory e2e_ms_xfer/cent01 this contains the multiple files some of which will be named below with unique date time stamps e2e_ms_edd_nom_CCYYMMDD_HHMM.csv What I want to do is in a loop 1) Get the oldest file 2) Rename 3) Move it up one level from e2e_ms_xfer/cent01 to... (1 Reply)
Discussion started by: andymay
1 Replies

8. Shell Programming and Scripting

• Write a shell script that upon invocation shows the time and date and lists all the logged-in user

help me (1 Reply)
Discussion started by: sonu pandey
1 Replies

9. UNIX for Beginners Questions & Answers

How to move files older than certain time?

If there are 100 files created in a directory /data/ today from 2:00 AM to 10:00 AM I need to move files older than 3:00 AM to a new directory /hold/ How to do that? Also, if I have to move files between 3:00 AM to 9:00 AM, how to achieve that (3 Replies)
Discussion started by: eskay
3 Replies

10. Shell Programming and Scripting

How to generate all combinations of group lists at the same time?

Hi, everyone I have a group lists like this (more lines are omitted: a b c d E F G H .... I want to generate all combinations of two elements in each group line. What I expected is this: a b a c a d b c b d c d E F E G E H F G F H G H (8 Replies)
Discussion started by: nengcheng
8 Replies
SHELL-QUOTE(1)						User Contributed Perl Documentation					    SHELL-QUOTE(1)

NAME
shell-quote - quote arguments for safe use, unmodified in a shell command SYNOPSIS
shell-quote [switch]... arg... DESCRIPTION
shell-quote lets you pass arbitrary strings through the shell so that they won't be changed by the shell. This lets you process commands or files with embedded white space or shell globbing characters safely. Here are a few examples. EXAMPLES
ssh preserving args When running a remote command with ssh, ssh doesn't preserve the separate arguments it receives. It just joins them with spaces and passes them to "$SHELL -c". This doesn't work as intended: ssh host touch 'hi there' # fails It creates 2 files, hi and there. Instead, do this: cmd=`shell-quote touch 'hi there'` ssh host "$cmd" This gives you just 1 file, hi there. process find output It's not ordinarily possible to process an arbitrary list of files output by find with a shell script. Anything you put in $IFS to split up the output could legitimately be in a file's name. Here's how you can do it using shell-quote: eval set -- `find -type f -print0 | xargs -0 shell-quote --` debug shell scripts shell-quote is better than echo for debugging shell scripts. debug() { [ -z "$debug" ] || shell-quote "debug:" "$@" } With echo you can't tell the difference between "debug 'foo bar'" and "debug foo bar", but with shell-quote you can. save a command for later shell-quote can be used to build up a shell command to run later. Say you want the user to be able to give you switches for a command you're going to run. If you don't want the switches to be re-evaluated by the shell (which is usually a good idea, else there are things the user can't pass through), you can do something like this: user_switches= while [ $# != 0 ] do case x$1 in x--pass-through) [ $# -gt 1 ] || die "need an argument for $1" user_switches="$user_switches "`shell-quote -- "$2"` shift;; # process other switches esac shift done # later eval "shell-quote some-command $user_switches my args" OPTIONS
--debug Turn debugging on. --help Show the usage message and die. --version Show the version number and exit. AVAILABILITY
The code is licensed under the GNU GPL. Check http://www.argon.org/~roderick/ or CPAN for updated versions. AUTHOR
Roderick Schertler <roderick@argon.org> perl v5.16.3 2010-06-11 SHELL-QUOTE(1)
All times are GMT -4. The time now is 08:35 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy