Sponsored Content
Top Forums Shell Programming and Scripting Process 2 lists at the same time and move files. Post 302960237 by Don Cragun on Thursday 12th of November 2015 03:32:37 PM
Old 11-12-2015
Just for my own sanity, could you please try the following with your data and let me know how the runtime compares to your script? I get the feeling it should be faster, but maybe all of the processing time is being spent copying files and the time used determining which files to copy doesn't matter...
Code:
#!/bin/ksh
year="2015"	# Replace with your desired year.
month="11"	# Replace with your desired month.
day="11"	# Replace with your desired day.

#Get full list of all files from the day in question
cd /archive/$year$month/storage/
ls *"#$year$month$day"* |

#Find all files that match list1 and list2...
awk '
FNR == 1 {
	f++
}
f == 1 {list1[$0]
	next
}
f == 2 {list2[$0]
	next
}
substr($0, 35, 7) in list1 && substr($0, 49, 7) in list2
' /home/login/scripts/list1.txt /home/login/scripts/list2.txt - |

#copy all files to temp folder
while read -r line
do	cp "$line" /foundfiles/
done

This User Gave Thanks to Don Cragun For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Archiving and move files in the same time

Hi All, I have tried so many command but none work like i wanted. I would like archive which i assume it will move the files and archive it somewhere. for example: if i have a folder and files: /home/blah/test /home/blah/hello /home/blah/foo/bar i would like to archive folder... (6 Replies)
Discussion started by: c00kie88
6 Replies

2. Shell Programming and Scripting

Shell script to move certain files on scheduled time

Hi Friends, I want a shell script which will move certain .jar files from a specified location (say /publish/content) to (/publish/archive) on every saturday morning 6 am. One more thing to add is that before moving files it must check free space at (/publish/archive), if it is more than 60 %... (7 Replies)
Discussion started by: abhishek27
7 Replies

3. Shell Programming and Scripting

Move files one at the time and wait until the previous file is handled

I'm a novice at unix and need it more and more to do my work. I seem running into problems getting this script "attempt" to work: I need to copy all files in a directory, which is containing 22000 files, into a directory one level up. There a tool monitors the content of the dir and processes... (2 Replies)
Discussion started by: compasscard
2 Replies

4. Shell Programming and Scripting

Shell Script to Create non-duplicate lists from two lists

File_A contains Strings: a b c d File_B contains Strings: a c z Need to have script written in either sh or ksh. Derive resultant files (File_New_A and File_New_B) from lists File_A and File_B where string elements in File_New_A and File_New_B are listed below. Resultant... (7 Replies)
Discussion started by: mlv_99
7 Replies

5. UNIX for Dummies Questions & Answers

move files between file systems with privileges, time stamp

Hi I have to move files between file systems but files in new file system must have the same attributes as in old one (privileges, time stamp etc). Which tool is best : - ufsdump / ufsrestore - tar - cpio - pax - dd - mv Or maybe there is sth else, you suggest to use. Thx for help (5 Replies)
Discussion started by: presul
5 Replies

6. UNIX for Dummies Questions & Answers

How to move files based on filetype and time created?

Hi, I'm trying to improve my Unix skills and I'm wondering what is the best way to move some files based on filetype and attributes like time created? For instance, lets suppose I have a directory with many different files in it and I'd like to move all the jpgs that were created between May... (6 Replies)
Discussion started by: LuckyTommy
6 Replies

7. Shell Programming and Scripting

Process files in a directory and move them

I have a directory e2e_ms_xfer/cent01 this contains the multiple files some of which will be named below with unique date time stamps e2e_ms_edd_nom_CCYYMMDD_HHMM.csv What I want to do is in a loop 1) Get the oldest file 2) Rename 3) Move it up one level from e2e_ms_xfer/cent01 to... (1 Reply)
Discussion started by: andymay
1 Replies

8. Shell Programming and Scripting

• Write a shell script that upon invocation shows the time and date and lists all the logged-in user

help me (1 Reply)
Discussion started by: sonu pandey
1 Replies

9. UNIX for Beginners Questions & Answers

How to move files older than certain time?

If there are 100 files created in a directory /data/ today from 2:00 AM to 10:00 AM I need to move files older than 3:00 AM to a new directory /hold/ How to do that? Also, if I have to move files between 3:00 AM to 9:00 AM, how to achieve that (3 Replies)
Discussion started by: eskay
3 Replies

10. Shell Programming and Scripting

How to generate all combinations of group lists at the same time?

Hi, everyone I have a group lists like this (more lines are omitted: a b c d E F G H .... I want to generate all combinations of two elements in each group line. What I expected is this: a b a c a d b c b d c d E F E G E H F G F H G H (8 Replies)
Discussion started by: nengcheng
8 Replies
foreach(n)						       Tcl Built-In Commands							foreach(n)

__________________________________________________________________________________________________________________________________________________

NAME
foreach - Iterate over all elements in one or more lists SYNOPSIS
foreach varname list body foreach varlist1 list1 ?varlist2 list2 ...? body _________________________________________________________________ DESCRIPTION
The foreach command implements a loop where the loop variable(s) take on values from one or more lists. In the simplest case there is one loop variable, varname, and one list, list, that is a list of values to assign to varname. The body argument is a Tcl script. For each element of list (in order from first to last), foreach assigns the contents of the element to varname as if the lindex command had been used to extract the element, then calls the Tcl interpreter to execute body. In the general case there can be more than one value list (e.g., list1 and list2), and each value list can be associated with a list of loop variables (e.g., varlist1 and varlist2). During each iteration of the loop the variables of each varlist are assigned consecutive values from the corresponding list. Values in each list are used in order from first to last, and each value is used exactly once. The total number of loop iterations is large enough to use up all the values from all the value lists. If a value list does not contain enough elements for each of its loop variables in each iteration, empty values are used for the missing elements. The break and continue statements may be invoked inside body, with the same effect as in the for command. Foreach returns an empty string. EXAMPLES
This loop prints every value in a list together with the square and cube of the value: set values {1 3 5 7 2 4 6 8} ;# Odd numbers first, for fun! puts "Value Square Cube" ;# Neat-looking header foreach x $values { ;# Now loop and print... puts " $x [expr {$x**2}] [expr {$x**3}]" } The following loop uses i and j as loop variables to iterate over pairs of elements of a single list. set x {} foreach {i j} {a b c d e f} { lappend x $j $i } # The value of x is "b a d c f e" # There are 3 iterations of the loop. The next loop uses i and j to iterate over two lists in parallel. set x {} foreach i {a b c} j {d e f g} { lappend x $i $j } # The value of x is "a d b e c f {} g" # There are 4 iterations of the loop. The two forms are combined in the following example. set x {} foreach i {a b c} {j k} {d e f g} { lappend x $i $j $k } # The value of x is "a d e b f g c {} {}" # There are 3 iterations of the loop. SEE ALSO
for(n), while(n), break(n), continue(n) KEYWORDS
foreach, iteration, list, looping Tcl foreach(n)
All times are GMT -4. The time now is 10:31 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy