Sponsored Content
Top Forums Shell Programming and Scripting Process 2 lists at the same time and move files. Post 302960127 by whegra on Tuesday 10th of November 2015 07:34:02 PM
Old 11-10-2015
Thank you for the assistance.

In the end I decided to dumb down my approach and keep it simple.

Here is what I did.

Code:
#Build full list of all files from the day in question
cd /archive/$year$month/storage/
ls *#$year$month$day* > /home/login/scripts/fulllist.txt

#Find all files that match list1 and output to new file
while read line
        do awk 'substr($0,35,7)=='"$line"' {print}' /home/login/scripts/fulllist.txt >> /home/login/scripts/fulllist2.txt
done < /home/login/scripts/list1.txt

#Find all files that match list2 from the list created above
while read line
        do awk 'substr($0,49,7)=='"$line"' {print}' /home/login/scripts/fulllist2.txt >> /home/login/scripts/fulllist3.txt
done < /home/login/scripts/list2.txt

#copy all files to temp folder
while read line ; do
        cp /archive/$year$month/storage/$line /foundfiles/
done < /home/login/scripts/fulllist3.txt



time ./program
real 0m6.327s
user 0m4.822s
sys 0m1.336s

The other methods I tried took way longer to process, my original method was taking many hours.

Last edited by whegra; 11-10-2015 at 09:58 PM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Archiving and move files in the same time

Hi All, I have tried so many command but none work like i wanted. I would like archive which i assume it will move the files and archive it somewhere. for example: if i have a folder and files: /home/blah/test /home/blah/hello /home/blah/foo/bar i would like to archive folder... (6 Replies)
Discussion started by: c00kie88
6 Replies

2. Shell Programming and Scripting

Shell script to move certain files on scheduled time

Hi Friends, I want a shell script which will move certain .jar files from a specified location (say /publish/content) to (/publish/archive) on every saturday morning 6 am. One more thing to add is that before moving files it must check free space at (/publish/archive), if it is more than 60 %... (7 Replies)
Discussion started by: abhishek27
7 Replies

3. Shell Programming and Scripting

Move files one at the time and wait until the previous file is handled

I'm a novice at unix and need it more and more to do my work. I seem running into problems getting this script "attempt" to work: I need to copy all files in a directory, which is containing 22000 files, into a directory one level up. There a tool monitors the content of the dir and processes... (2 Replies)
Discussion started by: compasscard
2 Replies

4. Shell Programming and Scripting

Shell Script to Create non-duplicate lists from two lists

File_A contains Strings: a b c d File_B contains Strings: a c z Need to have script written in either sh or ksh. Derive resultant files (File_New_A and File_New_B) from lists File_A and File_B where string elements in File_New_A and File_New_B are listed below. Resultant... (7 Replies)
Discussion started by: mlv_99
7 Replies

5. UNIX for Dummies Questions & Answers

move files between file systems with privileges, time stamp

Hi I have to move files between file systems but files in new file system must have the same attributes as in old one (privileges, time stamp etc). Which tool is best : - ufsdump / ufsrestore - tar - cpio - pax - dd - mv Or maybe there is sth else, you suggest to use. Thx for help (5 Replies)
Discussion started by: presul
5 Replies

6. UNIX for Dummies Questions & Answers

How to move files based on filetype and time created?

Hi, I'm trying to improve my Unix skills and I'm wondering what is the best way to move some files based on filetype and attributes like time created? For instance, lets suppose I have a directory with many different files in it and I'd like to move all the jpgs that were created between May... (6 Replies)
Discussion started by: LuckyTommy
6 Replies

7. Shell Programming and Scripting

Process files in a directory and move them

I have a directory e2e_ms_xfer/cent01 this contains the multiple files some of which will be named below with unique date time stamps e2e_ms_edd_nom_CCYYMMDD_HHMM.csv What I want to do is in a loop 1) Get the oldest file 2) Rename 3) Move it up one level from e2e_ms_xfer/cent01 to... (1 Reply)
Discussion started by: andymay
1 Replies

8. Shell Programming and Scripting

• Write a shell script that upon invocation shows the time and date and lists all the logged-in user

help me (1 Reply)
Discussion started by: sonu pandey
1 Replies

9. UNIX for Beginners Questions & Answers

How to move files older than certain time?

If there are 100 files created in a directory /data/ today from 2:00 AM to 10:00 AM I need to move files older than 3:00 AM to a new directory /hold/ How to do that? Also, if I have to move files between 3:00 AM to 9:00 AM, how to achieve that (3 Replies)
Discussion started by: eskay
3 Replies

10. Shell Programming and Scripting

How to generate all combinations of group lists at the same time?

Hi, everyone I have a group lists like this (more lines are omitted: a b c d E F G H .... I want to generate all combinations of two elements in each group line. What I expected is this: a b a c a d b c b d c d E F E G E H F G F H G H (8 Replies)
Discussion started by: nengcheng
8 Replies
CHPASS(1)						    BSD General Commands Manual 						 CHPASS(1)

NAME
chpass, chfn, chsh -- add or change user database information SYNOPSIS
chpass [-a list] [-s newshell] [-l] [user] chpass [-a list] [-s newshell] [-y] [user] DESCRIPTION
chpass allows editing of the user database information associated with user or, by default, the current user. The information is formatted and supplied to an editor for changes. Only the information that the user is allowed to change is displayed. The options are as follows: -a The super-user is allowed to directly supply a user database entry, in the format specified by passwd(5), as an argument. This argu- ment must be a colon (``:'') separated list of all the user database fields, although they may be empty. -s The -s option attempts to change the user's shell to newshell. -l This option causes the password to be updated only in the local password file. When changing only the local password, pwd_mkdb(8) is used to update the password databases. -y This forces the YP password database entry to be changed, even if the user has an entry in the local database. The rpc.yppasswdd(8) daemon should be running on the YP master server. Possible display items are as follows: Login: user's login name Password: user's encrypted password Uid: user's login Gid: user's login group Change: password change time Expire: account expiration time Class: user's general classification Home Directory: user's home directory Shell: user's login shell Full Name: user's real name Location: user's normal location Home Phone: user's home phone Office Phone: user's office phone The login field is the user name used to access the computer account. The password field contains the encrypted form of the user's password. The uid field is the number associated with the login field. Both of these fields should be unique across the system (and often across a group of systems) as they control file access. While it is possible to have multiple entries with identical login names and/or identical user id's, it is usually a mistake to do so. Rou- tines that manipulate these files will often return only one of the multiple entries, and that one by random selection. The group field is the group that the user will be placed in at login. Since BSD supports multiple groups (see groups(1)) this field cur- rently has little special meaning. This field may be filled in with either a number or a group name (see group(5)). The change field is the date by which the password must be changed. The expire field is the date on which the account expires. Both the change and expire fields should be entered in the form ``month day year'' where month is the month name (the first three characters are sufficient), day is the day of the month, and year is the year. The class field is a key for a user's login class. Login classes are defined in login.conf(5), which is a termcap(5) style database of user attributes, accounting, resource and environment settings. The user's home directory is the full UNIX path name where the user will be placed at login. The shell field is the command interpreter the user prefers. If the shell field is empty, the Bourne shell, /bin/sh, is assumed. When altering a login shell, and not the super-user, the user may not change from a non-standard shell or to a non-standard shell. Non-standard is defined as a shell not found in /etc/shells. The last four fields are for storing the user's full name, office location, and home and work telephone numbers. Once the information has been verified, chpass uses pwd_mkdb(8) to update the user database. ENVIRONMENT
The vi(1) editor will be used unless the environment variable EDITOR is set to an alternative editor. When the editor terminates, the infor- mation is re-read and used to update the user database itself. Only the user, or the super-user, may edit the information associated with the user. FILES
/etc/master.passwd The user database /etc/passwd A Version 7 format password file /etc/ptmp Lock file for the passwd database /tmp/pw.XXXXXX Temporary copy of the user passwd information /etc/shells The list of approved shells SEE ALSO
finger(1), login(1), passwd(1), pwhash(1), getusershell(3), passwd(5), passwd.conf(5), pwd_mkdb(8), vipw(8) Robert Morris and Ken Thompson, UNIX Password Security. HISTORY
The chpass command appeared in 4.3BSD-Reno. BUGS
This program's interface is poorly suited to cryptographic systems such as Kerberos, and consequently Kerberos password changing is not a feature of this program. User information should (and eventually will) be stored elsewhere. BSD
October 7, 2006 BSD
All times are GMT -4. The time now is 10:30 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy