The UNIX and Linux Forums  
Hello and Welcome from United States to the UNIX and Linux Forums! Thank You for Visiting and Joining Our Global Community.

Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
.
google unix.com



Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Finding the oldest file in a particular directory pavan_movva Shell Programming and Scripting 4 04-08-2009 07:24 AM
Finding the oldest file janet Shell Programming and Scripting 5 03-23-2009 10:31 AM
Finding missing sequential file names Julolidine Shell Programming and Scripting 2 07-02-2008 11:52 PM
Finding file version info davewg UNIX for Dummies Questions & Answers 1 11-09-2007 01:24 PM
how to find capital letter names in a file without finding words at start of sentence kev269 Shell Programming and Scripting 1 04-10-2006 09:35 PM

Closed Thread
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish Powered by Powered by Google
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 07-30-2008
nikosey nikosey is offline
Registered User
  
 

Join Date: Jul 2008
Posts: 3
Finding & Moving Oldest File by Parsing/Sorting Date Info in File Names

I'm trying to write a script that will look in an /exports folder for the oldest export file and move it to a /staging folder. "Oldest" in this case is actually determined by date information embedded in the file names themselves.

Also, the script should only move a file from /exports to /staging if staging is empty; if there's already a file in /staging then the script would exit without doing anything.

Here's an example of what the folders and files might look like:

/exports
job1_Friday_05302008.xml
job1_Monday_05262008.xml
job1_Monday_06022008.xml
job1_Thursday_05292008.xml
job1_Tuesday_06032008.xml
job1_Wednesday_05282008.xml

/staging

In this case, because /staging is empty, the script should move /exports/job1_Monday_05262008.xml to /staging. The next time the script runs it should exit if /exports still contains job1_Monday_05262008.xml, otherwise it should move job1_Wednesday_05282008.

Also, the "job1" prefix is static.

I've got a script that uses ls and cat to loop through file names and parse out the date information but i'm getting bogged down with how to handle the results and i'm starting to think i might be going down the wrong path...

Any suggestions would be appreciated
  #2 (permalink)  
Old 07-30-2008
joeyg's Avatar
joeyg joeyg is offline Forum Staff  
modérateur
  
 

Join Date: Dec 2007
Location: Home of 17-time world champion Boston Celtics
Posts: 1,311
Wink Here is a piece of it - I think anyway...

Assuming the file list is in a file already. (You could pipe the ls directly to the rest of the commands, if you were so inclined.)
Code:
> cat exp_contents 
job1_Friday_05302008.xml
job1_Monday_05262008.xml
job1_Monday_06022008.xml
job1_Thursday_05292008.xml
job1_Tuesday_06032008.xml
job1_Wednesday_05282008.xml
Now, try out the following command:
Code:
> cat exp_contents | tr "_" " " | awk '{print substr($3,5,4)"."substr($3,1,4)"|"$1"_"$2"_"$3}' | sort
2008.0526|job1_Monday_05262008.xml
2008.0528|job1_Wednesday_05282008.xml
2008.0529|job1_Thursday_05292008.xml
2008.0530|job1_Friday_05302008.xml
2008.0602|job1_Monday_06022008.xml
2008.0603|job1_Tuesday_06032008.xml
You can append to this a head -1 to get the top line.
You can also do a cut -d"|" -f2 to get the filename back for the oldest entry.

If this works, then the rest of the logic will flow much smoother.
  #3 (permalink)  
Old 07-30-2008
danmero danmero is offline Forum Advisor  
  
 

Join Date: Nov 2007
Location: 45.48-73.63
Posts: 1,420
Look like a homework.
Please read Simple rules of the UNIX.COM forums: before posting, especially 5 and 6.
  #4 (permalink)  
Old 07-30-2008
nikosey nikosey is offline
Registered User
  
 

Join Date: Jul 2008
Posts: 3
Clarification...

No, it's a real-world problem

It's just written out like a homework assignment because i've done enough technical work to know that when you ask for help, clarity is king.

I'm working on an interface that gets daily xml exports from an ftp site and synchronizes them with Siebel CRM. I've got Oracle Fusion Middleware to:

A) transfer files from the FTP site to an /exports folder

C) route files from a /staging folder to an Siebel inbound web service.

But I need a Step B) to meter out the appropriate files from /exports to /staging in the right order (first in, first out). I'm trying to do this with shell scripting but this is a bit outside my expertise, as you can tell...
  #5 (permalink)  
Old 07-30-2008
joeyg's Avatar
joeyg joeyg is offline Forum Staff  
modérateur
  
 

Join Date: Dec 2007
Location: Home of 17-time world champion Boston Celtics
Posts: 1,311
Wink Can you get thru what I posted previously?

From my earlier post, you should be able to determine the oldest file. Are you there yet?

You can set a variable such like:

Code:
old_file=$(cat exp_contents | tr "_" " " | awk '{print substr($3,5,4)"."substr($3,1,4)"|"$1"_"$2"_"$3}' | sort | head -1 | cut -d"|" -f2)
echo $old_file
If you are that far, then onto the 'easier' part of the requirements.
  #6 (permalink)  
Old 07-30-2008
nikosey nikosey is offline
Registered User
  
 

Join Date: Jul 2008
Posts: 3
Working Script

Yes, it's working perfectly!

Here's the full script i'm using in case it's helpful for anyone else.

It has an extra line at the top to purge files in an /archive folder that are > 7 days old.

And the folder names and locations are changed a bit from what they were in my original post.

But it seems to be working nicely.

Thanks again for the help!


Code:
#!/bin/ksh
find ../archive -name *.xml -type f -mtime +7 -exec rm {} \;
PROCESSING=$(ls ../processing | head -1)
if [[ -z $PROCESSING ]];then
   INCOMING=$(ls ../incoming/*.xml | cat | tr "_" " " | awk '{print substr($3,5,4)"."substr ($3,1,4)"|"$1"_"$2"_"$3}' | sort | head -1 | cut -d"|" -f2)
   if [[ -n $INCOMING ]];then
      mv $INCOMING ../processing
   fi
fi
  #7 (permalink)  
Old 07-30-2008
danmero danmero is offline Forum Advisor  
  
 

Join Date: Nov 2007
Location: 45.48-73.63
Posts: 1,420
You can use the file time, if staging is empty move the oldest xml file from exports to staging.
Code:
[ "$(ls -A staging)" ] || mv /exports/$(ls -t /exports/*.xml | tail -1) /staging/
Closed Thread

Bookmarks

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On




All times are GMT -4. The time now is 06:02 PM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0