Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Trouble with archiving; only works if file exists already

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Trouble with archiving; only works if file exists already
# 1  
Old 09-21-2016
Trouble with archiving; only works if file exists already
Hi Folks -

I have a script I am executing that I wish to archive the output(logfile) file once the script is done executing to a folder based on the following date format:

YYYYMM

For this example purpose, I am performing a simple MKDIR. The script works fine, but the archiving is the issue. If the archive directory doesn't exist, upon first execution, it's made, but then the log file is no where to be found. THEN, if I execute a second time, it moves the file to the archive folder.

I'll share my code:

Code:
#:: SET SCRIPT NAME
_SN=TEST_SCRIPT

#:: SET PATHS
_MAINPATH=/home/essadmin/Hyperion_Batch/
_LOGPATH=Logs/
_ERRORPATH=Errors/

_LOG_IPATH=${_MAINPATH}${_LOGPATH}
_ERROR_IPATH=${_MAINPATH}${_ERRORPATH}

#:: SET DATE & TIME VARIBLES:
_DAY=$(date +%d)
_MONTH=$(date +%m)
_YEAR=$(date +%Y)
_DATESTAMP=${_YEAR}${_MONTH}${_DAY}
_HOUR=$(date +%H)
_MINUTE=$(date +%M)
_SECOND=$(date +%S)
_TIME=${_HOUR}${_MINUTE}
_DATETIMESTAMP=$_DATESTAMP-$_TIME

_LOGFILE=${_LOG_IPATH}${_DATETIMESTAMP}_${_SN}.log
_ERRORFILE=${_ERROR_IPATH}${_DATETIMESTAMP}_${_SN}.err

#:: Prepare folder structure
_ARC_LF=${_MAINPATH}${_LOGPATH}${_YEAR}${_MONTH}
_ARC_EF=${_MAINPATH}${_ERRORPATH}${_YEAR}${_MONTH}

#:: Execute script commands
echo --------------------------------------------------------->>${_LOGFILE}
echo "${_SN} beginning at ${_TIME}"                           >>${_LOGFILE}
echo .                                                          >>${_LOGFILE}
echo .                                                          >>${_LOGFILE}
echo "Make Directory"                                         >>${_LOGFILE}
echo --------------------------------------------------------->>${_LOGFILE}

mkdir -p ${_MAINPATH}TEST

if [ $? -eq 0 ]
then
  echo --------------------------------------------------------->>${_LOGFILE}
  echo "Directory Made Successfully!"                           >>${_LOGFILE}
  echo --------------------------------------------------------->>${_LOGFILE}
  fi
        if [[ ! -e ${_ARC_LF} ]]; then
            mkdir -p ${_ARC_LF}
        fi
  mv ${_MAINPATH}${_LOGPATH}* ${_ARC_LF} 
  exit 0
  
else
  echo ------------------------------------------------------->>${_ERRORFILE}
  echo "Directory Made Unsuccessfully!"                       >>${_ERRORFILE}
  echo ------------------------------------------------------->>${_ERRORFILE}
  fi
      if [[ ! -e ${_ARC_EF} ]]; then
        mkdir -p ${_ARC_EF}
    fi
  mv ${_MAINPATH}${_ERRORPATH}* ${_ARC_EF} 
  exit 1

Any help is greatly appreciated! Thanks!
# 2  
Old 09-21-2016
The logfile would be created before the mkdir either way, so I think it must be permission issues preventing you from creating the log file. If you don't have permission to mkdir, you might not have permission to create a logfile, either.

Remember that file and directory permissions are separate. You need write permission on the directory to create a file or folder inside it.

I think you can simplify your code a lot:

Code:
# redirect stdout
exec > "${_LOG_IPATH}${_DATETIMESTAMP}_${_SN}.log"
# No need for >>FILENAME any more
echo "All stdout including this now goes to the log file"

# No need to test $?, just run the command directly in if
if mkdir -p "${_MAINPATH}TEST"
then
        # success
else
        # failure
fi

# 3  
Old 09-21-2016
Noticed that you define IPATH to write the log file.
But then you do not use IPATH when attempting to move the log file, but use the two variables. Unsure if this has any impact on your issue, but it was odd.

Code:
mv ${_MAINPATH}${_LOGPATH}* ${_ARC_LF}

# 4  
Old 09-21-2016
Ah - yes you guys are right, my code is a little messy. But, i'm reletively new at this and learning each day! My wheelhouse is DOS. (yes yes I know LOL )

I amended my script as such. However, an *.err file get written to the error file directory even when there isn't an error when using the exec 2> > command.

Ultimately, I dont even want to err file directory even created unless an error arises. Any way to amend my code to meet that goal?

Here's my code:

Code:
#:: SET SCRIPT NAME
_SN=TEST_SCRIPT

#:: SET PATHS
_MAINPATH=/home/essadmin/Hyperion_Batch/
_LOGPATH=Logs/
_ERRORPATH=Errors/

#:: SET DATE & TIME VARIBLES:
_DAY=$(date +%d)
_MONTH=$(date +%m)
_YEAR=$(date +%Y)
_DATESTAMP=${_YEAR}${_MONTH}${_DAY}
_HOUR=$(date +%H)
_MINUTE=$(date +%M)
_SECOND=$(date +%S)
_TIME=${_HOUR}${_MINUTE}
_DATETIMESTAMP=$_DATESTAMP-$_TIME

#:: Prepare Peloton POV folder structure
_ARC_LF=${_MAINPATH}${_LOGPATH}${_YEAR}${_MONTH}
_ARC_EF=${_MAINPATH}${_ERRORPATH}${_YEAR}${_MONTH}
    mkdir -p ${_ARC_LF}
    mkdir -p ${_ARC_EF}

_FILE=${_DATESTAMP}_${_SN}
_LOG_OUT=${_ARC_LF}/${_FILE}.log
_ERR_OUT=${_ARC_EF}/${_FILE}.err
    
exec 2>${_ERR_OUT} > ${_LOG_OUT}

#:: Execute script commands
echo ---------------------------------------------------------
echo "${_SN} beginning at ${_TIME}"                           
echo .                                                          
echo .                                                          
echo "Make Directory"                                         
echo ---------------------------------------------------------

mkdir -p ${_MAINPATH}TEST

if [ $? -eq 0 ]
then
  echo ---------------------------------------------------------
  echo "Directory Made Successfully!"                           
  echo ---------------------------------------------------------
  fi
  exit 0
  
else
  echo -------------------------------------------------------
  echo "Directory Made Unsuccessfully!"                       
  echo -------------------------------------------------------
  fi
  exit 1

Thank you!
# 5  
Old 09-21-2016
Opening file descriptor 2 like that allows you to catch the error messages from mkdir / mv themselves, which would otherwise go straight to terminal. You have to open it in advance, or mkdir / mv won't have it when they need it.

You could delete the .err file on quit, if it happens to still be zero in size.

Code:
exec 2>${_ERR_OUT} > ${_LOG_OUT}
# Delete empty .err file on exit
trap "[ -s ${_ERR_OUT} ] || rm -f ${_ERR_OUT} ]" EXIT

|| is a short-circuit if-else. [ -s ${_ERR_OUT} ] fails if the log file is empty, causing the rm -f after it to be executed.
This User Gave Thanks to Corona688 For This Post:
SIMMS7400
# 6  
Old 09-21-2016
Works like a charm!! Thank you!

Can we also trap a piece of code in there that will then deletes the directory it was in to, provided the directory is EMPTY?

For instance, it makes this error path during script execution:

/home/essadmin/Hyperion_Batch/Errors/201609

The error file gets deleted if EMPTY, but I'd also like to delete 201609 as well if empty.

Thank you very much for all your help!
# 7  
Old 09-21-2016
How's this?

Code:
trap "[ -s ${_ERR_OUT} ] || rm -f ${_ERR_OUT} ] && rmdir ${_ARC_EF}" EXIT

&& is the opposite of ||, it runs code if the previous statement succeeded. If rm succeeds, rmdir will run after it.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Waking Up USB External Hard Drive for file archiving

Hello Experts, I hope I'm writing to the correct category for my question. I have a very basic shell script for doing file archiving to the external usb hard drive (WD studio edition II 2TB formatted as FAT32 for compatibility). The shell script only needs to run once per day. It basically... (8 Replies)
Discussion started by: johankor
8 Replies

2. Shell Programming and Scripting

Archiving or removing few data from log file in real time

Hi, I have a log file that gets updated every second. Currently the size has grown to 20+ GB. I need to have a command/script, that will try to get the actual size of the file and will remove 50% of the data that are in the log file. I don't mind removing the data as the size has grown to huge... (8 Replies)
Discussion started by: Souvik Patra
8 Replies

3. Shell Programming and Scripting

Archiving a log file on monthly basis

OS : RedHat Linux 6.2 Shell : Bash Our application write messages, warnings,..etc to the following log file . /app/cms/diagnostics/logs/cms_messages.log This file has become huge now. I want this file to be archived on monthly basis using a small shell script. ie. On the 1st day of... (1 Reply)
Discussion started by: omega3
1 Replies

4. Shell Programming and Scripting

File exists, but cannot be opened.How to check- whether it could be opened to read when it exists

Hi #Testing for file existence if ; then echo 'SCHOOL data is available for processing' else echo 'SCHOOL DATA IS NOT AVAILABLE FOR PROCESSING' : i wrote a script, where it begins by checking if file exists or not. If it exists, it truncates the database... (2 Replies)
Discussion started by: rxg
2 Replies

5. Windows & DOS: Issues & Discussions

Script that, if file exists in Samba share, moves file to Unix server

I'm looking to do pretty much what the title says. I want a script that runs, it can run on Unix or Windows, doesn't matter, and searches a Samba shares for a .txt file. If the file exists, the script will move (or possibly copy) the file from the Samba share into a directory on our Unix... (3 Replies)
Discussion started by: twcostello
3 Replies

6. Shell Programming and Scripting

Script to check for the file existence, if file exists it should echo the no of modified days

Hi, I am looking for a shell script with the following. 1. It should check whether a particular file exists in a location #!/bin/sh if ; then echo "xxx.txt File Exists" else echo "File Not Found" fi 2. If file exists, it should check for the modified date and run a command... (2 Replies)
Discussion started by: karthikeyan_mac
2 Replies

7. Shell Programming and Scripting

Newbie.. Find if a file exists and open, if not create the desired file..

Hey all, I'm brand new to script writing, I'm wanting to make a script that will ask for a file and then retrieve that file if it exists, and if it doesn't exist, create the file with the desired name, and I'm completely stuck.. so far.. #! bin/bash echo "Enter desired file" read "$file" if ... (5 Replies)
Discussion started by: Byrang
5 Replies

8. UNIX for Dummies Questions & Answers

Archiving the files in a .txt file

HI , I have a file abc.txt, which has some .csv files listed. example. abc.txt 1.csv 2.csv 3.csv 4.csv 5.csv I want to move all the files listed in abc.txt to a archive directory,and zip the moved files. Can anyone help me with the script. Thanks,sai (1 Reply)
Discussion started by: saii
1 Replies

9. Shell Programming and Scripting

Check File Exists and compare to previous day file script

We have data files that are ftp'd every morning to a SUN server. The file names are exactly the same except for that each has the date included in its name. I have to write script to do 2 things: STEP 1) Verify that the file arrived in morning. STEP 2) Compare the file size of the current... (3 Replies)
Discussion started by: rbknisely
3 Replies

10. UNIX for Dummies Questions & Answers

Archiving Files by selecting years file created

I have recently taken on a new position and want to clean up several file locations that currently hold data back through 1999. While I need to keep this data for business reasons, I have created directories to help sort out the individual years. Is there a quick command that I can use to archive... (2 Replies)
Discussion started by: dmhammond
2 Replies
Login or Register to Ask a Question