Visit The New, Modern Unix Linux Community


Portable shell script advise


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Portable shell script advise
# 1  
Portable shell script advise

All,

I have a need for a portable shell script for LInux and HPUX. The script has a simple need; Check for local files of a specific name (i.e. filename*), scp them to another system, and archive them to a folder. The script runs via cron.

I first wrote the script in Linux (bash with gnu tools) pretty quickly just to get it to function. Then I was asked to do the same type of process running on HPUX (we have 11.31 and 11.23) and did some quick tweaks to get it to run on HPUX. I'm running into issues with the way I wrote it and timing of files written, sent, and archived (I didn't really have the scope of frequency).

Basically I want to put together a portable script with assurances that files will not get 'missed'.

My thought is:

- do a listing of the desired files when the script is run

- If no files, exit
- If files exist, drop the listing into an array

- use the array to send and archive files

Does this sound like the best/easiest way to accomplish this?

The current problem code I have is:

Code:
IFS=!
dest=servername
inst=INST2
logdir=/path/to/log/dir
month=`date +%b%Y`
logfile=${logdir}/xfer_${1}_${month}.log

if [ "$1" = "us" ]; then

        filedir=/path/to/us/dir
        destdir=/path/${inst}/path/us

elif [ "$1" = "eu" ]; then

        filedir=/path/to/eu/dir
        destdir=/path/${inst}/path/eu

fi

# --------------------------------------------------
# functions
# --------------------------------------------------

usage () {

        echo "usage info"

}

header () {

        divider===============================
        divider=$divider$divider$divider

        printf "\n$divider\n" >> $logfile 2>&1
        echo "$1 -- `date`" >> $logfile 2>&1
        printf "$divider\n" >> $logfile 2>&1

}

# --------------------------------------------------
# main
# --------------------------------------------------

if [ $# -ne 1 ]; then

        usage
        exit 1

fi

cd $filedir

        files=`find $filedir -type f -name "iS*" | xargs -n1 printf %f!`

                if [ ${#files[@]} -eq 0 ]; then

                        header "$1 files to send"
                        echo "\n No $1 files to transfer" >> $logfile
                        exit 1

                else

                        header "$1 files to send"
                        ls -l iS* | awk '{print $9}' >> $logfile

                        header "Sending $1 files"
                        cd $filedir; scp iS* $dest:$destdir >> $logfile 2>&1

                        if [ $? -ne 0 ]; then

                                echo "\n  There are no files to transfer..." >> $logfile 2>&1
                                header "Complete"
                                exit 1

                        else

                                header "$1 files successfully sent"

                                header "Moving files and purge 30+ days"
                                mv iS* ${filedir}/archive >> $logfile 2>&1
                                find ${filedir}/archive -type f -mtime +30 -ls -exec rm -f -- {} \; >> $logfile 2>&1
                                header "Finished moving files.."

                        fi

                fi

exit

I know this code is not written very well as there is no checks on archiving only the files that were sent.

Any advise on improving for portability and simplifying is greatly appreciated.

TIA,

Herb
# 2  
echo "\n" isn't portable. Neither is the echo -e some versions of echo require to make \n work. If you want escapes like \n in the string, you must use printf.

Note that it's better to use %s in a printf command string than to substitute variables into it, since you don't want to put more escape sequences into a printf string by accident.

Code:
STR="This should print percent signs like %s without bombing or mangling them"
# safe
printf "\n%s\n" "$STR"
# unsafe
printf "\n$STR\n"


Last edited by Corona688; 04-27-2016 at 06:26 PM..
# 3  
Here is one that should be safe, efficient, and portable even to older Solaris and HP-UX.

Code:
#!/bin/sh

# we want to find programs here, not in an inherited PATH
PATH=/bin:/usr/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/sbin
export PATH

dest=servername
inst=INST2
month=`date +%b%Y`
logdir=/path/to/log/dir
logfile=${logdir}/xfer_${1}_${month}.log
fmask="iS*"

# open once with descriptor 4
exec 4>"$logfile"
find 
#exec 4>>"$logfile"
# writing to descriptor 4 will append to the open file

case $1 in

us)
    filedir=/path/to/us/dir
    destdir=/path/${inst}/path/us
;;
eu)
    filedir=/path/to/eu/dir
    destdir=/path/${inst}/path/eu
;;
*)
    echo "unexpected argument: $1"
    exit 1
;;

esac

# --------------------------------------------------
# functions
# --------------------------------------------------

usage () {

    echo "usage info"

}

header () {

    divider="=============================="
    divider=$divider$divider$divider

    printf "\n$divider\n" >&4
    echo "$1 -- `date`" >&4
    printf "$divider\n" >&4

}

# --------------------------------------------------
# main
# --------------------------------------------------

if [ $# -ne 1 ]; then

    usage
    exit 1

fi

# make sure we can go there
if cd $filedir
then

    files=`ls -- $fmask 2>/dev/null`
    header "$1 files to send"

    if [ -z "$files" ]; then

        printf "\n No $1 files to transfer" >&4

    else

        # a lazy printf "$files\n" or echo "$files" is at risk
        printf "%s\n" "$files" >&4

        header "Sending $1 files"
        # a trailing / makes sure it is a directory
        scp -- $fmask ${dest}:${destdir}/ >&4 2>&1

        if [ $? -ne 0 ]; then

            printf "\n  There are no files to transfer..." >&4
            header "Complete"

        else

            header "$1 files successfully sent"

            header "Moving files and purge 30+ days"
            mv -f -- $fmask ${filedir}/archive/ >&4 2>&1
# + collects arguments and calls the program very few times -> faster; no -- needed in find -exec prog {}
# HP-UX does not have -ls
            find ${filedir}/archive/ -type f -mtime +30 -exec ls -l {} + -exec rm -f {} + >&4 2>&1
            header "Finished moving files.."

        fi

    fi

fi

# explicit close of logfile not needed before an exit or end of file
#exec 4>&-

exit


Last edited by MadeInGermany; 04-28-2016 at 03:05 PM.. Reason: no -- needed in find -exec prog {}
# 4  
@Corona688, @MadeInGermany thank you for your responses..

@MadeInGermany - Quick question:

Regarding the code you post, does the way $fmask is used solve the issue I have been seeing where:

use $fmask to scp files to $destdir

- between scp and archive of files a new file is written to $filedir

use $fmask to archive files

In this scenario, would any files that were written to $filedir between scp and archive get archived without being sent to $destdir?

Thanks again for any guidance.

Herb
# 5  
No, because I tried to keep the functionality.
The $fmask simply allows to specify the "iS*" once at the beginning of the script. The shell will re-evaluate it every time.

But it is no big deal to replace the $fmask by $files for the scp and mv commands.
Then $fmask is evaluated once with the ls command, where the result goes to $files - and that is static.

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #113
Difficulty: Easy
Linux is not Unix per se, but Linux is a Unix-like operating system.
True or False?

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Tip: template for a safe and portable script

In an attempt to finally end this article I start this new thread. Here is a template for a safe and portable script. #!/bin/bash # /bin/bash exists on most still supported Unixes # # tr and date behave better with if ; then export LC_ALL=C; else export LANG=C; fi # # Unix optional packages... (2 Replies)
Discussion started by: MadeInGermany
2 Replies

2. Shell Programming and Scripting

Optimizing the Shell Script [Expert Advise Needed]

I have prepared a shell script to find the duplicates based on the part of filename and retain latest. #!/bin/bash if ; then mkdir -p dup fi NOW=$(date +"%F-%H:%M:%S") LOGFILE="purge_duplicate_log-$NOW.log" LOGTIME=`date "+%Y-%m-%d %H:%M:%S"` echo... (6 Replies)
Discussion started by: gold2k8
6 Replies

3. Shell Programming and Scripting

Portable Shell Script - Determine Which Version of Binary is Installed?

I currently have a shell script that utilizes the "Date" binary - this application is slightly different on OS X (BSD General Commmand) and Linux systems (gnu date). In particular, the version on OS X requires the following to get a date 14 days in the future "date -v+14d -u +%Y-%m-%d" where gnu... (1 Reply)
Discussion started by: colinjohnson
1 Replies

4. Shell Programming and Scripting

Solaris script using awk giving errors - please advise

I'm using solaris 10 Scenario as follows I have a logfile with 2 columns: column 1 = source directory + filename column 2 = destination directory + filename Using cron, my script polls for new files and adds them to the logfile ($ELOG) as described above. Using sed, the distination... (2 Replies)
Discussion started by: davidra
2 Replies

5. Shell Programming and Scripting

advise some script to make the changes..!!

Hi Folks, got the solution..!! (5 Replies)
Discussion started by: tuntun
5 Replies

6. Shell Programming and Scripting

Query regarding portable script

Dear Experts, I want to write a script which has to work on Solaris & Linux sytems. The problem which i am facing is, there are commands whose options are different on both OS's. For example ping. On Solaris i have to write: ping $host 1 to check if the host is alive On Linux i... (4 Replies)
Discussion started by: dhiraj4mann
4 Replies

7. Shell Programming and Scripting

mail program on shell script didn't work, please advise.

Hi, everyone: I post a new thread because previous post may sink and I hope the new one can be caught by your eyes. I created a shell script and the script works fine. However, the mail program part on script didn't send email to my email box and it also didn't provide any traceable... (7 Replies)
Discussion started by: duke0001
7 Replies

8. Windows & DOS: Issues & Discussions

Portable GUI shell scripting?

I have some scripts that run in Windows, and the ability to put a GUI layer over them is kind of appealing. So when I stumbled over http://wizapp.sourceforge.net/ I thought that was rather cute. (Reading the "advanced batch scripting" art of the manual impresses upon me that no sane person... (2 Replies)
Discussion started by: luke
2 Replies

9. Shell Programming and Scripting

Can somebody advise any free Linux sever for shell programming?

Hi, everybody. I just wonder whether there are a couple of free Linux servers running as terminals where people can practice Unix Shell Programming? I'd like to set up one myself but unfortunatly can't do it. I can't switch to Linux now coz I run a couple of servers on my machine. Cygwin is... (3 Replies)
Discussion started by: belgampaul
3 Replies

Featured Tech Videos