Pls review this code and suggest if it can be written in a better way
Pls review this code and provide your feedbacks to make it more efficient.I have tried to add to each section.
Code:
Code
###############################################################
#!/bin/ksh
RRSRC=/test
RREP=/test
#Directories
test_dir=/test
#Imp Files
FILENAME=/test/files.txt
# rename the datasets to test1 and test2 ----Is there a better method to rename at RENAME STEP
llll=test1.sas7bdat
mmmm=test2.sas7bdat
set +x
# Sas prg to be executed
pgm_name=from_tytt
rpt_pgm_name=ppppp
export log=$RLOG/${seq_nbr}_${pgm_name}_${year}${emth}.log
export log1=$RLOG/${seq_nbr}_${rpt_pgm_name}_${year}${emth}.log
#
# read in the ending month as they are used to process the files for that
# Quarter.
#
touch ${log}
touch ${log1}
echo ${pgm_name}
echo ${rpt_pgm_name}
log_date=`date +%m%d%y`
echo ${log_date}
echo "Starting SAS"
#Variable for 2 digit month calculation
a=0
# Create the directory with the eod date this directory will be purged at the completion of the process
eod=`date +%m%d%y`
archive_dir=/test/"$eod"
mkdir $archive_dir
#Read the files for the quarterie if it for the quarter =1 then read files for jan feb march
# the files start with
#'a_d_c'
#'kpp_s'
#'ax_P_Tl_Gr'
#'T_P_al_Gd_At'
#'x_P_Rd'
#'a_P_l_ot_Ng'
# The FILENAME contains these strings and is used to read only these files
# the files are in the string_yyyymmdd_hhmmss.txt format so calculate the mth variable
# to get a two digit number
while IFS= read -r LINE
do
Mth=9
COUNT=1
while [ $COUNT -lt 4 ];
do
if [ $Mth -lt 10 ]; then
Mth="$a$Mth"
fi
# Copy the files tothe temporary archive directory
cp "$test_dir/$LINE"_"$YEAR$Mth"*.txt $archive_dir/.
Mth=`expr $Mth - 1`
COUNT=`expr $COUNT + 1`
done
done < "$FILENAME"
# From the list of files for the quarter
# read each file and create dataset.
# Since each file has different variable so
# pass the number of variables dynamically
# to the Sas prg.Export the file name(obj) and the number of variable parameter(var2)
ls -1 "$archive_dir|while read obj
do
echo "Inside DO $LINE loop"
export obj
echo $obj
if [ "$LINE" = 'a_d_c' ];then var2=var25
elif [ "$LINE" = 'kpp_s' ];then var2=var11
elif [ "$LINE" = 'ax_P_Tl_Gr' ];then var2=var7
elif [ "$LINE" = 'T_P_al_Gd_At' ];then var2=var7
elif [ "$LINE" = 'x_P_Rd' ];then var2=var17
elif [ "$LINE" = 'a_P_l_ot_Ng' ];then var2=var7
fi
echo $LINE
export var2
echo $var2
/sas $RRSRC/$pgm_name.sas -print $RREP/$pgm_name -log $log
return_code=$?
if [ $return_code -gt 1 ]
then
echo "$pgm_name.sas failed -- return code=$return_code"
exit $return_code
fi
done
# For the process we just need to two datasets but we need the latest ones.
#RENAME STEP
do
ls -r1 "$archive_dir/a_d_c_????????_??????_??????.????????|sort -t2 +1 -r|head -1|while read obj
do
mv $obj $llll
done
done
do
ls -r1 "$archive_dir/x_P_Rd_????????_??????_??????.????????|sort -t2 +1 -r|head -1|while read obj
do
mv $obj $mmmmm
done
done
# End of RENAME Step
# read and process further
/sas $RRSRC/$rpt_pgm_name.sas -print $RREP/$rpt_pgm_name -log $log1
return_code=$?
if [ $return_code -gt 1 ]
then
echo "$rpt_pgm_name.sas failed -- return code=$return_code"
exit $return_code
fi
# purge the archive directory
_purge $archive_dir
_purge()
{
# there had better be a directory
if [ ! -d $1 ]; then
echo $1: No such directory 1>&2
return
fi
}
###############################################
use "getopts" to get params from command. Need replace black with a specified string like "%20
DEFAULT_DELIM=%20
...
while getopts dek:f:t:vh OPTION
do
case $OPTION in
t)
DELIM=`tvar=/'"$OPTARG"'/ svar="$DEFAULT_DELIM" awk 'BEGIN{T=ENVIRON;S=ENVIRON; while(index(T,S)!=0){S=S"0"};print... (0 Replies)
I am getting an error "ftpNotes.sh: syntax error at line 8 : `<<' unmatched"
#!/bin/ksh
PATH=/usr/sbin/:/usr/bin:/usr/ucb:/etc:/usr/local/bin:.
cd $HOME
if ;then
if ; then
echo 'DSC file already ftp to epm server'
else
ftp -n epmdev00 <<SCRIPT... (1 Reply)
Started writing my code.
my read input is not even asking nor working?
And I get a EOF script error.
echo "1) aragorn.domain.net"
echo "2) marvel.domain.net"
echo "3) athena.domain.net"
echo "4) gandalf.domain.net"
echo "5) griffin.domain.net"
echo "What server would you like... (4 Replies)
A very n00b question: After compiling and installing software, where does the original source code reside? I'd like to study the source code of some of the ports I've installed. Thanks! :D (1 Reply)
Hi All,
I have made the following code which is taking 10-15 mins to execute completely. Here the file am using is very big( around 1050993843). Can any one make some changes in my code which reduces the time it takes to execute as am very much new to Unix(learner).
One more thing, i have... (10 Replies)
In case you have not seen it, we have a UNIX and Linux Timeline on FB.
If you have a few free moments, could you review the timeline and post back here in this thread with your suggestions and ideas for new entries and milestones?
Thanks! (1 Reply)
Hello mates,
There is a task of Sol 10 installation and configuration of it by customer's demands on a few Solaris boxes (SPARC).
The boxes have different capacity of IDE disks and one root ufs file system per box.
I have completely prepared one of them. It took couple of days.
"Completely... (0 Replies)
Discussion started by: wolfgang
0 Replies
LEARN ABOUT DEBIAN
bup-margin
bup-margin(1) General Commands Manual bup-margin(1)NAME
bup-margin - figure out your deduplication safety margin
SYNOPSIS
bup margin [options...]
DESCRIPTION
bup margin iterates through all objects in your bup repository, calculating the largest number of prefix bits shared between any two
entries. This number, n, identifies the longest subset of SHA-1 you could use and still encounter a collision between your object ids.
For example, one system that was tested had a collection of 11 million objects (70 GB), and bup margin returned 45. That means a 46-bit
hash would be sufficient to avoid all collisions among that set of objects; each object in that repository could be uniquely identified by
its first 46 bits.
The number of bits needed seems to increase by about 1 or 2 for every doubling of the number of objects. Since SHA-1 hashes have 160 bits,
that leaves 115 bits of margin. Of course, because SHA-1 hashes are essentially random, it's theoretically possible to use many more bits
with far fewer objects.
If you're paranoid about the possibility of SHA-1 collisions, you can monitor your repository by running bup margin occasionally to see if
you're getting dangerously close to 160 bits.
OPTIONS --predict
Guess the offset into each index file where a particular object will appear, and report the maximum deviation of the correct answer
from the guess. This is potentially useful for tuning an interpolation search algorithm.
--ignore-midx
don't use .midx files, use only .idx files. This is only really useful when used with --predict.
EXAMPLE
$ bup margin
Reading indexes: 100.00% (1612581/1612581), done.
40
40 matching prefix bits
1.94 bits per doubling
120 bits (61.86 doublings) remaining
4.19338e+18 times larger is possible
Everyone on earth could have 625878182 data sets
like yours, all in one repository, and we would
expect 1 object collision.
$ bup margin --predict
PackIdxList: using 1 index.
Reading indexes: 100.00% (1612581/1612581), done.
915 of 1612581 (0.057%)
SEE ALSO bup-midx(1), bup-save(1)BUP
Part of the bup(1) suite.
AUTHORS
Avery Pennarun <apenwarr@gmail.com>.
Bup unknown-bup-margin(1)