Unix script enhancement


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Unix script enhancement
# 1  
Old 10-19-2010
Unix script enhancement

Hi,

Please help me out with this new enhancment with this shell script which i have written below.
********************************************************
Code:
#!/bin/ksh
#File contains Only New Records that sent in the Current Load
export ORACLE_BASE=/oracle
export ORACLE_HOME=/oracle/product/11.2.0
export LIBPATH=$ORACLE_HOME/lib:$ORACLE_HOME/lib32:/usr/lib
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/usr/lib
export TNS_ADMIN=$ORACLE_HOME/network/admin
export PATH=$PATH:$ORACLE_HOME/bin

dbfile=/space/dbexport/PHR/data/eligExport.csv

ControlFileNm="/space/dbexport/PHR/control/bene_eligibility.ctl"
logfile="/space/dbexport/PHR/log/bene_eligibility-`date +%Y%m%d`.log"
#COUNT=$(FILECOUNT=$(find /space/dbexport/PHR/data/eligExport.csv -type f -mtime -1 -print | wc -l))
#STATE1="The total number of counts of the input files is: $(COUNT)"
touch $logfile
chmod 777 $logfile
#Checks for any duplicate records in the file
diff -f /space/dbexport/PHR/data/eligExport.csv /space/dbexport/PHR/data/eligExport-20101018.csv
#sort -d -f -u -k 1.${start},1.${end} ${file}
#sort -d -f -u ${file} > ${LOG}
 
#Checks whether the Input file exists. If file found the load process starts else throws error that file not found
if [ -f $dbfile ]
then
echo 'The input file "eligExport.csv" exists, and the data has been loaded'
#mv /space/dbexport/PHR/data/eligExport.csv /space/dbexport/PHR/archive/eligExport-`date +%Y%m%d`.csv
#chmod 777 /space/dbexport/PHR/archive/eligExport-`date +%Y%m%d`.csv
/space/dbexport/PHR/log/bene_eligibility-`date +%Y%m%d`.log ; cat dbfile | wc -l | read linecount ; echo $linecount | to=id11355@noridian.com
basename=`basename $logfile`
subject="BENE ELIGIBILITY TABLE LOAD STATUS"
(echo 'THE INPUT FILE "eligExport.csv" EXISTS AND THE DATA HAS BEEN LOADED' ; uuencode $logfile $basename ) | mailx -s "$subject" "$to"
else
 echo 'The input file does not exist in the folder'
/space/dbexport/PHR/log/bene_eligibility-`date +%Y%m%d`.log | to=id11355@noridian.com
subject="BENE ELIGIBILITY TABLE DID NOT LOAD AS THERE IS NO DATA FILE PRESENT"
(echo 'DATA LOAD FAILED AS THERE IS NO INPUT FILE PRESENT IN THE FOLDER' ; uuencode $logfile $basename ) | mailx -s "$subject" "$to"
fi
if [ $(hostname) = elect ]
then
export userName="zx01182"
export ORACLE_SID=db86
export pwd_file=db86zx01182pw
elif [ $(hostname) = chi ]
then
export userName="zx01204"
export ORACLE_SID=db87
export pwd_file=db87zx01204pw
elif [ $(hostname) = zack ]
then
export userName="zx01205"
export ORACLE_SID=db88
export pwd_file=db88zx01205pw
else exit
fi
/usr/local/pl/util_scripts/perlencrypt.pl -k /export/home/oracle/secure/ealgorithm -d /sdr1/system/$pwd_file | sqlplus $userName @/space/dbexport/scripts/tru
ncate_phr_bene_1.sql
/usr/local/pl/util_scripts/perlencrypt.pl -k /export/home/oracle/secure/ealgorithm -d /sdr1/system/$pwd_file | sqlldr $userName  control=$ControlFileNm log=$
logfile bad=/space/dbexport/PHR/bad/bene_eligibility-`date +%Y%m%d`.bad

****************************************************

I need the following enhancement from this:
1) The duplicate records exist in file ven_clm_data_20100901449.csv
If the input file contains any duplicate records, it should e-mail the user with the file name.
2) The total number of records in the file is 75

Should be able to count the total no. of records in the input file and e-mail the user
3) The duplicate count within the incoming file is 0.

If there are no duplicate records, should e-mail the user. This should be done aftere reading the input file.
4) The total number of records loaded into the VEN_CLM table is 0.

This is when there are duplicate records in the input file,, the user should get the message that no records are loaded as there are duplicate records in the file.

Please help me out with this, as this is very critical.

Regards
Rahman...

Last edited by DukeNuke2; 10-19-2010 at 06:30 PM..
# 2  
Old 10-19-2010
1) compare $( wc -l <file ) to $( sort -u file | wc -l ) to detect dups, echo "Message $variables more message!" | mailx -s subject-line email_addr@their_host

2) echo "Message $variables more message $( wc -l <file ) !" | mailx -s subject-line email_addr@their_host

3) see 1.

4) if ( rows affected from whatever db tool output) == 0 != ( wc -l file ) then echo "Message $variables more ( rows affected from whatever db tool output) message $( wc -l <file ) !" | mailx -s subject-line email_addr@their_host

BTW, Date/Time stamps in the email subject are nice. Host names or something to identify test versus production is very good, too.
# 3  
Old 10-19-2010
Tried point No. 2.

I tried the second point. It loads the data, and e-mails the log file to the user, but it does not e-mail the total no. of files loaded in the table to the user.

Instead it diplays the result on the background itself.
Code:
 
Message  more message       95 !
SQL*Plus: Release 11.2.0.1.0 Production on Tue Oct 19 16:07:37 2010
Copyright (c) 1982, 2009, Oracle.  All rights reserved.
Enter password:
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
drop index zx01193.bene_elig
                   *
ERROR at line 1:
ORA-01418: specified index does not exist
 
PL/SQL procedure successfully completed.
SQL> Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
Password:
SQL*Loader: Release 11.2.0.1.0 - Production on Tue Oct 19 16:07:37 2010
Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.
Commit point reached - logical record count 64
Commit point reached - logical record count 95

Here's the script that i added to the current script:
Code:
echo "Message $variables more message $(wc -l </space/dbexport/PHR/data/eligExport.csv ) !" ; uuencode $logfile $basename  | mailx -s "$subject" "$to"

Can you please do the changes to the above script, which will e-mail the total records loaded into the table via e-mail to the user.

Thanks

---------- Post updated at 05:03 PM ---------- Previous update was at 04:11 PM ----------

Tried Point No.1.
Table loaded with data,, but the e-mail does not e-mail the user that the input file contains duplicate records in it.

Thoughts please,,
Code:
compare
$( wc -l </space/dbexport/PHR/data/eligExport.csv ) to $( sort -u /space/dbexport/PHR/data/eligExport.csv | wc -l )
echo "Message $variables more message!" ; uuencode $logfile $basename  | mailx -s subject-line abc@gmail.com


Last edited by Scott; 10-26-2010 at 06:06 PM..
# 4  
Old 10-19-2010
You want the output of the echo and the uuencode to be sent to mailx, so put them in ( ) That way both commands run (1 after the other) and their combined output is piped on:

Code:
 ( echo "Message $variables more message $(wc -l </space/dbexport/PHR/data/eligExport.csv ) !" ; uuencode $logfile $basename ) | mailx -s "$subject" "$to"

# 5  
Old 10-20-2010
One success, another to go....

Thanks Chubler,
It worked. I am getting the output like this :
Message The total number of records in the file is: 143 !
along with the log file.

I am also trying to compare two files for any duplicate records in it. I used this command, but its not working. I need this script to compare two file of the same type,, but if any duplicate records in file B, then the user should be intimated by e-mail that the file B contains duplicate records.

The script that i am working is :
Code:
cmp $( wc -l </space/dbexport/PHR/data/eligExport.csv ) to $( sort -u /space/dbexport/PHR/data/eligExport-20101019.csv | wc -l )
( echo "Message: $variables The total number of records in the input file is: $(wc -l </space/dbexport/PHR/data/eligExport.csv ) !" ; u
uencode $logfile $basename ) | mailx -s "$subject" "$to"

Please let me know how to compare two files and send an e-mail to the user if the file B contains any duplicate records in it.

Thanks,
Rahman

Last edited by Scott; 10-26-2010 at 06:07 PM.. Reason: Code tags, please...
# 6  
Old 10-20-2010
This will check if any record is duplicated in the combined fileA+fileB, is this what you were after?

Code:
DUP_RECORDS=$( cat fileA fileB | sort | uniq -d | wc -l )
 
if [[ $DUP_RECORDS > 0 ]]
then
    # mailx command here
fi

# 7  
Old 10-20-2010
One email per file with duplicates?

Code:
for f in fileA fileB
do
 sort $f | uniq -d | wc -l | read DUP_REC_CT
 if (( $DUP_REC_CT > 0 ))
 then
    # mailx command here
 fi
done

Login or Register to Ask a Question

Previous Thread | Next Thread

3 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Pagination need enhancement.

I have below folder structure . |--summit | |----xpnldetails | |----trades | |----svar | | |------checksum | |----xpnl | |----adjustment | | |------summit | | |------ap | |----risks | |----hypopnl | |----fxeffect | |----audittrail |... (0 Replies)
Discussion started by: manas_ranjan
0 Replies

2. UNIX for Dummies Questions & Answers

Linux Multipath Enhancement

Hi.. when i delete a Lun from a Array on a SAN, and again when i create a Lun it shows that the newly created Lun has the ID of the Old deleted Lun which is leading to data corruption. All this is handled by the Multipath software. can anybody help. (0 Replies)
Discussion started by: Praveen13
0 Replies

3. UNIX for Advanced & Expert Users

Linux Multipathing Enhancement

We are searching for DEVICE MAPPER TABLE FORMAT and how to access it through user space?? We are working on enhancement in multipatrhing as 1] scalable path testing 2] event mechanism 3] i/o load sharing. So we need to access device info as well as all paths to that device, devices status,paths... (1 Reply)
Discussion started by: rajaryan99
1 Replies
Login or Register to Ask a Question