07-21-2014
Thanks Don, i will take your inputs for faster and simple scripts going forward.
Please find here my actual code.
Code :
#!/bin/ksh
#########################################################################
# Date: 25 June 2014 #
# Author : Raju Bolla #
# Email: raju.bolla@hp.com #
# This script will Segregate data from CARRs Germany source files #
# Germany Files For data load activities #
# #
# Sustain will execute this Script as a last pre-load activty step #
# #
#########################################################################
#Function to check the return code of the command executed
chckRC()
{
if [ $1 -ne 0 ]
then
echo "$2" >/dev/null
exit 1
fi
}
##Copy Germany Files to GERMANY Folder
##Copy Germany Files to Greece Folder
#Bkp Path Germany Path
SOURCE_COMMON=$( echo "/pgrrappkg3/DataStage/CONSUMER_BI/datap5/WSLGER_WSLGREECE")
TARGET_GREECE=$( echo "/pgrrappkg3/DataStage/CONSUMER_BI/datap5/WSLGER")
cd $SOURCE_COMMON
cp Crs_*Extract.txt $TARGET_GREECE/
cp Crs_*CTL.txt $TARGET_GREECE/
RC=$?
chckRC $RC "Aborted while executing Copy Germany Files to Greece Folder Part"
echo "Completed the execution of Copy Germany Files to Greece Folder Part" >/dev/null
cd $TARGET_GREECE
config_file_path=$( echo "/pgrrappkg3/DataStage/CONSUMER_BI/etlisp5/cfg")
if [[ -f $config_file_path/crt_extracts_branch_gswid_cntl ]]
then
set -A FILENAME $( cat $config_file_path/crt_extracts_branch_gswid_cntl | grep -v "^#" | awk '{ print $1 }')
set -A COLPOS $( cat $config_file_path/crt_extracts_branch_gswid_cntl | grep -v "^#" | awk '{ print $4 }')
set -A CTLCOLPOS $( cat $config_file_path/crt_extracts_branch_gswid_cntl | grep -v "^#" | awk '{ print $3 }')
SOURCE_DIR=$( echo "/pgrrappkg3/DataStage/CONSUMER_BI/datap5/WSLGER")
x=0
while (( $x < ${#FILENAME[*]} ))
do
if [[ -f $SOURCE_DIR/${FILENAME[$x]} ]]
then
MSG="Processing file $SOURCE_DIR/${FILENAME[$x]}:INFO"
echo $MSG;
PART1FILENAME=$( echo ${FILENAME[$x]} | awk -F"." '{ print $1 }')
PART2FILENAME=$( echo ${FILENAME[$x]} | awk -F"." '{ print $2 }')
NEWFILENAME=$( echo "$SOURCE_DIR/GERMANY_$PART1FILENAME.$PART2FILENAME")
touch $NEWFILENAME
##Reading GERMANY_Obligor_filter_ctl file
##set -x
if [[ -f $config_file_path/GERMANY_Obligor_filter_ctl ]]
then
##this an if then else condition values for command_construction variable
ifcondition_val=$( grep -v "^#" $config_file_path/GERMANY_Obligor_filter_ctl)
command_construction=$( echo "cat $SOURCE_DIR/${FILENAME[$x]} | awk -F\"|\" -v colpos=\"${COLPOS[$x]}\" '{if ("$ifcondition_val") { print \$0 }}' > $NEWFILENAME")
##Executing cunstructed command with EVAL command
eval $command_construction
RC=$?
if [[ $RC -ne 0 ]]
then
MSG="Error in creating file $NEWFILENAME:ALERT"
echo $MSG;
return -1
else
MSG="$NEWFILENAME file created:INFO"
echo $MSG;
fi
fi
##set +x
##CTL File generation
for CTL in $( echo ${CTLCOLPOS[$x]} | awk '{ FldCnt=split($0,Arr,"-"); print FldCnt }' )
do
CTLCTR=1
DISPLAYVAL=""
while (( $CTLCTR <= $CTL ))
do
VAL=$( echo ${CTLCOLPOS[$x]} | awk -v ctr="${CTLCTR}" '{ FldCnt=split($0,Arr,"-"); print Arr[ctr] }')
if [[ $VAL = 0 ]];
then
ROWCNT=$(cat $NEWFILENAME | wc -l )
DISPLAYVAL="$DISPLAYVAL""$ROWCNT|"
else
SUMVAL=$(cat $NEWFILENAME | awk -F"|" -v val="${VAL}" '
BEGIN { tot=0 } {tot=tot+$val } END { printf "%.2f",tot }')
DISPLAYVAL="$DISPLAYVAL""$SUMVAL|"
fi
CTLCTR=$(echo "$CTLCTR+1" | bc)
done
CTLNEWFILENAME=$( echo $NEWFILENAME | sed s/Extract/CTL/g )
CTLORGFILENAME=$( echo ${FILENAME[$x]} | sed s/Extract/CTL/g )
if [[ -f $SOURCE_DIR/$CTLORGFILENAME ]]
then
cat $SOURCE_DIR/$CTLORGFILENAME | head -1 > $CTLNEWFILENAME
else
echo "No header for CTL file found" > $CTLNEWFILENAME
MSG="$CTLORGFILENAME file not found :WARN"
echo $MSG;
fi
echo $DISPLAYVAL | sed s/\|$//g >> $CTLNEWFILENAME
done
MSG="Completed processing file ${FILENAME[$x]}"
echo $MSG;
else
MSG="File not found $SOURCE_DIR/${FILENAME[$x]}. Proceeding to next file:WARN"
echo $MSG;
fi
x=$(echo "$x+1" | bc)
done
else
MSG="Invalid file $config_file_path/crt_extracts_branch_gswid_cntl:ALERT"
echo $MSG;
fi;
## End of Generating Greece and Germany Files GREECE_* and Names
##Modifying Germany Data with Greece data and writing to Sed_*GREE
##Greece Files Path
cd $SOURCE_DIR
##Renaming file to Atcual name
for o in `ls -1 GERMANY_Crs_*_Extract.txt`
do
Actual_FILE_Name=`echo "$o" | sed -e 's/GERMANY\_//'`; ##Indetifying Extract File Name
mv $o $Actual_FILE_Name
RC=$?
chckRC $RC "Aborted while executing Renaming file to Atcual name Part"
echo "Completed the execution of Renaming file to Atcual name Part" >/dev/null
done;
##Renaming file to Atcual name
for p in `ls -1 GERMANY_Crs_*_CTL.txt`
do
Actual_FILE_Name=$( echo "$p" | sed -e 's/GERMANY\_//'); ##Indetifying Extract File Name
mv $p $Actual_FILE_Name
RC=$?
chckRC $RC "Aborted while executing Renaming file to Atcual name Part"
echo "Completed the execution of Renaming file to Atcual name Part" >/dev/null
done;
##set +x
###End of Script
Please find control files data.
Code :
cat GERMANY_Obligor_filter_ctl
##Obligor information file
##For new obligor for search contion add "&& $colpos !=xxxxxx" in format
##make sure all condition values in single line
$colpos != 228070 && $colpos != 228071 && $colpos != 228072 && $colpos != 228073 && $colpos != 228074 && $colpos != 228075 && $colpos != 228076 && $colpos != 228077 && $colpos != 228078 && $colpos != 228079 && $colpos != 228080 && $colpos != 228081 && $colpos != 228082 && $colpos != 228083 && $colpos != 228084 && $colpos != 228085 && $colpos != 228086 && $colpos != 228196 && $colpos != 228299
cat crt_extracts_branch_gsw_cntl
################################################################################
#
# carrs_extracts_branch_gswid_cntl
#
################################################################################
#
# carrs_extracts_branch_cntl is a control file containing carrs extract file names
# and column position of Branch nbr in that extract file
#
################################################################################
#
# Column Definition:
# 1st Column Carrs Extract file name
# 2nd Column Colump position of "Branch Number"
# 4th Column Colump position of "GSW ID or Obligor ID"
##############################################################################################################################################
#Carrs Extract File Column Position Control File column position GSWID Column Possition
##############################################################################################################################################
Crs_Assessment_Extract.txt 4 0-17 3
Crs_Involved_Party_Risk_Rating_Extract.txt 4 0 3
Crs_Model_Input_Extract.txt 7 0-12 6
Crs_Involved_Party_Watchlist_Extract.txt 2 0 1
Crs_Involved_Party_Default_Extract.txt 2 0-9 1
Crs_AR_X_IP_Extract.txt 3 0 4
Crs_Oprtnl_Risk_Assessment_Extract.txt 4 0 3
Crs_Financial_Resource_Item_Extract.txt 11 0-5 8
#Crs_County_Extract.txt 5 0
#Crs_State_Extract.txt 4 0
#Crs_City_Extract.txt 6 0
Crs_Involved_Party_Involved_Party_Extract.txt 6 0 1
Crs_Customer_Obligor_Extract.txt 6 0 15
#Crs_Post_Addr_Extract.txt 13 0
Crs_Involved_Party_Post_Address_Extract.txt 23 0 3
#Crs_IP_Product_Plan_Grp_Rating_Extract.txt 8 0 1
Crs_Financial_Resource_Item_Hdr_Extract.txt 8 0 5
Crs_IP_Credit_Facility_Rating_Extract.txt 2 0-10 1
Crs_Txn_Ev_Dlr_Charges_Extract.txt 1 0-10 2
Please provide your suggestion on above code
Code :
command_construction=$( echo "cat $SOURCE_DIR/${FILENAME[$x]} | awk -F\"|\" -v colpos=\"${COLPOS[$x]}\" '{if ("$ifcondition_val") { print \$0 }}' > $NEWFILENAME")
especially here i am looking for a refine my script and simple use of
Code :
cat GERMANY_Obligor_filter_ctl
file
9 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi Gurus,
In my file I have an amount field from position 74 to 87, which contains values starting with '+' as well as '-'. I want to add all positive values in a varible called "CREDIT" and all negative values in a variable "DEBIT". I know, we can use grep to identify values with positive and... (4 Replies)
Discussion started by: berlin_germany
4 Replies
2. Shell Programming and Scripting
hi i am using printf in a script and it is not printing negative values..i have to use printf to get rid of the newline..here is my code:
fin=`echo $a - $b | bc`
printf "${fin}," >> test
these statements are in a loop. here is what i get when i try to subtract 4 from 8:
./scr1: line... (2 Replies)
Discussion started by: npatwardhan
2 Replies
3. Shell Programming and Scripting
Hello,
I have a list like this :
1
2
-4
0
-3
-7
5
6 etc.
Is there a way to remove all the positive values and print only the negative values, without using grep, sed or awk?
Thanks,
Prasanna (4 Replies)
Discussion started by: prasanna1157
4 Replies
4. Shell Programming and Scripting
Hi all,
I have a file that looks like shown below. I want to find places where the value in column 2 change from negative to positive and vice versa and return the value on column 1 at that point. I wonder if this is possible in shell script or awk .. please help!
Here is the original data
... (6 Replies)
Discussion started by: malandisa
6 Replies
5. UNIX for Dummies Questions & Answers
Ok, so i monitor disk space on remote machines using snmp. Works great for me. But whenever a particular partition happens to have Terabytes of data, snmp starts reporting negative values.
Can someone please tell me how to get around this problem?
The AllocationUnit is 512 bytes. Weird... (0 Replies)
Discussion started by: SkySmart
0 Replies
6. Shell Programming and Scripting
a=10.02
pattern=-11.01
b=$(echo | awk '{ print $a*$pattern}')
echo $b
its not working even
ALso tried `expr $a \* $pattern`
No LUCK (3 Replies)
Discussion started by: saluja.deepak
3 Replies
7. Shell Programming and Scripting
Hi, i need help on replacing negative values in a column with 0. any quick fix on this? thanks much. for instance,
input:
1
2.3
-0.4
-25
12
13
45
-12
desired output
1
2.3
0
0
12
13
45 (4 Replies)
Discussion started by: ida1215
4 Replies
8. Shell Programming and Scripting
Can anyone please assist me?
Please find the attached input and output file for ur reference.
a)Incase if i get negative value (ex:-000100) in the 11th column then i have to convert the value to 0000000(7 zeros-length is 7) and then
print the entire record.
b)Incase if there is no... (2 Replies)
Discussion started by: vinus
2 Replies
9. Shell Programming and Scripting
Hi Gurus,
I have requirement need to sum the value, the logic is if the value is negative then time -1, I tried below two ways. one is failed, another one doesn't work.
awk -F"," '{if($8< 0 $8*-1 else $8) sum+=$8}{print sum, $8} END{printf("%.2f\n",sum)}'
awk -F","... (4 Replies)
Discussion started by: ken6503
4 Replies