Parsing a column of text file - best practices

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Parsing a column of text file - best practices
# 50  
Old 09-26-2017
Hi Don -

Any update? Thank you! Smilie
# 51  
Old 10-09-2017
I sincerely apologize for the lengthy delays. I had hoped to clean everything up with the log files, but I keep running into things where I just don't know what was wanted. So, here is the updated code that calculates and prints your variables. If you invoke your script with no arguments, it will print variables for the date on which the script was run if it is run on a Saturday and for the previous Saturday if it is not run on a Saturday. If you run it with one operand (a date in the format MM/DD/YYYY), and that date is a Saturday it will print variables for that date; if that date is not a Saturday, it will print variables for the Saturday prior to the given date.

Note that the date utility that I have on my system does not have a -d option and the bash on my system does not have ksh93's:
Code:
printf '%(format_string)T' "date_string"

date formatting capabilities. Therefore, the following code was tested using ksh93 instead of bash and a function named date was included in the script to translate the calls to date in this script to calls to ksh93's printf.

Hopefully, you can use the following as a basis for the changes you need to correctly calculate the date variables for your code. First, change _env.sh to:
Code:
#!/bin/bash
#::--:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
#::-- Script Name: _env.sh						--::
#::--									--::
#::-- Description: This environment file is set to hold all variables	--::
#::--              for all Hyperion/Oracle scripts requiring:		--::
#::--									--::
#::--              1. Path variables                                        --::
#::--              2. User ID(s)                                            --::
#::--              3. Logon(s)                                              --::
#::--              4. Password(s)                                           --::
#::--              5. Server(s)                                             --::
#::--              6. Application(s)                                        --::
#::--              7. Database(s)                                          --::
#::--              8. etc                                                  --::
#::--									--::
#::--  Calls:      Not Applicable					--::
#::--  Called By:  All Scripts						--::
#::--									--::
#::-- Parameters:  Not Applicable					--::
#::--									--::
#::-- Author:      Name (Company )					--::
#::-- Date:								--::
#::									--::
#::-- Author:      Date vaiable handling updates by Donald W. Cragun    --::
#::-- Date:	   October 9, 2017					--::
#::									--::
#::--:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

#::::::::::::::::::::::::::::
#::-- Set Path Variables --::
#::::::::::::::::::::::::::::

cd $HOME

_MAINPATH=$(pwd)/Hyperion_Batch/
_LOGPATH=Logs/
_ERRORPATH=Errors/
_FILEPATH=Files/
_FLAGFILEPATH=${_MAINPATH}${_FILEPATH}FlagFiles/
_BATCHPATH=${_MAINPATH}Scripts/Batch/
_MAXLSCRIPTPATH=${_MAINPATH}Scripts/MaxL/
_SUBVARPATH=${_MAINPATH}${_FILEPATH}Subvar/

#:::::::::::::::::::::::::::::::
#::-- Set Essbase Variables --::
#:::::::::::::::::::::::::::::::

_ESSB_USER=user
_ESSB_PSWD=password
_ESSB_SRVR=exalytics-madc-01.server.lan

_STARTMAXLPATH=/u02/EssbaseServer/essbaseserver1/bin/startMaxl.sh

#::::::::::::::::::::::::::::
#::-- Set Time Variables --::
#::::::::::::::::::::::::::::

read _DAY _MONTH _YEAR _HOUR _MINUTE _SECOND <<-EOF
	$(date '+%d %m %Y %H %M %S')
EOF
_DATESTAMP=${_YEAR}${_MONTH}${_DAY}
_TIME=${_HOUR}${_MINUTE}${_SECOND}
_DATETIMESTAMP=${_DATESTAMP}_${_TIME}

read _PREV_DAY _PREV_MONTH _PREV_MONTH_YEAR <<-EOF
	$(date -d "$_MONTH/$_DAY/$_YEAR 1 day ago" '+%d %m %Y')
EOF

#:::::::::::::::::::::::::::::::::::
#::-- Export Relevant Variables --::
#:::::::::::::::::::::::::::::::::::

export _MAINPATH _LOGPATH _ERRORPATH  _FILEPATH _FLAGFILEPATH _SCRIPTPATH _MAXLSCRIPTPATH _SUBVARPATH
export _ESSB_USER _ESSB_PSWD _ESSB_SRVR _STARTMAXLPATH
export _DAY _MONTH _QUARTER _YEAR _DATESTAMP _HOUR _MINUTE _SECOND _TIME _DATETIMESTAMP _PREV_DAY _PREV_MONTH

and then change the first part of CS_Subvar_Advancement.sh to:
Code:
#!/bin/bash
set -u			# abort on expansions of unset variables
IAm=${0##*/}		# basename of executing script
months=(DEC JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC)

# Set path and Essbase variables... 
.  /home/oracle/Hyperion_Batch/Scripts/Batch/_env.sh

# findSaturdayOnOrBefore [date]
# This function finds the date that is the 1st date not later than the given
# date operand (in the format mm/dd/YYYY) or today's date if no date operand is
# given, and sets the following variables that apply to that date:
#	_Y	Full year.
#	_0y	Last two digits of year (00-99).
#	_m	Month number (1-12, no leading zero).
#	_0m	Two digit month number (01-12).
#	_d	Day of month (1-31, no leading zero).
#	_0d	Two digit day of month (01-31).
findSaturdayOnOrBefore() {
	__date=${1:-today}
	# Break down the given date into its component parts.
	read _Y _0y _0m _m _0d _d __dow <<-EOF
		$(date -d "$__date" '+%Y %y %m %-m %d %-d %u')
	EOF
	# date +%u output:  1=Monday, ..., 5=Friday, 6=Saturday, 7=Sunday
	# Is the given date a Saturday?
	if [ $__dow -ne 6 ]
	then	# No.  Adjust the given date to the previous Saturday.
		read _Y _0y _0m _m _0d _d <<-EOF
			$(date -d "$_0m/$_0d/$_Y $((__dow % 7 + 1)) days ago" \
			    '+%Y %y %m %-m %d %-d')
		EOF
	fi
}

# The following can be simplified if this script will only be run on a Saturday
# in the week you want to process.  The code here allows this script to be run
# on any day of the week (in which case it will use the date of the previous
# Saturday if it isn't run on Saturday) or to specify a date as a command line
# operand (in MM/DD/YYYY format) to test processing of edge date conditions).
if [ $# -eq 0 ]
then	# Use today's date.
	findSaturdayOnOrBefore 
else	# Use command-line operand date if it appears to be a valid date.
	printf 'Date given as operand: "%s"\n' "$1"
	if date -d "$1" '+%m/%d/%Y' 2>/dev/null >&2
	then	findSaturdayOnOrBefore "$1"
	else	printf 'Usage: %s [date]\n\t%s\n' \
		    "$IAm" 'where "date" is of the form "MM/DD/YYYY"' >&2
		exit 2
	fi
fi
printf 'Saturday at the start of the week is %s/%s/%s\n' $_0m $_0d $_Y

# Preserve values from this Saturday as Calendar Date values.
C0d=$_0d
C0m=$_0m
C0y=$_0y
CY=$_Y
Cm=$_m

# Now that we have Saturday's Calendar date in the given week, determine the
# Fiscal year and month from the Calendar year and month of Friday at the end
# of this week.
read FY F0y F0m Fm F0d <<-EOF
	$(date -d "$C0m/$C0d/$CY + 6 days" '+%Y %y %m %-m %d')
EOF
printf 'Friday at the end of this week is %s/%s/%s\n' $F0m $F0d $FY

# Calculate fiscal quarter and half.
FQ=$(((Fm + 2) / 3))
FQY=$FY
FQ0y=$F0y
FH=$(((FQ + 1) / 2))
FHY=$FY
FH0y=$F0y
printf 'Therefore, current week is in %sQ%s(%s) and %sH%s(%s)\n' \
    $FQ $FQY $FQ0y $FH $FHY $FH0y

# Calculate next fiscal quarter, previous two fiscal quarters, and previous two
# fiscal months..
FNQ=$(((FQ + 1) - 4 * (FQ == 4)))
FNQY=$((FQY + (FNQ < FQ)))
FNQ0y=$(printf '%02d' $((FNQY % 100)))

FPQ=$(((FQ - 1) + 4 * (FQ == 1)))
FPQY=$((FQY - (FQ < FPQ)))
FPQ0y=$(printf '%02d' $((FPQY % 100)))
FPPQ=$(((FPQ - 1) + 4 * (FPQ == 1)))
FPPQY=$((FQY - (FQ < FPPQ)))
FPPQ0y=$(printf '%02d' $((FPPQY % 100)))

FPm=$(((Fm - 1) + 12 * (Fm == 1)))
FPmY=$((FY - (Fm == 1)))
FPm0y=$(printf '%02d' $((FPmY % 100)))
FPPm=$(((FPm - 1) + 12 * (FPm == 1)))
FPPmY=$((FPmY - (FPm == 1)))
FPPm0y=$(printf '%02d' $((FPPmY % 100)))
printf 'FNQ=%sQ%s(%s)\n' $FNQ $FNQY $FNQ0y
printf 'FPQ=%sQ%s(%s), FPPQ=%sQ%s(%s)\n' $FPQ $FPQY $FPQ0y $FPPQ $FPPQY $FPPQ0y
printf 'Fiscal Month: %s%s(%s)\n' ${months[$Fm]} $FY $F0y
printf 'Previous Fiscal Month: %s%s(%s)\n' ${months[$FPm]} $FPmY $FPm0y
printf '2nd Previous Fiscal Month: %s%s(%s)\n' ${months[$FPPm]} $FPPmY $FPPm0y

# Find the 1st Saturday in this quarter.
findSaturdayOnOrBefore $(printf '%02d' $((FQ * 3 - 2)))/01/$FQY
FQ1stSat0d=$_0d
FQ1stSat0m=$_0m
FQ1stSat0y=$_0y
FQ1stSatY=$_Y
FQ1stSatd=$_d
FQ1stSatm=$_m
printf 'Calculated 1st Saturday of %sQ%s(%s): %s/%s/%s(%s/%s/%s)\n' \
    $FQ $FQY $FQ0y $FQ1stSat0m $FQ1stSat0d $FQ1stSatY \
    $FQ1stSatm $FQ1stSatd $FQ1stSat0y

# Find the last Saturday in this quarter.  Start by finding the 1st Saturday in
# the next quarter and then find the Saturday before that.
findSaturdayOnOrBefore $(printf '%02d' $((FNQ * 3 - 2)))/01/$FNQY
printf 'Calculated 1st Saturday of %sQ%s(%s): %s/%s/%s(%s/%s/%s)\n' \
    $FNQ $FNQY $FNQ0y $_0m $_0d $_Y $_m $_d $_0y
read FQlastSatY FQlastSat0y FQlastSat0m FQlastSatm FQlastSat0d FQlastSatd <<-EOF
	$(date -d "$_0m/$_0d/$_Y 7 days ago" '+%Y %y %m %-m %d %-d')
EOF
printf 'Calculated last Saturday of %sQ%s(%s): %s/%s/%s(%s/%s/%s)\n' \
    $FQ $FQY $FQ0y $FQlastSat0m $FQlastSat0d $FQlastSatY \
    $FQlastSatm $FQlastSatd $FQlastSat0y

printf '\nCalculations are done & Variables are set, print results:\n\n'

printf '"1PeriodPrior",%s%s\n' ${months[$FPm]} $FPm0y
printf "\"1PeriodPriorq\",'\"%s%s\"'\n" ${months[$FPm]} $FPm0y
printf '"2PeriodPrior",%s%s\n' ${months[$FPPm]} $FPPm0y
printf "\"2PeriodPriorq\",'\"%s%s\"'\n" ${months[$FPPm]} $FPPm0y
printf 'ALLC_CurrentPeriod,%s%s\n' ${months[$F0m]} $F0y
printf "ALLC_CurrentPeriodq,'\"%s%s\"'\n" ${months[$F0m]} $F0y
printf "ALLC_CurrentWeek,'%s/%s/%s'\n" $C0m $C0d $C0y
printf "ALLC_CurrentWeekq,'\"%s/%s/%s\"'\n" $C0m $C0d $C0y
#printf "CurrentHalf,'FY %sH%s'\n" $FH $FHY  # Not requested but seems strange.
printf "CurrentHalfq,'\"FY %sH%s\"'\n" $FH $FHY
printf 'CurrentPeriod,%s%s\n' ${months[$F0m]} $F0y
printf "CurrentPeriodq,'\"%s%s\"'\n" ${months[$F0m]} $F0y
printf "CurrentPlanYear,'FY %s'\n" $FQY
printf "CurrentPlanYearq,'\"FY %s\"'\n" $FQY
printf "CurrentQtrInput,'FY %sQ%s_input'\n" $FQ $FQY
printf "CurrentQtrInputq,'\"FY %sQ%s_input\"'\n" $FQ $FQY
printf "CurrentQuarter,'FY %sQ%s'\n" $FQ $FQY
printf "CurrentQuarterq,'\"FY %sQ%s\"'\n" $FQ $FQY
printf "CurrentWeek,'%s/%s/%s'\n" $C0m $C0d $C0y
printf "CurrentWeekq,'\"%s/%s/%s\"'\n" $C0m $C0d $C0y
printf "CurrentYear,'FY %s'\n" $FQY
printf "CurrentYearq,'\"FY %s\"'\n" $FQY
printf "FirstQtrWeek,'%s/%s/%s'\n" $FQ1stSat0m $FQ1stSat0d $FQ1stSatY
printf "LastQtrWeek,'%s/%s/%s'\n" $FQlastSat0m $FQlastSat0d $FQlastSatY
printf "PriorQtrInput,'FY %sQ%s_input'\n" $FPQ $FPQY
printf "PriorQuarter,'FY %sQ%s'\n" $FPQ $FPQY
printf "PriorQuarterq,'\"FY %sQ%s\"'\n" $FPQ $FPQY
printf "PriorQuarterAD,'AD%s-%s'\n" $FPQ $FPQ0y
printf "PriorQuarterADq,'\"AD%s-%s\"'\n" $FPQ $FPQ0y
printf 'PriorQuarterMnth1,%s%s\n' ${months[$(($FPQ * 3 - 2))]} $FPQ0y
printf "PriorQuarterMnth1q,'\"%s%s\"'\n" ${months[$(($FPQ * 3 - 2))]} $FPQ0y
printf 'PriorQuarterMnth2,%s%s\n' ${months[$(($FPQ * 3 - 1))]} $FPQ0y
printf "PriorQuarterMnth2q,'\"%s%s\"'\n" ${months[$(($FPQ * 3 - 1))]} $FPQ0y
printf 'PriorQuarterMnth3,%s%s\n' ${months[$(($FPQ * 3))]} $FPQ0y
printf "PriorQuarterMnth3q,'\"%s%s\"'\n" ${months[$(($FPQ * 3))]} $FPQ0y

Note that I was using the set -u at the start of this script to find and set values for variables you were using that had not been set. You might (or might not) want to remove that line.

Note also that the other lines shown in red in the above script can be removed without affecting the variables set by this script. But, they were very useful while debugging earlier attempts at getting the results you wanted.
This User Gave Thanks to Don Cragun For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parsing a fixed column text file in sed

I have a text file with records of the form: A X1 Y1 X2 Y2 X3 Y3 where A is character length 10, Xi is character length 4 and Yi is numeric length 10. I want to parse the line, and output records like: A X1 Y1 A X2 Y2 A X3 Y3 etc Can anyone please give me an idea of how to do this. ... (4 Replies)
Discussion started by: wvdeijk
4 Replies

2. Shell Programming and Scripting

Parsing text file

Hi Friends, I am back for the second round today - :D My input text file is this way Home friends friendship meter Tools Mirrors Downloads My Data About Us Help My own results BLAT Search Results ACTIONS QUERY SCORE START END QSIZE IDENTITY CHRO STRAND ... (7 Replies)
Discussion started by: jacobs.smith
7 Replies

3. Shell Programming and Scripting

Parsing text file

I'm totally stumped with how to handle this huge text file I'm trying to deal with. I really need some help! Here is what is looks like: ab1ba67c331a3d731396322fad8dd71a3b627f89359827697645c806091c40b9 0.2 812a3c3684310045f1cb3157bf5eebc4379804e98c82b56f3944564e7bf5dab5 0.6 0.6... (3 Replies)
Discussion started by: comp8765
3 Replies

4. Programming

Parsing a Text file using C++

I was trying to parse the text file, which will looks like this ###XYZABC#### ############ int = 4 char = 1 float = 1 . . ############ like this my text file will contains lots of entries and I need to store these entries in the map eg. map.first = int and map.second = 4 same way I... (5 Replies)
Discussion started by: agupta2
5 Replies

5. UNIX for Dummies Questions & Answers

Replacing a specific column of a text file with another column

Hi, I have a text file in the following format: Code: 13412 NA06985 0 0 2 46.6432798439 4 4 4 4 13412 NA06991 NA06993 NA06985 2 48.8478948517 4 4 2 4 13412 NA06993 0 0 1 45.8022601455 4 4 2 4 13401 NA06994 0 0 1 48.780669145 4 4 4 4 13401 NA07000 0 0 2 47.7312017846 2 4 4 4 ... (2 Replies)
Discussion started by: evelibertine
2 Replies

6. UNIX for Dummies Questions & Answers

Replacing a specific column of a text file with another column

I have a text file in the following format: 13412 NA06985 0 0 2 46.6432798439 4 4 4 4 13412 NA06991 NA06993 NA06985 2 48.8478948517 4 4 2 4 13412 NA06993 0 0 1 45.8022601455 4 4 2 4 13401 NA06994 0 0 1 48.780669145 4 4 4 4 13401 NA07000 0 0 2 47.7312017846 2 4 4 4 13402 NA07019... (3 Replies)
Discussion started by: evelibertine
3 Replies

7. Shell Programming and Scripting

Need help parsing a text file

I have a text file: router1#sh ip blah blah | incl --- Gi2/8 10.60.4.181 --- 10.60.123.175 11 0000 0000 355K Gi2/8 10.60.83.28 --- 224.10.10.26 11 F9FF 3840 154K Gi2/8 10.60.83.198 --- ... (1 Reply)
Discussion started by: streetfighter2
1 Replies

8. Shell Programming and Scripting

Column wise file parsing.

Shell script for the below operation : File "A" contains : SEQ++1' MOA+9:000,00:ABC' RFF+AIK:000000007' FII+PH+0170++AA' NAD+PL+++XXXXXXXXXXX XXXXXXX XX++XXX XXXX XXXX X.X. XXXXXXXXX+++NL' SEQ++2' MOA+9:389,47:ABC' RFF+AIK:02110300000008' FII+PH+0PSTBNL2A:25:5+BB'... (5 Replies)
Discussion started by: navojit dutta
5 Replies

9. Shell Programming and Scripting

Parsing text from file

Any ideas? 1)loop through text file 2)extract everything between SOL and EOL 3)output files, for example: 123.txt and 124.txt for the file below So far I have: sed -n "/SOL/,/EOL/{p;/EOL/q;}" file Here is an example of my text file. SOL-123.go something goes here something goes... (0 Replies)
Discussion started by: ndnkyd
0 Replies

10. Shell Programming and Scripting

Text File Parsing

Hey Guys.I am a newbie on Bash Shell Scripting and Perl.And I have a question about file parsing. I have a log file which contains reports about a communication device.I need to take some of the reports from the log file.Its hard to explain the issue.but shortly I can say that, the reports has a... (2 Replies)
Discussion started by: Djlethal
2 Replies
Login or Register to Ask a Question