Parsing a column of text file - best practices

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Parsing a column of text file - best practices
# 22  
Old 05-30-2017
Also, please provide the English definitions of what is to be printed in your output following the text: CurrentQtrInput, and PriorQtrInput,.
# 23  
Old 05-30-2017
Hi Don -

Here are the contents of _env.sh:

Code:
#::::::::::::::::::::::::::::
#::-- Set Time Variables --::
#::::::::::::::::::::::::::::

_DAY=$(date +%d)
_MONTH=$(date +%m)
_QUARTER=$(((_MONTH+2)/3))
_YEAR=$(date +%Y)
_DATESTAMP=${_YEAR}${_MONTH}${_DAY}
_HOUR=$(date +%H)
_MINUTE=$(date +%M)
_SECOND=$(date +%S)
_TIME=${_HOUR}${_MINUTE}${_SECOND}
_DATETIMESTAMP=${_DATESTAMP}_${_TIME}

_PREV_DAY=$(date --date yesterday "+%d")
_PREV_MONTH=$(date --date yesterday "+%m")

And yes, you are right, if run during those special occasions, my code would be wrong for quite a few variables. My next task was going to be trying to solve for that.

1. ALLC_CurrentWeek - Day in which script is ran

2.ALLC_CurrentPeriod - Assumes the same behavior of what we spoke about before, your posy #18.

3.PriorQuarterAD - Same concept as post #18.
4.CurrentQtrInput -Same concept as post #18
5.PriorQtrInput - Same concept as post #18.



For instance, if day (Saturday which is the day the script is ran) is 12/31, ALLC_CurrentWeek would be 12/31/16, ALLC_CurrentPeriod would be Jan, and PriorQuarter would be De, CurrentQtrInput would be Jan, and PriorQtrInput would be Dec.


Thank you, Don!
# 24  
Old 05-30-2017
Quote:
Originally Posted by SIMMS7400
Hi Don -

... ... ...

And yes, you are right, if run during those special occasions, my code would be wrong for quite a few variables. My next task was going to be trying to solve for that.

1. ALLC_CurrentWeek - Day in which script is ran

2.ALLC_CurrentPeriod - Assumes the same behavior of what we spoke about before, your posy #18.

3.PriorQuarterAD - Same concept as post #18.
4.CurrentQtrInput -Same concept as post #18
5.PriorQtrInput - Same concept as post #18.



For instance, if day (Saturday which is the day the script is ran) is 12/31, ALLC_CurrentWeek would be 12/31/16, ALLC_CurrentPeriod would be Jan, and PriorQuarter would be De, CurrentQtrInput would be Jan, and PriorQtrInput would be Dec.


Thank you, Don!
You're giving examples again; not definitions. And the strings associated with these values do not correlate with the values you seem to want following them.

You seem to be saying that ALLC_CurrentWeek is a synonym for CurrentWeek. Why have two names for the same thing? If they aren't the same, what is the difference between them?

Note that I say the names don't correlate to the strings because a day (12/31/16) is not a week (12/31/16-01/06/17), but the string you put before the output specifying the Saturday at the start of the week in which you invoke this program is named CurrentWeek (and maybe also ALLC_CurrentWeek). Sometimes you call a month a Period and sometimes you call a month a Mnth and in your examples above you seem to be calling a month a Qtr. Naming something PriorQtrInput and assigning it a value that is an abbreviated month name provides me with no mnemonic relationship.

Please define the above terms in English; not just with examples from which you hope I'll be able to guess at a definition.
# 25  
Old 06-05-2017
Hi Don -

Sorry for the delay - was away on PTO fly-fishing for the last week.

To your comments about certain substitution variables (ALLC_CurrentWeek & CurrentWeek) being synonymous, yes that is correct. Certain target systems required different naming conventions for the subvars, even though each are the same value.

Yes, 12/31/16 is NOT a week, but rather a day. However, this particular client forecasts every week, and therefore, this indicates the start of the current Forecast week.

Mnth and Period are the same thing = Month. Again, the name convention is dictated by which target system they are going to, but they mean the same thing.

PriorQtrInput is the prior quarter (i.e. 1 2 3 or 4)
PriorQtrMnth1= the first month of the previous quarter

1. ALLC_CurrentWeek - The date in MM/DD/YY format that the script is executed. (In the 12/31/16 example, ALLC_CurrentWeek would be 12/31/16)
2. ALLC_CurrentPeriod - The current Month in which the script is executed, unless the next Saturday is greater than or equal to the 2nd day of the month, then ALLC_CurrentPeriod would be that month (In the 12/31/16 example, ALLC_CurrentPeriod would be JAN17.)
3. PriorQuarterAD - Prior quarter number, unless the next Saturday is greater than or equal to the 2nd day of the month AND in the next quarter, then PriorQuarter would actually be the CURRENT quarter (In the 12/31/16 example, PriorQuarterAD would be AD4-16 ...note the "4")
4. CurrentQtrInput - Current quarter number, unless the next Saturday is greater than or equal to the 2nd day of the month AND in the next quarter, then CurrentQtrInput would actually be the next quarter (In the 12/31/16 example, CurrentQtrInput value would be FY 1Q2017_input ...note "1")
5. PriorQtrInput - Same definition as #3 from above (In the 12/31/16 example, PriorQtrInput would be FY 4Q2016_input)

Also, here is the output when I ran it on 5/23/17 (I ran it on a Tuesday, rather than a Saturday, but you get the point):

Code:
CurrentWeek,'\05/23/17\'
CurrentWeekq,'\"05/23/17\"'
CurrentPeriod,MAY17
CurrentPeriodq,'\"MAY17\"'
"1PeriodPrior",APR17
"1PeriodPriorq",'\"APR17\"'
"2PeriodPrior",MAR17
"2PeriodPriorq",'\"MAR17\"'
CurrentQuarter,'\FY 2Q2017\'
CurrentQuarterq,'\"FY 2Q2017\"'
CurrentHalfq,'\"FY 1H2017\"'
CurrentPlanYear,'\FY 2017\'
CurrentPlanYearq,'\"FY 2017\"'
CurrentYear,'\FY 2017\'
CurrentYearq,'\"FY 2017\"'
PriorQuarterAD,'\AD1-17\'
PriorQuarterADq,'\"AD1-17\"'
PriorQuarterMnth1,JAN17
PriorQuarterMnth2,FEB17
PriorQuarterMnth3,MAR17
PriorQuarterMnth1q,'\"JAN17\"'
PriorQuarterMnth2q,'\"FEB17\"'
PriorQuarterMnth3q,'\"MAR17\"'
PriorQuarter,'\FY 1Q2017\'
PriorQuarterq,'\"FY 1Q2017\"'
CurrentQtrInput,'\FY 2Q2017_input\'
CurrentQtrInputq,'\"FY 2Q2017_input\"'
PriorQtrInput,'\FY 1Q2017_input\'
ALLC_CurrentWeek,'\05/23/17\'
ALLC_CurrentWeekq,'\"05/23/17\"'
ALLC_CurrentPeriod,MAY17
ALLC_CurrentPeriodq,'\"MAY17\"'
FirstQtrWeek,'\04/01/2017\'
LastQtrWeek,'\06/24/2017\'

Thank you for all your help, Don! Please let me know if you need any additional clarity. Again, the /\"' symbols encasing the values of each subvar are required to be able to import spaces and special characters.

Last edited by SIMMS7400; 06-05-2017 at 01:04 PM..
# 26  
Old 06-05-2017
Please show us the actual contents of one of the actual files that you have shipped to one of your customers (preferably for the data you would have gotten by running your script on Saturday, May 28, 2017 so we can more easily verify which month and quarter values are based on Calendar week dates and which are based on Fiscal week dates). After playing around with your problem a bit, it seems that calendar week values are based on the date values for the Saturday in the week and fiscal week values are based on the date values for the Friday at the end of the same week. Note that the code I'm playing with always produces data for a Saturday (the current date if I run it on Saturday or give it a date to process that is a Saturday OR the immediately prior Saturday if I run it on any other day of the week or give it a date to process that is not a Saturday), so showing me values showing results for a Tuesday just confuses things.

The quoting you have shown us in post #25 for some of these variable can't possibly be correct. For example, the quoting on:
Code:
CurrentWeek,'\05/23/17\'

with a zero backslash-escaped and one of two single-quotes backslash-escaped makes absolutely no sense to me as a programmer trying to imagine why any parser would want that input. If you have historical data for the files you have produced for your customers, please show us contents of the files you sent out on 05/28/2017 (presumably the last data you sent out), 01/28/2017, and 12/31/2016.
This User Gave Thanks to Don Cragun For This Post:
# 27  
Old 06-05-2017
Hi Don -

Thanks for the reply.

I do not have historical content as I'm going through an effort to automate this as it's currently done manually. Then, I'll enable an archive piece (as you see in the script).

The content does not get shipped to customers. What happens is it produces the content output as shown above. Then, the shell script reads col1 and col2 and imports that into the target system as substitution variable and its value.

The \/' symbols are used to import. The particular engine that does the actual import (maxl) requires those symbols as escape characters to import properly.

So, lets just test one, CurrentWeek.

${_SUBVARPATH}${_YEAR}_${_MONTH}${_DAY}/Subvar_List.txt is created with the following content:

Quote:
CurrentWeek,'\06/05/17\'
Then, the shell script reads col1 and col2, passes those values to a Maxl script as shown here:

Code:
#::-- Read file contents into variables and pass to MaxL --::#

while IFS=',' read col1 col2
do

export subvar=$col2

_MAXLLOGFILE=${_INTRAPATH}/${_FN}_${col1}_MAXL.log

. ${_STARTMAXLPATH} ${_MAXLSCRIPTPATH}Update_Subvars.mxl ${_ESSB_USER} ${_ESSB_PSWD} ${_ESSB_SRVR} ${_MAXLLOGFILE} $col1 $subvar

done < ${_SUBVARPATH}${_YEAR}_${_MONTH}${_DAY}/Subvar_List.txt

The MaxL file (${_MAXLSCRIPTPATH}Update_Subvars.mxl) is as follows:

Code:
/* Spool MaxL output to log file */

spool on to $4;

echo MaxL script parameters;
echo Passed in Subvar       = $5;
echo Passed in Subvar value = $subvar;

/* login <UserName> <Password> on <Essbase Server> */
login $1 $2 on $3;

alter system drop variable $5;
alter system add variable $5 $subvar;

logout;
spool off;
Exit 0;

In the above code, $5 is the subvar (col1) and $subvar is the value (col2).

The log file is as follows:

Code:
---------------------------------------------------------
CS_Subvar_Advancement.sh beginning at 164709
---------------------------------------------------------
---------------------------------------------------------
Prepare Subvars
---------------------------------------------------------
---------------------------------------------------------
Prepare Subvars : Successful
---------------------------------------------------------
---------------------------------------------------------
Advance Subvar Values
---------------------------------------------------------
---------------------------------------------------------
Advance Subvar Values : Successful
---------------------------------------------------------
---------------------------------------------------------
Update Subvars on exalytics-madc-01.server.lan
---------------------------------------------------------
~/Hyperion_Batch/Files/Subvar/2017_0605 ~
~

 Essbase MaxL Shell 64-bit - Release 11.1.2 (ESB11.1.2.4.008B019)
 Copyright (c) 2000, 2016, Oracle and/or its affiliates.
 All rights reserved.

MaxL script parameters

Passed in Subvar       = CurrentWeek

Passed in Subvar value = 06/05/17

MAXL> login admin welcome1 on exalytics-madc-01.server.lan;

 OK/INFO - 1051034 - Logging in user [admin@Native Directory].
 OK/INFO - 1241001 - Logged in to Essbase.

MAXL> alter system drop variable CurrentWeek;


MAXL> alter system add variable CurrentWeek '06/05/17';

 OK/INFO - 1056090 - System altered.

MAXL> logout;

      User admin is logged out


  MaxL Shell completed

---------------------------------------------------------
Update Subvars on exalytics-madc-01.server.lan : Successful
---------------------------------------------------------

---------------------------------------------------------
CS_Subvar_Advancement.sh - Completed Successfully

Normal Exit
---------------------------------------------------------

And then the the import process reflects the subvar and value as follows:

(see attachment).

So again, the \/'" symbols are used as escape characters for the MAXL engine to import correctly with spaces, slashes and double quotes in the value.

Does that make more sense?

Last edited by SIMMS7400; 06-05-2017 at 07:48 PM..
# 28  
Old 06-05-2017
Attachment
Parsing a column of text file - best practices-essbase_subvarpng
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parsing a fixed column text file in sed

I have a text file with records of the form: A X1 Y1 X2 Y2 X3 Y3 where A is character length 10, Xi is character length 4 and Yi is numeric length 10. I want to parse the line, and output records like: A X1 Y1 A X2 Y2 A X3 Y3 etc Can anyone please give me an idea of how to do this. ... (4 Replies)
Discussion started by: wvdeijk
4 Replies

2. Shell Programming and Scripting

Parsing text file

Hi Friends, I am back for the second round today - :D My input text file is this way Home friends friendship meter Tools Mirrors Downloads My Data About Us Help My own results BLAT Search Results ACTIONS QUERY SCORE START END QSIZE IDENTITY CHRO STRAND ... (7 Replies)
Discussion started by: jacobs.smith
7 Replies

3. Shell Programming and Scripting

Parsing text file

I'm totally stumped with how to handle this huge text file I'm trying to deal with. I really need some help! Here is what is looks like: ab1ba67c331a3d731396322fad8dd71a3b627f89359827697645c806091c40b9 0.2 812a3c3684310045f1cb3157bf5eebc4379804e98c82b56f3944564e7bf5dab5 0.6 0.6... (3 Replies)
Discussion started by: comp8765
3 Replies

4. Programming

Parsing a Text file using C++

I was trying to parse the text file, which will looks like this ###XYZABC#### ############ int = 4 char = 1 float = 1 . . ############ like this my text file will contains lots of entries and I need to store these entries in the map eg. map.first = int and map.second = 4 same way I... (5 Replies)
Discussion started by: agupta2
5 Replies

5. UNIX for Dummies Questions & Answers

Replacing a specific column of a text file with another column

Hi, I have a text file in the following format: Code: 13412 NA06985 0 0 2 46.6432798439 4 4 4 4 13412 NA06991 NA06993 NA06985 2 48.8478948517 4 4 2 4 13412 NA06993 0 0 1 45.8022601455 4 4 2 4 13401 NA06994 0 0 1 48.780669145 4 4 4 4 13401 NA07000 0 0 2 47.7312017846 2 4 4 4 ... (2 Replies)
Discussion started by: evelibertine
2 Replies

6. UNIX for Dummies Questions & Answers

Replacing a specific column of a text file with another column

I have a text file in the following format: 13412 NA06985 0 0 2 46.6432798439 4 4 4 4 13412 NA06991 NA06993 NA06985 2 48.8478948517 4 4 2 4 13412 NA06993 0 0 1 45.8022601455 4 4 2 4 13401 NA06994 0 0 1 48.780669145 4 4 4 4 13401 NA07000 0 0 2 47.7312017846 2 4 4 4 13402 NA07019... (3 Replies)
Discussion started by: evelibertine
3 Replies

7. Shell Programming and Scripting

Need help parsing a text file

I have a text file: router1#sh ip blah blah | incl --- Gi2/8 10.60.4.181 --- 10.60.123.175 11 0000 0000 355K Gi2/8 10.60.83.28 --- 224.10.10.26 11 F9FF 3840 154K Gi2/8 10.60.83.198 --- ... (1 Reply)
Discussion started by: streetfighter2
1 Replies

8. Shell Programming and Scripting

Column wise file parsing.

Shell script for the below operation : File "A" contains : SEQ++1' MOA+9:000,00:ABC' RFF+AIK:000000007' FII+PH+0170++AA' NAD+PL+++XXXXXXXXXXX XXXXXXX XX++XXX XXXX XXXX X.X. XXXXXXXXX+++NL' SEQ++2' MOA+9:389,47:ABC' RFF+AIK:02110300000008' FII+PH+0PSTBNL2A:25:5+BB'... (5 Replies)
Discussion started by: navojit dutta
5 Replies

9. Shell Programming and Scripting

Parsing text from file

Any ideas? 1)loop through text file 2)extract everything between SOL and EOL 3)output files, for example: 123.txt and 124.txt for the file below So far I have: sed -n "/SOL/,/EOL/{p;/EOL/q;}" file Here is an example of my text file. SOL-123.go something goes here something goes... (0 Replies)
Discussion started by: ndnkyd
0 Replies

10. Shell Programming and Scripting

Text File Parsing

Hey Guys.I am a newbie on Bash Shell Scripting and Perl.And I have a question about file parsing. I have a log file which contains reports about a communication device.I need to take some of the reports from the log file.Its hard to explain the issue.but shortly I can say that, the reports has a... (2 Replies)
Discussion started by: Djlethal
2 Replies
Login or Register to Ask a Question