Parsing a column of text file - best practices

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Parsing a column of text file - best practices
# 8  
Old 05-20-2017
So, the question boils down to what operating system are you running? (If Linux or you have access to the GNU date utility, you can use what Scrutinizer suggested. If not, do you have access to a recent version of the Korn shell (93u+ or later)? If so, you can use ksh's printf built-in's %(format)T format specifier to do the same thing. If not, do you have perl installed. ...) We need to know the environment you're using to tell you how to get what you want as easily as possible.
This User Gave Thanks to Don Cragun For This Post:
# 9  
Old 05-21-2017
Hi Don -

Thank you.

So it's Linux, therefore I've been able to use Scrutinzer's solution with success:

Code:
echo $(date +CurrentWeek,%m/%d/%y)
echo $(date +CurrentWeekq,'"%m/%d/%y"')
echo $(date +CurrentPeriod,%^b%y)
echo $(date +CurrentPeriodq,'"%^b%y"')
echo $(date +1PeriodPrior,%^b%y)
echo $(date +1PeriodPriorq,'"%^b%y"' --date='1 month ago')
echo $(date +2PeriodPrior,%^b%y)
echo $(date +2PeriodPriorq,'"%^b%y"' --date='2 months ago')
echo $(date +CurrentPlanYear,"FY %Y")
echo $(date +CurrentPlanYearq,'"FY %Y"')

SO thank you both!

I do have a few more questions as to how to grab certain values.

Here are the additional variables:

Code:
CurrentQuarter,FY 2Q2017
CurrentQuarterq,"FY 2Q2017"
PriorQuarter,FY 1Q2017
PriorQuarterq,"FY 1Q2017"

PriorQuarterMnth1,JAN17
PriorQuarterMnth2,FAB17
PriorQuarterMnth3,MAR17
PriorQuarterMnth1q,"JAN17"
PriorQuarterMnth2q,"FEB17"
PriorQuarterMnth3q,"MAR17"

CurrentHalf,FY 1H2017
CurrentHalfq,"FY 1H2017"

I have the following code for the Current and Prior quarters, but I need help putting the output in the correct format (as shown above):

Code:
echo $(date +"%Y %m" | awk '{q=int($2/4)+1; printf("%sq%s\n", $1, q);}')
echo $(date +"%Y %m" | awk '{q=int($2/4);y=$1;if (q==0){q=4;y=y-1;}; printf("%sq%s\n", y, q);}')

As you can see, I also need help deriving PriorQuarter months 1-3.

Thank you for all your help!!

---------- Post updated 05-21-17 at 12:30 AM ---------- Previous update was 05-20-17 at 09:04 PM ----------

Hi -

Based on your suggestions, I've been able to acheeive my goal. However, I did need to use some if logic to get get there which helps with PriorQuarter Months 1-3 as well as prior year when in Quarter 1.

Please let me know if there is a better way, as I'm sure there is. Here is my code,
thanks!

_CH= Current Half
_PQ=Prior Quarter
_PQM1= Prior Quarter Month 1

Code:
echo ---------------------------------------------------------                                                                                                
echo "Advance Subvar Values"                                         
echo ---------------------------------------------------------

if [ $(($(date +%m)/4+1)) -eq 1 ]
then
    _CH=1
    _PQ=4
    _PQM1=OCT
    _PQM2=NOV
    _PQM3=DEC
    _SVYEARL=$(date +%Y --date="1 year ago")
    _SVYEARS=$(date +%y --date="1 year ago")
elif [ $(($(date +%m)/4+1)) -eq 2 ]
then
    _CH=1
    _PQM1=JAN
    _PQM2=FEB
    _PQM3=MAR
    _SVYEARL=$(date +%Y)
    _SVYEARS=$(date +%y)
elif [ $(($(date +%m)/4+1)) -eq 3 ]
then
    _CH=2
    _PQM1=APR
    _PQM2=MAY
    _PQM3=JUN
    _SVYEARL=$(date +%Y)
    _SVYEARS=$(date +%y)
elif [ $(($(date +%m)/4+1)) -eq 4 ]
then
    _CH=2
    _PQM1=JUL
    _PQM2=AUG
    _PQM3=SEP
    _SVYEARL=$(date +%Y)
    _SVYEARS=$(date +%y)
else
    _PQ=$(($(($(date +%m)/4+1))-1))
fi

#::-- Read file contents into variables and pass to MaxL --::#

echo $(date +CurrentWeek,%m/%d/%y)
echo $(date +CurrentWeekq,'"%m/%d/%y"')
echo $(date +CurrentPeriod,%^b%y)
echo $(date +CurrentPeriodq,'"%^b%y"')
echo $(date +1PeriodPrior,%^b%y)
echo $(date +1PeriodPriorq,'"%^b%y"' --date='1 month ago')
echo $(date +2PeriodPrior,%^b%y)
echo $(date +2PeriodPriorq,'"%^b%y"' --date='2 months ago')
echo $(date +CurrentQuarter,"FY $(expr $(expr $(date -d '-1 month' +%m) - 1) / 3 + 1)Q%Y")
echo $(date +CurrentQuarterq,'"'"FY $(expr $(expr $(date -d '-1 month' +%m) - 1) / 3 + 1)Q%Y"'"')
echo $(date +CurrentHalfq,'"'"FY ${_CH}H%Y"'"')
echo $(date +CurrentPlanYear,"FY %Y")
echo $(date +CurrentPlanYearq,'"FY %Y"')
echo $(date +CurrentYear,"FY %Y")
echo $(date +CurrentYearq,'"FY %Y"')
echo $(date +PriorQuarterAD,AD${_PQ}-${_SVYEARS})
echo $(date +PriorQuarterADq,'"'"AD${_PQ}-${_SVYEARS}"'"')
echo $(date +PriorQuarterMnth1,"${_PQM1}${_SVYEARS}")
echo $(date +PriorQuarterMnth2,"${_PQM2}${_SVYEARS}")
echo $(date +PriorQuarterMnth3,"${_PQM3}${_SVYEARS}")
echo $(date +PriorQuarterMnth1q,'"'"${_PQM1}${_SVYEARS}"'"')
echo $(date +PriorQuarterMnth2q,'"'"${_PQM2}${_SVYEARS}"'"')
echo $(date +PriorQuarterMnth3q,'"'"${_PQM3}${_SVYEARS}"'"')
echo $(date +PriorQuarter,"FY ${_PQ}Q${_SVYEARL}")
echo $(date +PriorQuarterq,'"'"FY ${_PQ}Q${_SVYEARL}"'"')
echo $(date +CurrentQtrInput,"FY $(expr $(expr $(date -d '-1 month' +%m) - 1) / 3 + 1)Q%Y_input")
echo $(date +CurrentQtrInputq,'"'"FY $(expr $(expr $(date -d '-1 month' +%m) - 1) / 3 + 1)Q%Y_input"'"')
echo $(date +PriorQtrInput,"FY ${_PQ}Q${_SVYEARL}_input")
echo $(date +ALLC_CurrentWeek,%m/%d/%y)
echo $(date +ALLC_CurrentWeekq,'"%m/%d/%y"')
echo $(date +ALLC_CurrentPeriod,%^b%y)
echo $(date +ALLC_CurrentPeriodq,'"%^b%y"')


Last edited by SIMMS7400; 05-20-2017 at 10:55 PM..
# 10  
Old 05-21-2017
Quote:
Originally Posted by SIMMS7400
Hi Don -

Thank you.

So it's Linux, therefore I've been able to use Scrutinzer's solution with success:

Code:
echo $(date +CurrentWeek,%m/%d/%y)
echo $(date +CurrentWeekq,'"%m/%d/%y"')
echo $(date +CurrentPeriod,%^b%y)
echo $(date +CurrentPeriodq,'"%^b%y"')
echo $(date +1PeriodPrior,%^b%y)
echo $(date +1PeriodPriorq,'"%^b%y"' --date='1 month ago')
echo $(date +2PeriodPrior,%^b%y)
echo $(date +2PeriodPriorq,'"%^b%y"' --date='2 months ago')
echo $(date +CurrentPlanYear,"FY %Y")
echo $(date +CurrentPlanYearq,'"FY %Y"')

SO thank you both!

I do have a few more questions as to how to grab certain values.

Here are the additional variables:

Code:
CurrentQuarter,FY 2Q2017
CurrentQuarterq,"FY 2Q2017"
PriorQuarter,FY 1Q2017
PriorQuarterq,"FY 1Q2017"

PriorQuarterMnth1,JAN17
PriorQuarterMnth2,FAB17
PriorQuarterMnth3,MAR17
PriorQuarterMnth1q,"JAN17"
PriorQuarterMnth2q,"FEB17"
PriorQuarterMnth3q,"MAR17"

CurrentHalf,FY 1H2017
CurrentHalfq,"FY 1H2017"

I have the following code for the Current and Prior quarters, but I need help putting the output in the correct format (as shown above):

Code:
echo $(date +"%Y %m" | awk '{q=int($2/4)+1; printf("%sq%s\n", $1, q);}')
echo $(date +"%Y %m" | awk '{q=int($2/4);y=$1;if (q==0){q=4;y=y-1;}; printf("%sq%s\n", y, q);}')

As you can see, I also need help deriving PriorQuarter months 1-3.

Thank you for all your help!!
You seem to need some help calculating the quarter as well as the months in a quarter. Three are three months in a quarter; not 4. If you want to know what calendar quarter a given month is in, try:
Code:
quarter=$(((month+2)/3))

Note that if you're using bash (and some other shells) as your shell, the above calculation will fail in August and September if $month contains a leading zero. If you're using ksh, a leading zero won't be a problem. With the GNU date utility, you can get the month number without a leading zero with:
Code:
month=$(date +%-m)

or with any date utility and any shell that supports parameter substitutions required by the POSIX standards:
Code:
month=$(date +%m)
month=${month#0}

To find months 1, 2, and 3 in a quarter, try:
Code:
month1=$((quarter*3-2))
month2=$((quarter*3-1))
month3=$((quarter*3))

Second note that to work properly, I think you'd need to change two of Scrutinizer's suggestions:
Code:
echo $(date +1PeriodPrior,%^b%y)
echo $(date +1PeriodPriorq,'"%^b%y"' --date='1 month ago')
echo $(date +2PeriodPrior,%^b%y)
echo $(date +2PeriodPriorq,'"%^b%y"' --date='2 months ago')

to:
Code:
echo $(date +1PeriodPrior,%^b%y --date='1 month ago')
echo $(date +1PeriodPriorq,'"%^b%y"' --date='1 month ago')
echo $(date +2PeriodPrior,%^b%y) --date='2 months ago'
echo $(date +2PeriodPriorq,'"%^b%y"' --date='2 months ago')

# 11  
Old 05-21-2017
Hi Don -

Thank you!

Where are you seeing 4months in a quarter for 3?

My if logic is checking what quarter I'm currently in:

Code:
if [ $(($(date +%m)/4+1)) -eq 1 ]
then

Is that what you meant?

---------- Post updated at 07:59 AM ---------- Previous update was at 07:39 AM ----------

Also, what would be the best way to derive these two values?

Code:
FirstQtrWeek,12/31/16
LastQtrWeek,03/25/17

My week starts on a Saturday.

Thanks!
# 12  
Old 05-21-2017
Quote:
Originally Posted by SIMMS7400
Hi Don -

Thank you!

Where are you seeing 4months in a quarter for 3?

My if logic is checking what quarter I'm currently in:

Code:
if [ $(($(date +%m)/4+1)) -eq 1 ]
then

Is that what you meant?
Hi SIMMS7400,
No. That code had not been presented in the mail to which I responded. But we can use it for discussion now. Although the code above happens to work because there is no month zero. Let's look at the tests for the other three quarters:
Code:
if [ $(($(date +%m)/4+1)) -eq 1 ]
... # Happens to work for months 01, 02, and 03
elif [ $(($(date +%m)/4+1)) -eq 2 ]
... # Treats months 04, 05, 06, and 07 as 2nd quarter.
elif [ $(($(date +%m)/4+1)) -eq 3 ]
... # With ksh, treats months 08, 09, 10, and 11 as 3rd quarter.
... # With bash, treats months 08 and 09 as syntax errors and treats months
... #   10 and 11 as 3rd quarter.
elif [ $(($(date +%m)/4+1)) -eq 4 ]
... # With ksh, treats month 12 as 4th quarter.
... # With bash, treats 08 and 09 as syntax errors and treats 12 as 4th quarter.

Everything marked in red in the comments above look wrong to me! Do you believe that October and November should be in 3rd quarter instead of the 4th quarter? Do you believe that July should be in the 2nd quarter instead of the 3rd quarter? What shell are you using? If you're using bash, do you believe that August and September should not be in any quarter in a calendar year?

I believe that Q1 should be months January, February, and March; Q2 should be months April, May, and June; Q3 should be months July, August, and September; and Q3 should be months October, November, and December. That is not what your code does!
Quote:
---------- Post updated at 07:59 AM ---------- Previous update was at 07:39 AM ----------

Also, what would be the best way to derive these two values?

Code:
FirstQtrWeek,12/31/16
LastQtrWeek,03/25/17

My week starts on a Saturday.

Thanks!
Define your terms!
Is FirstQtrWeek supposed to be the 1st Saturday before January 2, of the current year?

Is LastQtrWeek supposed to be the 1st Saturday before XX/02/YYYY where XX is the 1st month of the current quarter in calendar year YYYY?

And, I repeat, what shell are you using?
# 13  
Old 05-21-2017
Hi DOn -

Thank you for the replies! I'm operating on a AIX environment.

Since the quarter syntax is whacky, what would you suggest instead? My thoughts are to still utilize the if/elif logic, but instead use month instead. While there would be more conditions using month, it seems safer.

As far as :
FirstQtrWeek : The script should determine if it it the first Saturday of the 1st quarter month. If so, set it. If not, it needs to refer to that week until the next quarter.

& LastQtrWeek,the script should determine if it is the 3rd quarter month of the current quarter, and then determine if it is the last Saturday.

I hope I explained that well enough?

---------- Post updated at 08:50 PM ---------- Previous update was at 08:06 PM ----------

I wrote the quarter derivation portion like this:

Code:
if [[ $(date +%-m) -ge 1 && $(date +%-m) -le 3 ]]
then
    _CH=1
    _PQ=4
    _PQM1=OCT
    _PQM2=NOV
    _PQM3=DEC
    _SVYEARL=$(date +%Y --date="1 year ago")
    _SVYEARS=$(date +%y --date="1 year ago")
elif [[ $(date +%-m) -ge 4 && $(date +%-m) -le 6 ]]
then
    _CH=1
    _PQM1=JAN
    _PQM2=FEB
    _PQM3=MAR
    _SVYEARL=$(date +%Y)
    _SVYEARS=$(date +%y)
    _PQ=$(($(($(date +%-m)/4+1))-1))
elif [[ $(date +%-m) -ge 7 && $(date +%-m) -le 9 ]]
then
    _CH=2
    _PQM1=APR
    _PQM2=MAY
    _PQM3=JUN
    _SVYEARL=$(date +%Y)
    _SVYEARS=$(date +%y)
    _PQ=$(($(($(date +%-m)/4+1))-1))
elif [[ $(date +%-m) -ge 10 && $(date +%-m) -le 12 ]]
then
    _CH=2
    _PQM1=JUL
    _PQM2=AUG
    _PQM3=SEP
    _SVYEARL=$(date +%Y)
    _SVYEARS=$(date +%y)
    _PQ=$(($(($(date +%-m)/4+1))-1))
fi

# 14  
Old 05-22-2017
Hi SIMMS7400,
I will have to assume that you have everything under control and do not desire any further help from me. I have asked three times what shell you're using and have not received an answer. Therefore, I am unable to post a good suggested suited to your environment.

I suggested a simple way to calculate the calendar quarter from the month number (in post #10) and in that same post showed simple ways to calculate the month numbers for the three months in that quarter. Instead of using that code, you used other code that clearly calculates quarters incorrectly. (And, the code you showed us in the 1st update to post #13 still uses that incorrect calculation three out of four times when assigning values to _PQ.)

It is clear from your examples, that your definition of FirstQtrWeek and LastQtrWeek is a date that is not in that quarter with the possible exception of cases where the 1st day of a calendar quarter happens to be a Saturday. Your refusal to clearly define how those two values are calculated (instead of just giving ambiguous examples) makes it impossible to suggest code that might meet your requirements. Given that you say that 12/31/2016 is in Q12017 while your calculations (which are purely based on the month number) clearly compute that anything in December is in Q4; I have great difficulty correlating your stated requirements to your code.

I hope that some of my comments have been helpful. I won't bother you with more questions.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parsing a fixed column text file in sed

I have a text file with records of the form: A X1 Y1 X2 Y2 X3 Y3 where A is character length 10, Xi is character length 4 and Yi is numeric length 10. I want to parse the line, and output records like: A X1 Y1 A X2 Y2 A X3 Y3 etc Can anyone please give me an idea of how to do this. ... (4 Replies)
Discussion started by: wvdeijk
4 Replies

2. Shell Programming and Scripting

Parsing text file

Hi Friends, I am back for the second round today - :D My input text file is this way Home friends friendship meter Tools Mirrors Downloads My Data About Us Help My own results BLAT Search Results ACTIONS QUERY SCORE START END QSIZE IDENTITY CHRO STRAND ... (7 Replies)
Discussion started by: jacobs.smith
7 Replies

3. Shell Programming and Scripting

Parsing text file

I'm totally stumped with how to handle this huge text file I'm trying to deal with. I really need some help! Here is what is looks like: ab1ba67c331a3d731396322fad8dd71a3b627f89359827697645c806091c40b9 0.2 812a3c3684310045f1cb3157bf5eebc4379804e98c82b56f3944564e7bf5dab5 0.6 0.6... (3 Replies)
Discussion started by: comp8765
3 Replies

4. Programming

Parsing a Text file using C++

I was trying to parse the text file, which will looks like this ###XYZABC#### ############ int = 4 char = 1 float = 1 . . ############ like this my text file will contains lots of entries and I need to store these entries in the map eg. map.first = int and map.second = 4 same way I... (5 Replies)
Discussion started by: agupta2
5 Replies

5. UNIX for Dummies Questions & Answers

Replacing a specific column of a text file with another column

Hi, I have a text file in the following format: Code: 13412 NA06985 0 0 2 46.6432798439 4 4 4 4 13412 NA06991 NA06993 NA06985 2 48.8478948517 4 4 2 4 13412 NA06993 0 0 1 45.8022601455 4 4 2 4 13401 NA06994 0 0 1 48.780669145 4 4 4 4 13401 NA07000 0 0 2 47.7312017846 2 4 4 4 ... (2 Replies)
Discussion started by: evelibertine
2 Replies

6. UNIX for Dummies Questions & Answers

Replacing a specific column of a text file with another column

I have a text file in the following format: 13412 NA06985 0 0 2 46.6432798439 4 4 4 4 13412 NA06991 NA06993 NA06985 2 48.8478948517 4 4 2 4 13412 NA06993 0 0 1 45.8022601455 4 4 2 4 13401 NA06994 0 0 1 48.780669145 4 4 4 4 13401 NA07000 0 0 2 47.7312017846 2 4 4 4 13402 NA07019... (3 Replies)
Discussion started by: evelibertine
3 Replies

7. Shell Programming and Scripting

Need help parsing a text file

I have a text file: router1#sh ip blah blah | incl --- Gi2/8 10.60.4.181 --- 10.60.123.175 11 0000 0000 355K Gi2/8 10.60.83.28 --- 224.10.10.26 11 F9FF 3840 154K Gi2/8 10.60.83.198 --- ... (1 Reply)
Discussion started by: streetfighter2
1 Replies

8. Shell Programming and Scripting

Column wise file parsing.

Shell script for the below operation : File "A" contains : SEQ++1' MOA+9:000,00:ABC' RFF+AIK:000000007' FII+PH+0170++AA' NAD+PL+++XXXXXXXXXXX XXXXXXX XX++XXX XXXX XXXX X.X. XXXXXXXXX+++NL' SEQ++2' MOA+9:389,47:ABC' RFF+AIK:02110300000008' FII+PH+0PSTBNL2A:25:5+BB'... (5 Replies)
Discussion started by: navojit dutta
5 Replies

9. Shell Programming and Scripting

Parsing text from file

Any ideas? 1)loop through text file 2)extract everything between SOL and EOL 3)output files, for example: 123.txt and 124.txt for the file below So far I have: sed -n "/SOL/,/EOL/{p;/EOL/q;}" file Here is an example of my text file. SOL-123.go something goes here something goes... (0 Replies)
Discussion started by: ndnkyd
0 Replies

10. Shell Programming and Scripting

Text File Parsing

Hey Guys.I am a newbie on Bash Shell Scripting and Perl.And I have a question about file parsing. I have a log file which contains reports about a communication device.I need to take some of the reports from the log file.Its hard to explain the issue.but shortly I can say that, the reports has a... (2 Replies)
Discussion started by: Djlethal
2 Replies
Login or Register to Ask a Question