Shell Programming and Scripting

Hi,

I have a text with a number of paragraphs in them. My problem is I need to locate certain errors/warning and extract/count them. Problem is I do not know how many paras are there with that particular type of error/warning. I had thought that somehow if I could count the number of paras/blocks in the complete text file-> then extract all blocks/paras with a particular type of warning/error so that it would lessen those lines/blocks/paras from the original text file, it would ultimately give me a count of 0 in the original text file, but whatever I have found in google related to perl or bash, it is just confusing me since I know little of actual scripting.

I am on
Linux 2.6.18-417.el5 #1 SMP Sat Nov 19 14:54:59 EST 2016 x86_64 x86_64 x86_64 GNU/Linux

I have also included the complete text file for the gurus

Thanks a lot again.

Hi,

If these errors or warnings can only ever occur once per each section of your input file, then all you'd need to do is search for all instances of those errors or warnings and count how many you've found. That would then tell you how many sections contained these errors. If they can occur multiple times per section of course that would complicate things.

If you can provide information on what these error/warning messages in your input are expected to look like, and if they will only ever appear once per paragraph, that would be a good way forward for starters. As of just now you haven't actually said exactly what it is in the file that constitutes the warning/error that you're interested in.

Hi @drysdalk,

Sorry about that. totally missed to write about the specific errors/warnings.

1. Each block starts with a BEGIN MESSAGE and an END MESSAGE.

2. Each block would have just one error or warning

3. Each error/warning message is preceded by a Warning(space)! or Error!

Eg, 'Original Transaction Not Found !' -> This particular error message would occur only once in a block.

The challenge I am facing is how do I find out what that error/warning message text is.

Hi,

OK, thanks. Are you just wanting to count up how many errors/warnings there are for informational purposes, or do you need to find all blocks that contain these and print them out in their entirety ?

Hi,

To the point -> I am actually looking for the entire block which gives me that error/warning.

Here is what people in my office normally do using Windows -> Open up notepad++, search for a warning/error text -> on the first occurrence that is found, we cut that block out and move it to a new file; thereby reducing the overall count in the original file. Every occurrence of that error/warning in the original file, we cut that block out, and paste it under the new file was opened. This way each new file contains only those errors/warnings. This gives us the count of the error messages as well as a sorted output since each new file only contains those particular errors/warnings.

And then when that is done, we do other manual thing of finding the (tab)Institution Number: and the Acquirer Reference: but that is a totally different requirement.

So you what I meant. It's just a long and frustrating way of finding out information which can be repetitive and mistake ridden as well.

Sorry to be throwing all of the information at once. But I am just tired of this manual way. And thanks for all your help

Hi,

This is an adaptation of the script I provided to your question from the other week, which I think will do what you need.

Code:

#!/bin/bash

input=EXTRN071_copy.txt
tmp=/tmp/script.tmp

while read -r line
do
        case "$line" in
                *BEGIN\ MESSAGE*)
                        unset print
                        echo "$line" > "$tmp"
                        ;;
                *END\ MESSAGE*)
                        echo "$line" >> "$tmp"

                        if [ "$print" == "1" ]
                        then
                                /bin/cat "$tmp"
                                echo
                        fi
                        ;;
                Warning*|Error*)
                        print=1
                        echo "$line" >> "$tmp"
                        ;;
                *)
                        echo "$line" >> "$tmp"
                        ;;
        esac
done < "$input"

So the basic idea is:

Hope this does the trick. If not, let me now and I'll have another crack at it.

EDIT: If you need to preserve the spaces, tabs and other formatting at the start of the lines before the text begins, add a line like this at the top of the script after the shebang line:

IFS='' (that's two single-quotes, and not a double-quote)

Thanks. You're a genius. I remember the other script you gave me and I did try to fiddle around with it, but always messed up trying to find error and warning messages.

The output does print out all the errors and warning. Now if I wanna find out the particular type of warning/errors and their related blocks, do I need to copy the output and move it to a file and then do the related search? Is there a way to sort a complete block, like i would sort lines and uniq and then count it by a wc -l. Im sorry if I am asking a lot, but since you give me outputs in a jiffy, i thought I'll take the risk