Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Eliminate or ignore asterisks in data when parsing

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Eliminate or ignore asterisks in data when parsing
# 1  
Old 03-31-2017
Eliminate or ignore asterisks in data when parsing
I have data file that has this in it:

data.txt
Code:
.........
.........
PPJ97**2017PPJ97**2017-03-21-13.35.15.887208********************START ERROR LOGGING******************
PPJ97**2017-03-21-13.35.15.887208** PROMPT APPLICATION ERROR **
PPJ97**2017-03-21-13.35.15.887208** IN TIMESTAMP       | 2017-03-21-13.35.15.887208
PPJ97**2017-03-21-13.35.15.887208** OUT TIMESTAMP      | 2017-03-21-13.35.15.896223
PPJ97**2017-03-21-13.35.15.887208** RETURN CODE        | 08
PPJ97**2017-03-21-13.35.15.887208** ERROR KEY          | ed7-371-47e-a4-fff2ce|838101733 965079 M
PPJ97**2017-03-21-13.35.15.887208** ERROR MESSAGE      | MISMATCH IN PREPACK TYPE
.........
.........

I use the following code to only pull out sections of the log if and only if they contain 3 different patterns:

Code:
awk -v p1="START.*ERROR" -v p2="PROMPT.*APPLICATION.*ERROR" -v p3="ERROR.*KEY" ' s!="" { s=s RS $0 ; if($0~p3) { if (s~p2) print s ; count++ ; s="" } } ; $0~p1 { s=$0 } END {print count} ' data.txt

this code works if i alter the data.txt file and get rid of all asterisks "*" and pipes "|" since these can be mis-interpreted by the system or awk command.

however, since i wont be able to alter the data.txt file in a real world scenario (i dont have permissions to), im looking for a way to have my awk command do that before parsing. or of there's a way to have awk ignore the asterisks and pipes altogether, that'll be great.
Last edited by SkySmart; 03-31-2017 at 05:58 PM..
# 2  
Old 03-31-2017
Instead of just showing us your code, please explain what you are trying to do.

In what way are asterisks and vertical bars in your input causing problems?

Counting the number of lines that match one or more of your three patterns is easy. What is the purpose of combining input lines into the string s?

What output are you hoping to produce from your sample input?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk code to ignore the first occurence unknown number of rows in a data column

Hello experts, Shown below is the 2 column sample data(there are many data columns in actual input file), Key, Data A, 1 A, 2 A, 2 A, 3 A, 1 A, 1 A, 1 I need the below output. Key, Data A, 2 A, 2 A, 3 A, 1 A, 1 A, 1 (2 Replies)
Discussion started by: ks_reddy
2 Replies

2. Shell Programming and Scripting

Parsing XML (and insert data) then output data (bash / Solaris)

Hi folks I have a script I wrote that basically parses a bunch of config and xml files works out were to add in the new content then spits out the data into a new file. It all works - apart from the xml and config file format in the new file with XML files the original XML (that ends up in... (2 Replies)
Discussion started by: dfinch
2 Replies

3. Shell Programming and Scripting

Help with parsing data with awk , eliminating unwanted data

Experts , Below is the data: --- Physical volumes --- PV Name /dev/dsk/c1t2d0 VG Name /dev/vg00 PV Status available Allocatable yes VGDA 2 Cur LV 8 PE Size (Mbytes) 8 Total PE 4350 Free PE 2036 Allocated PE 2314 Stale PE 0 IO Timeout (Seconds) default --- Physical volumes ---... (5 Replies)
Discussion started by: rveri
5 Replies

4. Shell Programming and Scripting

Ignore Header and Footer and Sort the data in fixed width file

Hi Experts, I want to Sort the data in fixed width file where i have Header and Footer also in file. I m using below commad to do the sort based on field satarting from 15 position to 17 position , but it is not ignoring the Header and Footer of the file while sorting. In the output i am... (5 Replies)
Discussion started by: sasikari
5 Replies

5. Shell Programming and Scripting

Help in Parsing data

I have below string Transaction_ID:SDP-DM-151204679 , Transaction_DateTime:2011-02-11 00:00:15 GMT+05:30 , Transaction_Mode:WAP , Circle_ID:4 , Circle_Name:BJ ,Zone: , CustomerID:B_31563486 , MSISDN:7870904329 , IMSI:405876122068099 , IMEI: , Sub_Profile:Pre-Paid , CPID:Nazara , CPNAME:Nazara ,... (6 Replies)
Discussion started by: poweroflinux
6 Replies

6. Shell Programming and Scripting

Parsing the data

Hi friends, I need to parse the following data in the given format and get the desired output. I need a function, which takes the input as a parameter and the desired output will be returned from the function. INPUT(single parameter as complete string) A;BCF;DFG;FD ... (3 Replies)
Discussion started by: sumesh.1988
3 Replies

7. Shell Programming and Scripting

Parsing data

Hi all , I have a file with billing CDR records in it. I need to parse that information (row format) . The purpose is to compare full content. The example I have given below is a single line record but it has two portions, (1) the line start with “!” and end with “1.2.1.8” and (2) second part... (5 Replies)
Discussion started by: jaygamini
5 Replies

8. Shell Programming and Scripting

Eliminate redundant data pairs

Hello Experts: I appeal to you to see if you can help me with a small problem. I have a .log file where there is data in two columns (separated by a space). The file is thus: 0.0 3 0.0 6 0.0 6 0.0 6 0.0 7 0.0 7 0.0 7 0.0 7 0.0 11 0.0 11 0.0 11 0.0 11 0.0 11 0.1 17 (6 Replies)
Discussion started by: Flamex
6 Replies

9. Shell Programming and Scripting

Eliminate unwanted data

Hi, I am stuck on writing a script that reads a file, retains wanted data but discards unwanted data from a CSV file. This is a sample of my table {not the actual data) ID Color tel_num Name color2 color3 abcdef green 5551212 jj88 red blue acbdfe yellow... (5 Replies)
Discussion started by: t524ube
5 Replies

10. Shell Programming and Scripting

Parsing the data

Hi I need to parse the following data using shell script Table ----- stage4n_abc 48 stage4o_abcd 4 adashpg_abc_HeartBeat 1 stage4l_asc 168 Can anyone gimme the solution. I want each value to get stored in an array or variable and want the value to be greped from another file.... (1 Reply)
Discussion started by: Archana.Dheepan
1 Replies
Login or Register to Ask a Question