Sponsored Content
Top Forums Shell Programming and Scripting Eliminate or ignore asterisks in data when parsing Post 302995121 by SkySmart on Friday 31st of March 2017 04:51:23 PM
Old 03-31-2017
Eliminate or ignore asterisks in data when parsing

I have data file that has this in it:

data.txt
Code:
.........
.........
PPJ97**2017PPJ97**2017-03-21-13.35.15.887208********************START ERROR LOGGING******************
PPJ97**2017-03-21-13.35.15.887208** PROMPT APPLICATION ERROR **
PPJ97**2017-03-21-13.35.15.887208** IN TIMESTAMP       | 2017-03-21-13.35.15.887208
PPJ97**2017-03-21-13.35.15.887208** OUT TIMESTAMP      | 2017-03-21-13.35.15.896223
PPJ97**2017-03-21-13.35.15.887208** RETURN CODE        | 08
PPJ97**2017-03-21-13.35.15.887208** ERROR KEY          | ed7-371-47e-a4-fff2ce|838101733 965079 M
PPJ97**2017-03-21-13.35.15.887208** ERROR MESSAGE      | MISMATCH IN PREPACK TYPE
.........
.........

I use the following code to only pull out sections of the log if and only if they contain 3 different patterns:

Code:
awk -v p1="START.*ERROR" -v p2="PROMPT.*APPLICATION.*ERROR" -v p3="ERROR.*KEY" ' s!="" { s=s RS $0 ; if($0~p3) { if (s~p2) print s ; count++ ; s="" } } ; $0~p1 { s=$0 } END {print count} ' data.txt

this code works if i alter the data.txt file and get rid of all asterisks "*" and pipes "|" since these can be mis-interpreted by the system or awk command.

however, since i wont be able to alter the data.txt file in a real world scenario (i dont have permissions to), im looking for a way to have my awk command do that before parsing. or of there's a way to have awk ignore the asterisks and pipes altogether, that'll be great.

Last edited by SkySmart; 03-31-2017 at 05:58 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parsing the data

Hi I need to parse the following data using shell script Table ----- stage4n_abc 48 stage4o_abcd 4 adashpg_abc_HeartBeat 1 stage4l_asc 168 Can anyone gimme the solution. I want each value to get stored in an array or variable and want the value to be greped from another file.... (1 Reply)
Discussion started by: Archana.Dheepan
1 Replies

2. Shell Programming and Scripting

Eliminate unwanted data

Hi, I am stuck on writing a script that reads a file, retains wanted data but discards unwanted data from a CSV file. This is a sample of my table {not the actual data) ID Color tel_num Name color2 color3 abcdef green 5551212 jj88 red blue acbdfe yellow... (5 Replies)
Discussion started by: t524ube
5 Replies

3. Shell Programming and Scripting

Eliminate redundant data pairs

Hello Experts: I appeal to you to see if you can help me with a small problem. I have a .log file where there is data in two columns (separated by a space). The file is thus: 0.0 3 0.0 6 0.0 6 0.0 6 0.0 7 0.0 7 0.0 7 0.0 7 0.0 11 0.0 11 0.0 11 0.0 11 0.0 11 0.1 17 (6 Replies)
Discussion started by: Flamex
6 Replies

4. Shell Programming and Scripting

Parsing data

Hi all , I have a file with billing CDR records in it. I need to parse that information (row format) . The purpose is to compare full content. The example I have given below is a single line record but it has two portions, (1) the line start with “!” and end with “1.2.1.8” and (2) second part... (5 Replies)
Discussion started by: jaygamini
5 Replies

5. Shell Programming and Scripting

Parsing the data

Hi friends, I need to parse the following data in the given format and get the desired output. I need a function, which takes the input as a parameter and the desired output will be returned from the function. INPUT(single parameter as complete string) A;BCF;DFG;FD ... (3 Replies)
Discussion started by: sumesh.1988
3 Replies

6. Shell Programming and Scripting

Help in Parsing data

I have below string Transaction_ID:SDP-DM-151204679 , Transaction_DateTime:2011-02-11 00:00:15 GMT+05:30 , Transaction_Mode:WAP , Circle_ID:4 , Circle_Name:BJ ,Zone: , CustomerID:B_31563486 , MSISDN:7870904329 , IMSI:405876122068099 , IMEI: , Sub_Profile:Pre-Paid , CPID:Nazara , CPNAME:Nazara ,... (6 Replies)
Discussion started by: poweroflinux
6 Replies

7. Shell Programming and Scripting

Ignore Header and Footer and Sort the data in fixed width file

Hi Experts, I want to Sort the data in fixed width file where i have Header and Footer also in file. I m using below commad to do the sort based on field satarting from 15 position to 17 position , but it is not ignoring the Header and Footer of the file while sorting. In the output i am... (5 Replies)
Discussion started by: sasikari
5 Replies

8. Shell Programming and Scripting

Help with parsing data with awk , eliminating unwanted data

Experts , Below is the data: --- Physical volumes --- PV Name /dev/dsk/c1t2d0 VG Name /dev/vg00 PV Status available Allocatable yes VGDA 2 Cur LV 8 PE Size (Mbytes) 8 Total PE 4350 Free PE 2036 Allocated PE 2314 Stale PE 0 IO Timeout (Seconds) default --- Physical volumes ---... (5 Replies)
Discussion started by: rveri
5 Replies

9. Shell Programming and Scripting

Parsing XML (and insert data) then output data (bash / Solaris)

Hi folks I have a script I wrote that basically parses a bunch of config and xml files works out were to add in the new content then spits out the data into a new file. It all works - apart from the xml and config file format in the new file with XML files the original XML (that ends up in... (2 Replies)
Discussion started by: dfinch
2 Replies

10. Shell Programming and Scripting

awk code to ignore the first occurence unknown number of rows in a data column

Hello experts, Shown below is the 2 column sample data(there are many data columns in actual input file), Key, Data A, 1 A, 2 A, 2 A, 3 A, 1 A, 1 A, 1 I need the below output. Key, Data A, 2 A, 2 A, 3 A, 1 A, 1 A, 1 (2 Replies)
Discussion started by: ks_reddy
2 Replies
Devel::Refcount(3pm)					User Contributed Perl Documentation				      Devel::Refcount(3pm)

NAME
"Devel::Refcount" - obtain the REFCNT value of a referent SYNOPSIS
use Devel::Refcount qw( refcount ); my $anon = []; print "Anon ARRAY $anon has " . refcount($anon) . " reference "; my $otherref = $anon; print "Anon ARRAY $anon now has " . refcount($anon) . " references "; DESCRIPTION
This module provides a single function which obtains the reference count of the object being pointed to by the passed reference value. FUNCTIONS
$count = refcount($ref) Returns the reference count of the object being pointed to by $ref. COMPARISON WITH SvREFCNT This function differs from "Devel::Peek::SvREFCNT" in that SvREFCNT() gives the reference count of the SV object itself that it is passed, whereas refcount() gives the count of the object being pointed to. This allows it to give the count of any referent (i.e. ARRAY, HASH, CODE, GLOB and Regexp types) as well. Consider the following example program: use Devel::Peek qw( SvREFCNT ); use Devel::Refcount qw( refcount ); sub printcount { my $name = shift; printf "%30s has SvREFCNT=%d, refcount=%d ", $name, SvREFCNT($_[0]), refcount($_[0]); } my $var = []; printcount 'Initially, $var', $var; my $othervar = $var; printcount 'Before CODE ref, $var', $var; printcount '$othervar', $othervar; my $code = sub { undef $var }; printcount 'After CODE ref, $var', $var; printcount '$othervar', $othervar; This produces the output Initially, $var has SvREFCNT=1, refcount=1 Before CODE ref, $var has SvREFCNT=1, refcount=2 $othervar has SvREFCNT=1, refcount=2 After CODE ref, $var has SvREFCNT=2, refcount=2 $othervar has SvREFCNT=1, refcount=2 Here, we see that SvREFCNT() counts the number of references to the SV object passed in as the scalar value - the $var or $othervar respectively, whereas refcount() counts the number of reference values that point to the referent object - the anonymous ARRAY in this case. Before the CODE reference is constructed, both $var and $othervar have SvREFCNT() of 1, as they exist only in the current lexical pad. The anonymous ARRAY has a refcount() of 2, because both $var and $othervar store a reference to it. After the CODE reference is constructed, the $var variable now has an SvREFCNT() of 2, because it also appears in the lexical pad for the new anonymous CODE block. PURE-PERL FALLBACK An XS implementation of this function is provided, and is used by default. If the XS library cannot be loaded, a fallback implementation in pure perl using the "B" module is used instead. This will behave identically, but is much slower. Rate pp xs pp 225985/s -- -66% xs 669570/s 196% -- SEE ALSO
o Test::Refcount - assert reference counts on objects AUTHOR
Paul Evans <leonerd@leonerd.org.uk> perl v5.14.2 2011-11-15 Devel::Refcount(3pm)
All times are GMT -4. The time now is 07:34 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy