Sponsored Content
Full Discussion: Parsing syslog from Linux
Top Forums Shell Programming and Scripting Parsing syslog from Linux Post 303037250 by RudiC on Saturday 27th of July 2019 08:16:30 AM
Old 07-27-2019
Having the field separator character to be included in the target data makes things complicated. Plus the fact that the char may occur several (unpredictable) times in the data field, like "United States of America". So, additional text processing needs to be done. There are, as always, several approaches, of which this one seems to be the fastest, although it needs around 10% computing time on top:

Code:
awk '
BEGIN   {print HDLN = "eventtime|srcip|dstip|srcport|dstport|transip|transport|action|sessionid|dstcountry"
         MX = split (HDLN, HD, "|")
         for (i=1; i<=MX; i++) L[i] = length (HD[i]) + 1
        }
        {OUT = DL = ""
         for (i=1; i<=MX; i++)  {match ($0, HD[i] "=[^ ]*")
                                 TMP =  substr ($0, RSTART + L[i], RLENGTH - L[i])
                                if (gsub (/\"/, "&", TMP) %2)  {TMP2 = substr ($0, RSTART + RLENGTH)
                                                                TMP  = TMP substr (TMP2, 1, index (TMP2, "\""))
                                                                }
                                 OUT = OUT DL TMP
                                 DL = "|"
                                }
         print OUT 
        }
' file

Please check and report back.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need some help with parsing

I have a big xml file with little formatting in it. It contains over 600 messages that I need to break each message out in its own separate file. The xml file looks in the middle of it something like this: </Title></Msg><Msg><Opener> Hello how are you?<Title> Some says hello</Title><Body>... (3 Replies)
Discussion started by: quixoticking11
3 Replies

2. Shell Programming and Scripting

Perl parsing compared to Ksh parsing

#! /usr/local/bin/perl -w $ip = "$ARGV"; $rw = "$ARGV"; $snmpg = "/usr/local/bin/snmpbulkget -v2c -Cn1 -Cn2 -Os -c $rw"; $snmpw = "/usr/local/bin/snmpwalk -Os -c $rw"; $syst=`$snmpg $ip system sysName sysObjectID`; sysDescr.0 = STRING: Cisco Internetwork Operating System Software... (1 Reply)
Discussion started by: popeye
1 Replies

3. Shell Programming and Scripting

Parsing of file for Report Generation (String parsing and splitting)

Hey guys, I have this file generated by me... i want to create some HTML output from it. The problem is that i am really confused about how do I go about reading the file. The file is in the following format: TID1 Name1 ATime=xx AResult=yyy AExpected=yyy BTime=xx BResult=yyy... (8 Replies)
Discussion started by: umar.shaikh
8 Replies

4. Red Hat

Parsing a linux file and formatting it.

Hi, I have a linux file that has data like this.. REQUEST_ID|text^Ctext^Ctext^C REQUEST_ID|text^Ctext^C REQUEST_ID| REQUEST_ID| REQUEST_ID|text^Ctext^Ctext^Ctext^Ctext^Ctext^C.... Where ever I see a ^C character, I need to copy the corresponding REQUEST_ID and that part of the text to a new... (17 Replies)
Discussion started by: charithainfadev
17 Replies

5. Shell Programming and Scripting

Parsing kiwi syslog from Astaro

Hello, I am trying to parse this syslog pulling out and logging results to a file. The information I want is: scrip, scrport, dstip, dstport. I just want the numbers, not including the text part ie srcip=". Problem is, the column locations change, so I can't use the nice awk $1 $2 etc to... (4 Replies)
Discussion started by: rmelnik
4 Replies

6. UNIX for Dummies Questions & Answers

Parsing linux commands through FTP

Hi Techies, I have made a shell script which stores the output of it in a text file. then i wanted to fetch that text file using windows scheduler in my windows xp desktop which i did successfully using the below mentioned ftp .bat file : @echo off @echo ftp_user>ftp_test.scr @echo... (0 Replies)
Discussion started by: gemnian.g
0 Replies

7. Shell Programming and Scripting

Help - Parsing data in XML in Linux

Hi, I have an XML file in Linux and it contains a long string of characters. The last part of the file is like ....... ....... ....... CAD</MarketDescription></InvestorTransaction></AdvisorAccount></DivisionAdvisor></Division>... (3 Replies)
Discussion started by: naveed
3 Replies

8. Shell Programming and Scripting

Specific string parsing in Linux/UNIX

Hi, I have a string which can be completely unstructred. I am looking to parse out values within that String. Here is an example <Random Strings> String1=<some number a> String2=<some number b> String3=<some number c> Satish=<some number d> String4=<some number e> I only want to parse out... (1 Reply)
Discussion started by: satishrao
1 Replies

9. SuSE

Location and name of SYSLOG in SUSE Linux

Esteemed listers, Where is the location of SYSLOG file? In etc/auditd.conf script, the log_file location is '/var/log/audit/audit.log' as below. Is this the location where SYSLOG is stored? Thank you in advance, log_file = /var/log/audit/audit.log log_format = RAW... (3 Replies)
Discussion started by: JDBA
3 Replies

10. Programming

Openlog and syslog in red-hat Linux doesn't write any thing to /var/log/*

Using redhat 64 bit ver 6.2 I have simple c++ app that is trying to write to syslog like this: /* try to write massage into linux log */ void foo::writeToSyslog() { openlog("testlogfoo", 0, 24); // Send the message. ... (1 Reply)
Discussion started by: umen
1 Replies
nl_printf(3int) 														   nl_printf(3int)

Name
       nl_printf, nl_fprintf, nl_sprintf - print formatted output

Syntax
       #include <stdio.h>

       int nl_printf ( format [, arg ] ...  )
       char *format;

       int nl_fprintf ( stream, format [, arg ] ...  )
       FILE *stream;
       char *format;

       int nl_sprintf ( s, format [, arg ] ...	)
       char *s, format;

Description
       The  international  functions and are identical to and have been superceded by the international functions and in a library. You should use
       the and functions when you write new calls to print formatted output in an international program. For more information on these	functions,
       see the reference page.

       You  can  continue  to use existing calls to the or international functions.  These functions remain available for compatibility with XPG-2
       conformant software, but may not be supported in future releases of the ULTRIX system.

       The and international functions are similar to the standard I/O function. (For more information about the standard I/O  function,  see  the
       reference  page.)   The	difference is that the international functions allow you to use the I%digit$ conversion sequence in place of the %
       character you use in the standard I/O functions. The digit is a decimal digit n from 1 to 9.  The international functions apply conversions
       to the n th argument in the argument list, rather than to the next unused argument.

       You  can  use  %  conversion character in the international functions.  However, you cannot mix the % conversion character with the %digit$
       conversion sequence in a single call.

       You can indicate a field width or precision by an asterisk (*), instead of a digit string, in strings containing the %  conversion  charac-
       ter.  If  you  use  an asterisk, you can supply an integer argument that specifies the field width or precision.  In strings containing the
       %digit$ conversion character, you can indicate field width or precision by the sequence *digit$.  You use a decimal digit from 1  to  9	to
       indicate which argument contains an integer that specifies the field width or precision.

       The conversion characters and their meanings are identical to

       You must use each digit argument at least once.	The results of not using an argument are undefined.

   International Environment
       LC_NUMERIC     If  this	environment  is set and valid, uses the international language database named in the definition to determine radix
		      character rules.

       LANG	      If this environment variable is set and valid uses the international language database named in the definition to  determine
		      collation and character classification rules.  If is defined, its definition supercedes the definition of LANG.

Examples
       The following example illustrates using an argument to specify field width:
       nl_printf ("%1$d:%2$.*3$d:%4$.*3$d
",
			   hour, min, precision, sec);
       The format string *3$ indicates that the third argument, which is named precision, contains the integer field width specification.

       To print the language independent date and time format, use the following statement:
       nl_printf (format, weekday, month, day, hour, min);
       For United States of America use, could be a pointer to the following string:
       "%1$s,  %2$s %3$d, %4$d:%5$.2d
"
       This string produces the following message:
       Sunday, July 3, 10:02
       For use in a German environment, could be a pointer to the following string:
       "%1$s, %3$d. %2$s, %4$d:%5$.2d
"
       This produces the following message:
       Sonntag, 3. Juli, 10:02

See Also
       intro(3int), setlocale(3), nl_scanf(3int), printf(3int), scanf(3int), printf(3s), putc(3s), scanf(3s), stdio(3s)
       Guide to Developing International Software

																   nl_printf(3int)
All times are GMT -4. The time now is 02:45 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy