Sponsored Content
Full Discussion: Parsing syslog from Linux
Top Forums Shell Programming and Scripting Parsing syslog from Linux Post 303037216 by arm on Friday 26th of July 2019 04:22:04 AM
Old 07-26-2019
thanks guys but now the log gets bigger and some other fields are added some others are deleted , what I need now is to help me to

Quote:
Logsample
Code:
logver=56 idseq=63256900099118326 itime=1563205190 devid=FG-5KDTB18800138 devname=LAL-C1-FGT-03 vd=USER date=2019-07-15 time=18:39:49 logid="0000000013" type="traffic" subtype="forward" level="notice" eventtime=1563205189 srcip=11.3.3.17 srcport=50544 srcintf="SGI-CORE.123" srcintfrole="undefined" dstip=12.0.1.1 dstport=443 dstintf="FA-SPI.100" dstintfrole="undefined" poluuid="230d4d26-AAAA-51e9-b9d1-7bf4c828f000" sessionid=20639817 proto=6 action="server-rst" policyid=10 policytype="policy" service="HTTPS" dstcountry="Germany" srccountry="Reserved" trandisp="snat" transip=11.1.1.1 transport=5092 duration=71 sentbyte=093 rcvdbyte=213 sentpkt=11 rcvdpkt=16 appcat="unscanned"

logver=56 idseq=63256900099118326 itime=1563205190 devid=FG-5KDTB18800138 devname=LAL-C1-FGT-03 vd=USER date=2019-07-15 time=18:39:49 logid="0000000013" type="traffic" subtype="forward" level="notice" eventtime=1563205190 srcip=11.3.3.17 srcport=50544 srcintf="SGI-CORE.123" srcintfrole="undefined" dstip=13.0.1.1 dstport=80 dstintf="FA-SPI.100" dstintfrole="undefined" poluuid="230d4d26-AAAA-51e9-b9d1-7bf4c828f000" sessionid=20639824 proto=6 action="close" policyid=34 policytype="policy" service="UDP" dstcountry="United State" srccountry="Reserved" trandisp="snat" transip=14.1.1.1 transport=5092 duration=50 sentbyte=093 rcvdbyte=213 sentpkt=11 rcvdpkt=16 appcat="unscanned"


logver=56 idseq=63256900099118326 itime=1563205190 devid=FG-5KDTB18800138 devname=LAL-C1-FGT-03 vd=USER date=2019-07-15 time=18:39:49 logid="0000000013" type="traffic" subtype="forward" level="notice" eventtime=1563205100 srcip=11.3.3.17 srcport=50590 srcintf="SGI-CORE.123" srcintfrole="undefined" dstip=1.0.1.1 dstport=80 dstintf="FA-SPI.100" dstintfrole="undefined" poluuid="230d4d26-AAAA-51e9-b9d1-7bf4c828f000" sessionid=20639817 proto=6  policyid=34 policytype="policy" service="UDP/10"  srccountry="Reserved" trandisp="snat"  duration=60 sentbyte=093 rcvdbyte=213 sentpkt=11 rcvdpkt=16 appcat="unscanned"



eventtime|srcip|dstip|srcport|dstport|transip|transport|action|sessionid|service|policyid|dstcountry|duration    --> header no need to be shown on output and if any field was missing then it will left as empty between pipe delimiter as  the case at line 3 ||   

1563205189|11.3.3.17|12.0.1.1|50544|443|11.1.1.1|5092|server-rst|20639817|HTTPS|10|Germany|71
1563205190|11.3.3.17|13.0.1.1|50544|80|14.1.1.1|5092|closet|20639824|UDP|34|United State|50
1563205100|11.3.3.17|1.0.1.1|50590|80||||20639817|UDP/10|34||60


 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need some help with parsing

I have a big xml file with little formatting in it. It contains over 600 messages that I need to break each message out in its own separate file. The xml file looks in the middle of it something like this: </Title></Msg><Msg><Opener> Hello how are you?<Title> Some says hello</Title><Body>... (3 Replies)
Discussion started by: quixoticking11
3 Replies

2. Shell Programming and Scripting

Perl parsing compared to Ksh parsing

#! /usr/local/bin/perl -w $ip = "$ARGV"; $rw = "$ARGV"; $snmpg = "/usr/local/bin/snmpbulkget -v2c -Cn1 -Cn2 -Os -c $rw"; $snmpw = "/usr/local/bin/snmpwalk -Os -c $rw"; $syst=`$snmpg $ip system sysName sysObjectID`; sysDescr.0 = STRING: Cisco Internetwork Operating System Software... (1 Reply)
Discussion started by: popeye
1 Replies

3. Shell Programming and Scripting

Parsing of file for Report Generation (String parsing and splitting)

Hey guys, I have this file generated by me... i want to create some HTML output from it. The problem is that i am really confused about how do I go about reading the file. The file is in the following format: TID1 Name1 ATime=xx AResult=yyy AExpected=yyy BTime=xx BResult=yyy... (8 Replies)
Discussion started by: umar.shaikh
8 Replies

4. Red Hat

Parsing a linux file and formatting it.

Hi, I have a linux file that has data like this.. REQUEST_ID|text^Ctext^Ctext^C REQUEST_ID|text^Ctext^C REQUEST_ID| REQUEST_ID| REQUEST_ID|text^Ctext^Ctext^Ctext^Ctext^Ctext^C.... Where ever I see a ^C character, I need to copy the corresponding REQUEST_ID and that part of the text to a new... (17 Replies)
Discussion started by: charithainfadev
17 Replies

5. Shell Programming and Scripting

Parsing kiwi syslog from Astaro

Hello, I am trying to parse this syslog pulling out and logging results to a file. The information I want is: scrip, scrport, dstip, dstport. I just want the numbers, not including the text part ie srcip=". Problem is, the column locations change, so I can't use the nice awk $1 $2 etc to... (4 Replies)
Discussion started by: rmelnik
4 Replies

6. UNIX for Dummies Questions & Answers

Parsing linux commands through FTP

Hi Techies, I have made a shell script which stores the output of it in a text file. then i wanted to fetch that text file using windows scheduler in my windows xp desktop which i did successfully using the below mentioned ftp .bat file : @echo off @echo ftp_user>ftp_test.scr @echo... (0 Replies)
Discussion started by: gemnian.g
0 Replies

7. Shell Programming and Scripting

Help - Parsing data in XML in Linux

Hi, I have an XML file in Linux and it contains a long string of characters. The last part of the file is like ....... ....... ....... CAD</MarketDescription></InvestorTransaction></AdvisorAccount></DivisionAdvisor></Division>... (3 Replies)
Discussion started by: naveed
3 Replies

8. Shell Programming and Scripting

Specific string parsing in Linux/UNIX

Hi, I have a string which can be completely unstructred. I am looking to parse out values within that String. Here is an example <Random Strings> String1=<some number a> String2=<some number b> String3=<some number c> Satish=<some number d> String4=<some number e> I only want to parse out... (1 Reply)
Discussion started by: satishrao
1 Replies

9. SuSE

Location and name of SYSLOG in SUSE Linux

Esteemed listers, Where is the location of SYSLOG file? In etc/auditd.conf script, the log_file location is '/var/log/audit/audit.log' as below. Is this the location where SYSLOG is stored? Thank you in advance, log_file = /var/log/audit/audit.log log_format = RAW... (3 Replies)
Discussion started by: JDBA
3 Replies

10. Programming

Openlog and syslog in red-hat Linux doesn't write any thing to /var/log/*

Using redhat 64 bit ver 6.2 I have simple c++ app that is trying to write to syslog like this: /* try to write massage into linux log */ void foo::writeToSyslog() { openlog("testlogfoo", 0, 24); // Send the message. ... (1 Reply)
Discussion started by: umen
1 Replies
TEXT2PCAP(1)						   The Ethereal Network Analyzer					      TEXT2PCAP(1)

NAME
text2pcap - Generate a capture file from an ASCII hexdump of packets SYNOPSYS
text2pcap [ -h ] [ -d ] [ -q ] [ -o hex|oct ] [ -l typenum ] [ -e l3pid ] [ -i proto ] [ -u srcport,destport ] [ -s srcport,destport,tag ] [ -S srcport,destport,ppi ] [ -t timefmt ] infile outfile DESCRIPTION
Text2pcap is a program that reads in an ASCII hex dump and writes the data described into a libpcap-style capture file. text2pcap can read hexdumps with multiple packets in them, and build a capture file of multiple packets. text2pcap is also capable of generating dummy Ether- net, IP and UDP headers, in order to build fully processable packet dumps from hexdumps of application-level data only. Text2pcap understands a hexdump of the form generated by od -t x1. In other words, each byte is individually displayed and surrounded with a space. Each line begins with an offset describing the position in the file. The offset is a hex number (can also be octal - see -o), of more than two hex digits. Here is a sample dump that text2pcap can recognize: 000000 00 e0 1e a7 05 6f 00 10 ........ 000008 5a a0 b9 12 08 00 46 00 ........ 000010 03 68 00 00 00 00 0a 2e ........ 000018 ee 33 0f 19 08 7f 0f 19 ........ 000020 03 80 94 04 00 00 10 01 ........ 000028 16 a2 0a 00 03 50 00 0c ........ 000030 01 01 0f 19 03 80 11 01 ........ There is no limit on the width or number of bytes per line. Also the text dump at the end of the line is ignored. Bytes/hex numbers can be uppercase or lowercase. Any text before the offset is ignored, including email forwarding characters '>'. Any lines of text between the bytestring lines is ignored. The offsets are used to track the bytes, so offsets must be correct. Any line which has only bytes without a leading offset is ignored. An offset is recognized as being a hex number longer than two characters. Any text after the bytes is ignored (e.g. the character dump). Any hex numbers in this text are also ignored. An offset of zero is indicative of starting a new packet, so a single text file with a series of hexdumps can be converted into a packet capture with multiple packets. Multiple packets are read in with timestamps differing by one second each. In general, short of these restrictions, text2pcap is pretty liberal about reading in hexdumps and has been tested with a variety of mangled outputs (including being forwarded through email multiple times, with limited line wrap etc.) There are a couple of other special features to note. Any line where the first non-whitespace character is '#' will be ignored as a com- ment. Any line beginning with #TEXT2PCAP is a directive and options can be inserted after this command to be processed by text2pcap. Cur- rently there are no directives implemented; in the future, these may be used to give more fine grained control on the dump and the way it should be processed e.g. timestamps, encapsulation type etc. Text2pcap also allows the user to read in dumps of application-level data, by inserting dummy L2, L3 and L4 headers before each packet. The user can elect to insert Ethernet headers, Ethernet and IP, or Ethernet, IP and UDP headers before each packet. This allows Ethereal or any other full-packet decoder to handle these dumps. OPTIONS
-h Displays a help message. -d Displays debugging information during the process. Can be used multiple times to generate more debugging information. -q Be completely quiet during the process. -o hex|oct Specify the radix for the offsets (hex or octal). Defaults to hex. This corresponds to the "-A" option for od. -l Specify the link-layer type of this packet. Default is Ethernet (1). See net/bpf.h for the complete list of possible encapsulations. Note that this option should be used if your dump is a complete hex dump of an encapsulated packet and you wish to specify the exact type of encapsulation. Example: -l 7 for ARCNet packets. -e l3pid Include a dummy Ethernet header before each packet. Specify the L3PID for the Ethernet header in hex. Use this option if your dump has Layer 3 header and payload (e.g. IP header), but no Layer 2 encapsulation. Example: -e 0x806 to specify an ARP packet. For IP packets, instead of generating a fake Ethernet header you can also use -l 12 to indicate a raw IP packet to Ethereal. Note that -l 12 does not work for any non-IP Layer 3 packet (e.g. ARP), whereas generating a dummy Ethernet header with -e works for any sort of L3 packet. -i proto Include dummy IP headers before each packet. Specify the IP protocol for the packet in decimal. Use this option if your dump is the payload of an IP packet (i.e. has complete L4 information) but does not have an IP header. Note that this automatically includes an appropriate Ethernet header as well. Example: -i 46 to specify an RSVP packet (IP protocol 46). -u srcport,destport Include dummy UDP headers before each packet. Specify the source and destination UDP ports for the packet in decimal. Use this option if your dump is the UDP payload of a packet but does not include any UDP, IP or Ethernet headers. Note that this automatically includes appropriate Ethernet and IP headers with each packet. Example: -u 1000,69 to make the packets look like TFTP/UDP packets. -s srcport,destport,tag Include dummy SCTP headers before each packet. Specify, in decimal, the source and destination SCTP ports, and verification tag, for the packet. Use this option if your dump is the SCTP payload of a packet but does not include any SCTP, IP or Ethernet headers. Note that this automatically includes appropriate Ethernet and IP headers with each packet. A CRC32C checksum will be put into the SCTP header. -S srcport,destport,ppi Include dummy SCTP headers before each packet. Specify, in decimal, the source and destination SCTP ports, and a verification tag of 0, for the packet, and prepend a dummy SCTP DATA chunk header with a payload protocol identifier if ppi. Use this option if your dump is the SCTP payload of a packet but does not include any SCTP, IP or Ethernet headers. Note that this automatically includes appropri- ate Ethernet and IP headers with each packet. A CRC32C checksum will be put into the SCTP header. -t timefmt Treats the text before the packet as a date/time code; timefmt is a format string of the sort supported by strptime(3). Example: The time "10:15:14.5476" has the format code "%H:%M:%S." NOTE: The subsecond component delimiter must be specified (.) but no pattern is required; the remaining number is assumed to be frac- tions of a second. SEE ALSO
tcpdump(8), pcap(3), ethereal(1), editcap(1), strptime(3). NOTES
Text2pcap is part of the Ethereal distribution. The latest version of Ethereal can be found at http://www.ethereal.com. AUTHORS
Ashok Narayanan <ashokn@cisco.com> 0.9.8 2002-08-08 TEXT2PCAP(1)
All times are GMT -4. The time now is 09:22 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy