Sponsored Content
Top Forums Shell Programming and Scripting Extract data from a log file and put it in a file Post 303025353 by ctsgnb on Wednesday 31st of October 2018 11:09:33 AM
Old 10-31-2018
Code:
C:\WINDOWS\system32>cat mytst
2018-10-23 12:33:21 AI ERROR -- tpid: SAMPLE_TH  account: 123456789  aiSessionNumber: 660640464  mapName: xxx to yyy
errorDesc: Translation Error:ErrorNumber : 993
ErrorDescriptionSummary: Errors from sources other than XML Status File
ErrorDescriptionDetail : 504



sourceFileName: /apps/EGAPshared/Invoicing/gcs/large_file/XXXXXX001314509_952834960.es2eg.4X
:
--------------------------------------------------------------------------------------------------
2018-10-23 12:33:21 AI ERROR -- tpid: SAMPLE_XX  account: 123654789  aiSessionNumber: 660640457  mapName: xxx to yyy
errorDesc: Translation Error:ErrorNumber : 993
ErrorDescriptionSummary: Errors from sources other than XML Status File
ErrorDescriptionDetail : 504



sourceFileName: /apps/EGAPshared/Invoicing/gcs/large_file/YYYYYY001314509_952834960.es2eg.3X
:
--------------------------------------------------------------------------------------------------
2018-10-23 12:39:15 AI ERROR -- tpid: XXXXX_TH  account: 321456789  aiSessionNumber: 660642348  mapName: xxx to yyy
errorDesc: Translation Error:ErrorNumber : 993
ErrorDescriptionSummary: Errors from sources other than XML Status File
ErrorDescriptionDetail : 504



sourceFileName: /apps/EGAPshared/Invoicing/gcs/large_file/ZZZZZZ001314509_952834960.es2eg.3X

C:\WINDOWS\system32>gawk  -F"[: ]" "/tpid/{print toupper($8) \" = \" $10 OFS $12 \" =\"  $14}/^sourceFileName/{p=toupper(substr($1,1,1));s=tolower(substr($1,2,5)\" \"substr($1,7));sub($1\":\",\" =\",$0); print p s $0}" OFS=" | " mytst
TPID = SAMPLE_TH | account =123456789
Source filename = /apps/EGAPshared/Invoicing/gcs/large_file/XXXXXX001314509_952834960.es2eg.4X
TPID = SAMPLE_XX | account =123654789
Source filename = /apps/EGAPshared/Invoicing/gcs/large_file/YYYYYY001314509_952834960.es2eg.3X
TPID = XXXXX_TH | account =321456789
Source filename = /apps/EGAPshared/Invoicing/gcs/large_file/ZZZZZZ001314509_952834960.es2eg.3X

C:\WINDOWS\system32>




So... with the unix syntax :



Code:
gawk -F"[: ]" '/tpid/{print toupper($8) " = " $10 OFS $12 " =" $14}
/^sourceFileName/{
p=toupper(substr($1,1,1))
s=tolower(substr($1,2,5)" "substr($1,7))
sub($1":"," =",$0)
print p s $0
}' OFS=" | " inputfile


Another approach:
Code:
gawk '{gsub(" ",RS)}1' inputfile | gawk '/(tpid|account|sourceFileName)/{f=$0;getline;print f,$0}'

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

shell-script which extract data from log file

give me a shell-script which extract data from log file on a server by giving date and time as input (for both start time and end time) and it will give the logs generated during the given time as output. (4 Replies)
Discussion started by: abhishek27
4 Replies

2. UNIX for Dummies Questions & Answers

need to extract few lines from a file and put it in another file

Hi all, I have a file which contains comments,dotted lines(-------),actual records etc.I need to fetch only the actual records and put into a new file neglecting/deleting other rows.I have a header record too which contains the column names for the actual records.i need to discard this too.how... (5 Replies)
Discussion started by: ganesh_248
5 Replies

3. Shell Programming and Scripting

Extract data from log file from or after the specific date

Hi , I am having a script which will start a process and appends the process related logs to a log file. The log file writes logs with every line starting with date in the format of: date +"%Y %b %d %H:%M:%S". So, in the script, before I start the process, I am storing the date as DATE=`date +"%Y... (5 Replies)
Discussion started by: chiru_h
5 Replies

4. Shell Programming and Scripting

need a shell script to extract data from a log file.

If I have a log like : Mon Jul 19 05:07:34 2010; TCP; eth3; 52 bytes; from abc to def Mon Jul 19 05:07:35 2010; UDP; eth3; 46 bytes; from aaa to bbb Mon Jul 19 05:07:35 2010; TCP; eth3; 52 bytes; from def to ghi I will need an output like this : Time abc to def... (1 Reply)
Discussion started by: hitha87
1 Replies

5. Shell Programming and Scripting

Data Extract from XML Log File

Please help me out to extract the Data from the XML Log files. So here is the data ERROR|2010-08-26 00:05:52,958|SERIAL_ID=128279996|ST=2010-08-2600:05:52|DEVICE=113.2.21.12:601|TYPE=TransactionLog... (9 Replies)
Discussion started by: raghunsi
9 Replies

6. Shell Programming and Scripting

split input data file and put into same output file

Hi All, I have two input file and need to generate a CSV file. The existing report just "GREP" the records with the Header and Tailer records with the count of records. Now i need to split the data into 25 records each in the same CSV file. id_file (Input file ) 227050994 232510151... (4 Replies)
Discussion started by: rasmith
4 Replies

7. Shell Programming and Scripting

Use grep sed or awk to extract string from log file and put into CSV

I'd like to copy strings from a log file and put them into a CSV. The strings could be on different line numbers, depending on size of log. Example Log File: File = foo.bat Date = 11/11/11 User = Foo Bar Size = 1024 ... CSV should look like: "foo.bat","11/11/11","Foo Bar","1024" (7 Replies)
Discussion started by: chipperuga
7 Replies

8. Shell Programming and Scripting

Extract data from log file in specified range of time

I was searching for parsing a log file and found what I need in this link http://stackoverflow.com/questions/7575267/extract-data-from-log-file-in-specified-range-of-time But the most useful answer (posted by @Kent): # this variable you could customize, important is convert to seconds. # e.g... (2 Replies)
Discussion started by: kingk110
2 Replies

9. UNIX for Dummies Questions & Answers

Extract date ranged data from log file

Hi, I am trying to extract lines of data within a log file on a Redhat 5 Linux system. eg I need all the lines with a particular username over the last 3 minutes. the log file may read like this, and I want a way to search all the lines extracting all the relevant lines over the last 3... (2 Replies)
Discussion started by: mantis100
2 Replies

10. Shell Programming and Scripting

Extract data from log file within specified time

So, we have a script, that is supposed to have a couple of functions like showing number of failed connections, recieved bytes per IP-address, and so on. We are supposed to be able to limit the number of results to either 0-24 hours or X days back from the last data in the log file. Everything... (3 Replies)
Discussion started by: Plumpen
3 Replies
jfs_fscklog(8)						   Extract/Display JFS fsck Log 					    jfs_fscklog(8)

NAME
jfs_fscklog - extract a JFS fsck service log into a file and/or format and display the extracted file SYNOPSIS
jfs_fscklog [ -d ] [ -e device ] [ -f output.file ] [ -p ] [ -V ] DESCRIPTION
jfs_fscklog with option -e device extracts the contents of either the most recent or immediately prior (specified with option -p) JFS fsck service log from the specified device, and writes the output to a file. The file name may be specified with the -f option, or the default file name is <pwd>fscklog.new. If the -p option is used, the default file name is <pwd>fscklog.old. jfs_fscklog with option -d formats and displays the contents of the extracted file. jfs_fscklog with options -d and -e device extracts and displays the JFS fsck service log. OPTIONS
-d Format and display a previously extracted JFS fsck service log. -e device Extract the JFS fsck service log from device and store it in a file. (device is the special file name corresponding to the actual device from which the fsck log will be extracted (e.g. /dev/hdb1 )). -f file.name Specify the file name, overriding the default file name. When used with -e device, file.name specifies the name of the file into which the JFS fsck log will be extracted. When used with -d, file.name specifies the name of the extracted file whose contents will be formatted and displayed. file.name must be 127 characters or less in length. file.name will be created in the present working directory unless it is fully qualified. -p Extract the prior log (as opposed to the most recent log). -V Print version information and exit (regardless of any other chosen options). EXAMPLES
Extract the most recent JFS fsck log on /dev/hda5 into <pwd>fscklog.new: jfs_fscklog -e /dev/hda5 Extract the most recent JFS fsck log on /dev/hdb3 into /temp/l9651107.log: jfs_fscklog -e /dev/hdb3 -f /temp/l9651107.log Display the JFS fsck log that was extracted into /temp/l9651107.log: jfs_fscklog -d -f /temp/l9651107.log Extract and display the previous JFS fsck log from /dev/hda8: jfs_fscklog -e /dev/hda8 -d -p REPORTING BUGS
If you find a bug in JFS or jfs_fscklog, please report it via the bug tracking system ("Report Bugs" section) of the JFS project web site: http://jfs.sourceforge.net/ Please send as much pertinent information as possible, including any jfs_fscklog error messages and the complete output of running jfs_fsck with the -v option on the JFS device. SEE ALSO
jfs_fsck(8), jfs_mkfs(8), jfs_tune(8), jfs_logdump(8), jfs_debugfs(8) AUTHOR
Barry Arndt (barndt@us.ibm.com) jfs_fscklog is maintained by IBM. See the JFS project web site for more details: http://jfs.sourceforge.net/ October 28, 2002 jfs_fscklog(8)
All times are GMT -4. The time now is 03:01 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy