Sponsored Content
Full Discussion: Extracting from log file
Top Forums UNIX for Beginners Questions & Answers Extracting from log file Post 303003241 by nextStep on Monday 11th of September 2017 04:38:20 AM
Old 09-11-2017
This is what tried.

Step1: Converted the large log file into json file as below

Code:
awk -F ' Response:|\\(HttpClientUtil)|' '{print $2}' logfile >> logfile.json

Step 2:From the json file using jq extracted pointbalance,firstname and last name

Code:
cat logfile.json | jq . | egrep "pointsBalance|firstName|lastName"

Output:
Code:
"pointsBalance": 60153
    "firstName": "BETTY",
    "lastName": "BROOKS",
    "pointsBalance": 9870
    "firstName": "ROSS",
    "lastName": "MULLEN",


The problem here is there is anther text in the log file named named [(LoggingRestRequestResponseInterceptor)] . I am not able to get the fields in that pattern using the step 1.

Step 1 scans and takes all the fields which is in between (HttpClientUtil).

If i can combine the second pattern [(LoggingRestRequestResponseInterceptor)] in the step 1 i believe it will be outputing all the required fields.
This is where struggling.
---------- Post updated at 03:38 AM ---------- Previous update was at 01:41 AM ----------


Tried the below code in getting the fields between the keyword (LoggingRestRequestResponseInterceptor)

Code:
awk -F "{|}" '{print $9} {print $11}' logfile

Output

Code:
"pointsBalance":111834
"firstName":"KERRI","lastName":"MCGUIRE","address":
"additionalInfo":
,"address":


Then tried to combine both the awk statement,but still not getting the required output.

Code:
awk -F "{|}" '{print $9} {print $11}'| awk -F ' Response:|\\(HttpClientUtil)|' '{print $2}' logfile


Last edited by RudiC; 09-11-2017 at 05:55 AM.. Reason: corrected / adapted code tags
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Extracting the required text from log files

It would be highly appreciable if any one helps me in this. I am trying to get it done through Java but I love unix and believe it can be done within minutes with couple of lines. The input log file is a text file contains multiple entries seperated by a blank line. Each seperated entry... (7 Replies)
Discussion started by: hareeshram
7 Replies

2. UNIX for Dummies Questions & Answers

Extracting only words from a log file

hello: i have a file and i am trying to extract only unique words from that file. i used the command: cat messages.1 | tr " " "\n" | sort | uniq -c but using this command outputs everything unique in the file be it words, numbers, like all the characters..i need a command which will only... (6 Replies)
Discussion started by: vikbenq
6 Replies

3. Homework & Coursework Questions

extracting date from log file

You are given a 1 year logfile with each line starting with a date in the form “YYYY-MM-DD”. How would you extract logs from the 4th day of each month and put them into a new file (1 Reply)
Discussion started by: DOkuwa
1 Replies

4. Shell Programming and Scripting

Extracting data from a log file with date formats

Hello, I have a log file for the year, which contains lines starting with the data in the format of YYYY-MM-DD. I need to get all the lines that contain the DD being 04, how would I do this? I tried using grep "*-*04" but it didn't work. Any quick one liners I should know about? Thank you. (2 Replies)
Discussion started by: cpickering
2 Replies

5. Shell Programming and Scripting

Extracting log entries from a date onwards

One of the log file looks like entries as below. Wed Apr 6 14:51:18 2011 FAIL LOGIN: Client "9.191.21.54" Wed Apr 6 14:52:53 2011 CONNECT: Client "9.191.21.54" Wed Apr 6 14:52:54 2011 OK LOGIN: Client "9.191.21.54" Wed Apr 6 14:55:10 2011 CONNECT: Client "9.191.21.54" Wed Apr 6... (2 Replies)
Discussion started by: rijeshpp
2 Replies

6. Shell Programming and Scripting

Extracting strings from a log file.

I'm new to all this and I've been fiddling with this problem for HOURS and feel silly that I can't work it out! I have a .log file that VERY long and looks like this: 2011-08-31 10:03:34 SUESTART AG Amndmnt Client WebRequest DNU SUEEND Sequence: 600, 2011-08-31 10:03:34 SUESTART... (11 Replies)
Discussion started by: SusieSA
11 Replies

7. Shell Programming and Scripting

Extracting the last 10mins worth of data in a log file

Hi all, Hope someone here will be able to help me. Our system has some scripts that are run from a cron job every ten mins and is used to see how many error there are in that time frame. Problem is that in the scripts grep is used to find the data, but as the day goes on these log file grow to a... (7 Replies)
Discussion started by: Goldengreen
7 Replies

8. Shell Programming and Scripting

Extracting Delimiter 'TAG' Data From log files

Hi I am trying to extract data from within a log file and output format to a new file for further manipulation can someone provide script to do this? For example I have a file as below and just want to extract all delimited variances of tag 32=* up to the delimiter "|" and output to a new file... (2 Replies)
Discussion started by: Buddyluv
2 Replies

9. Shell Programming and Scripting

Error while extracting data from log file

I am running awk command to extract data from log file to calculate last 15 minutes log using below command and now i am getting bellow error: awk '$0>=$from' from=$(`date -u +"####<%d-%b-%Y %H:%M:%S o'clock GMT>"-15min`) test.log Error: date: 0551-402 Invalid character in date/time... (8 Replies)
Discussion started by: oberoi1403
8 Replies

10. Shell Programming and Scripting

Extracting data from rsync log

Hi I have a daily rsync that runs and i am trying to find a easy way of extracting the start time and end time of the sync and extract the details of how much data was copied. I would like to use this information to see if i can increase the amount of rsyncs that run in a day. so the log... (3 Replies)
Discussion started by: treds
3 Replies
All times are GMT -4. The time now is 02:34 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy