Login or Register to Ask a Question and Join Our Community


UNIX for Beginners Questions & Answers

Fetching address and user details from log file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Fetching address and user details from log file
# 1  
Old 05-29-2018
Fetching address and user details from log file
Hi All,

I have a requirement to get the address values from a large log file along with the user details.
line1,line2,city,stateCode,postalCode,countryCode. The below code as advised in the earlier post is giving the user data

Code:
zgrep -B1 "Failed to calculate  Tax" log.2018-05-23.gz | grep "CartService" | grep -Eo '.{4}-.{2}-[^]]*|.{2}:.{2}:.{8}' | paste - -|sort -u -k3,3

Results.
Code:
2018-05-23 11:01:31,252 XX:XX:CC061367
2018-05-23 23:17:44,611 XX:XX:EW976815

The log file is as below.

Code:
[2018-05-23 11:01:31,251] app14.xx.xx.net 1527087690803 XX:XX:CC061367 http-nio-8080-exec-6 ERROR Get Price Failed! CC061367 , XX , XX (AbstractLocalP
ricingServiceV1)
[2018-05-23 11:01:31,252] app14.xx.xx.net 1527087690803 XX:XX:CC061367 http-nio-8080-exec-6 ERROR Exception while search for products (CartService)
com.xx.xx.xx.xx.xx.xx: Error: Failed to calculate  Tax. Error Number: '2' Error Description: 'A valid city/state or zip code required.' | Params: Address={
  "line1" : "Haci # 34",
  "line2" : " Vallescondido",
  "city" : " Zaragoza",
  "stateCode" : "MEX",
  "postalCode" : "52937",
  "countryCode" : "US"
} taxType=Sales

Please advise how to get the address values along with the user details with heading like below.

Expected Results
Code:
Date    Program    line1    line2    city    stateCode    postalCode    countryCode
2018-05-23 11:01:31,252    XX:XX:CC061367    Haci # 34,     Vallescondido,     Zaragoza,    MEX,    52937,    US

2018-05-23 11:01:31,252    XX:XX:UWA38046    3-5-10 Haruhino,     Asao,     Kawa,    KN,    215,US

Attempted couple of commands ,one sample as below but was not giving any results.

Code:
zgrep -B1 "Failed to calculate US Tax" log.2018-05-23.gz | grep "CartService" | grep -Eo '.{4}-.{2}-[^]]*|.{2}:.{2}:.{8}' | paste - -|sort -u -k3,3|zgrep -A2 "Failed to calculate Tax" log.2018-05-23.gz|egrep "line(1|2)"|egrep "city" |sort -u




Moderator's Comments:
Mod Comment Please use CODE tags - for data as well! - as required by forum rules!
Last edited by RudiC; 05-29-2018 at 07:56 AM.. Reason: Added CODE tags.
# 2  
Old 05-29-2018
Did you consider that (the first) zgrep output is unzipped, clear text, so the second zgrep will mess that up?
# 3  
Old 05-29-2018
Sorry, I didnt get the point

---------- Post updated at 06:09 AM ---------- Previous update was at 06:06 AM ----------

---------- Post updated at 06:10 AM ---------- Previous update was at 06:09 AM ----------

I will put code tag for data as well. Thanks
# 4  
Old 05-29-2018
Double checking man zgrep I see it will accept and grep straight text as well - so this is not in error. But by supplying the input file again to the second zgrep as an argument, you lose the results of the first one, because the pipe pipes into nirvana.
Please be aware that
- neither search pattern will produce a match (there's no "US" in front of "Tax", but TWO spaces).
- egrepping "line(1|2)" would match two lines IF above had found a match, but egrepping "city" will drop both lines.
So - there are several reasons it "was not giving any results".
# 5  
Old 05-29-2018
Understood, there should be keyword US in the search pattern , the log file is having the entry , it was omitted accidentally.
Could you please advise any other alternative solutions.Am not familiar with multiple search patterns
# 6  
Old 05-29-2018
As any regex matching is utmost sensitive to the pattern, please provide the correct one.
# 7  
Old 05-29-2018
Please find the log

Code:
[2018-05-23 11:01:31,251] app14.xx.xx.net 1527087690803 XX:XX:CC061367 http-nio-8080-exec-6 ERROR Get Price Failed! CC061367 , XX , XX (AbstractLocalP
ricingServiceV1)
[2018-05-23 11:01:31,252] app14.xx.xx.net 1527087690803 XX:XX:CC061367 http-nio-8080-exec-6 ERROR Exception while search for products (CartService)
com.b2s.service.pricing.pricingapi.exception.InvalidCartRequestException: Error: Failed to calculate US Tax. Error Number: '2' Error Description: 'A valid city/state or zip code required.' | Params: Address={
  "line1" : "Haci # 34",
  "line2" : " Vallescondido",
  "city" : " Zaragoza",
  "stateCode" : "MEX",
  "postalCode" : "52937",
  "countryCode" : "US"
} taxType=Sales

Last edited by nextStep; 05-29-2018 at 10:58 AM.. Reason: Logs put under Code tag
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Cybersecurity

You are being directed to the US FBI where your IP address and details will also be logged.

Well, I thought I would share this code, which I normally do not share. The logs show this site is currently experiencing some "new hack" maybe attempting a buffer overflow or something similar. So, not really having any patience for this kind of criminal or childish behavior (have a touch... (7 Replies)
Discussion started by: Neo
7 Replies

2. Shell Programming and Scripting

How to log file processing details to database table usnig UNIX shell script?

we are getting files on daily basis.we need to process these files. i need a unix shell script where we can count 1-The no of files processed 2-No of data/record processed for each files. The script should log these details into a database table. If there is any error while file... (3 Replies)
Discussion started by: Atul kumar
3 Replies

3. Shell Programming and Scripting

Extract IP Address from Log File

I have a log file with several IP addresses in it:- 2012-12-06 16:05:05,885 NOTICE **SNMP** Alarm was created: (LicenseClientRejected) Client Remote Peer /125.111.64.125:2573 was Rejected Property=/125.111.64.125:2573, Client Remote Peer /125.111.64.125:2573 was Rejected **SNMP**.... (5 Replies)
Discussion started by: BHM
5 Replies

4. Shell Programming and Scripting

Formatting Report and Reading data and fetching the details from contents file

Data I was trying to write shell script which will be return the output in the below format First i was trying to do these using sed. sed -n '/.ksh/p' mainksh.ksh sed -e 's/*\(.*\)/\1/g' mainksh.ksh $RUN_DIR, $SUB_DIR and the variables which will be defined in the profile file. when i am... (0 Replies)
Discussion started by: rameshds
0 Replies

5. Solaris

Command History for a user with IP details

Hi We are sharing our envoirnment with our component teams. the plateform is SunOS 5.8 Generic_117350-41 sun4u sparc SUNW,Sun-Fire-V490. All team logs in the domain with same user and perform activities. Now for the system auditing purpose can somebody guide me how can I get the details when... (6 Replies)
Discussion started by: sukhvinder_Tm
6 Replies

6. UNIX for Dummies Questions & Answers

Panic kernal-mode address fault on user address 0x14

:) Firstly Hi all!!, im NEW!! and on here hoping that someone might be able to offer me some help... i have a server that keeps crashing every few days with the error message: PANIC KERNAL-MODE ADDRESS FAULT ON USER ADDRESS 0X14 KERNAL PAGE FAULT FROM (CS:EIP)=(100:EF71B5BD) EAX=EF822000... (10 Replies)
Discussion started by: Twix
10 Replies

7. Solaris

user logon details

how can i identifying whose are logged in last few days,time and date also want. what i will do for get that information (2 Replies)
Discussion started by: sijocg
2 Replies

8. UNIX for Dummies Questions & Answers

Script for parsing details in a log file to a seperate file

Hi Experts, Im a new bee for scripting, I would ned to do the following via linux shell scripting, I have an application which throws a log file, on each action of a particular work with the application, as sson as the action is done, the log file would vanish or stops updating there, the... (2 Replies)
Discussion started by: pingnagan
2 Replies

9. UNIX for Dummies Questions & Answers

logging SFTP details in a log file...

hi all..... i want to know how to log the details when logging into a server using SFTP......in FTP i used something like (ftp -uv<xxx.srp>>log_file.log) where the details will be logged to log_file...is there any options for doing the same in SFTP....i wanted to display the details abt... (4 Replies)
Discussion started by: santy
4 Replies

10. Shell Programming and Scripting

Read relevent details from Log File

Hi All, We have requirement to read XML details from large Log File and save in new file name. I have enclosed the sample xml which has start tag <OUTBOUND_MESSAGE xmlns="http://www.abc.com"> and the end tag as </OUTBOUND_MESSAGE> The log file contains many lines with other details which... (6 Replies)
Discussion started by: thinakarmani
6 Replies
Login or Register to Ask a Question