Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Logfile manipulation

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Logfile manipulation
# 1  
Old 11-21-2009
Logfile manipulation
Hi
First of all I m a complete newbie to Linux ... just started working on it exactly a week ago. I know a bit of programming in C but not very good in it.Smilie

I was given task in my workplace and need some help

Code:
nAJLR02F030879     9805 Thu Nov 19 13:27 <customerservice@YYY.com>
                 (Deferred)
                     <XXXX@yahoo.com>

This is a sample of the logfile I have. Using the great search engine Google and some of unix.com I was able to write

Code:
cat testdata | grep -o '[[:alnum:]+\.\_\-]*@[[:alnum:]+\.\_\-]*' | grep -vw '[[:alnum:]+\.\_\-]*@YYY.com'  >to
cat testdata | grep -o '[[0-9][0-9]]*:[[:digit:]]*' >time
cat testdata | grep -o '[[:alnum:]+\.\_\-]*@YYY.com' >from

paste time from to -d , >Dummy

The thing is I need to add month and date to the time file so that its looks like

Nov 19 13:27

and convert it to Epoch time .So that my final Dummy file will in this format

EpochTime,From@from.com,To@To.com

Any help will do...

Disclaimer : If this is a double post can u link this to the correct post. Thx Smilie
Last edited by pludi; 11-21-2009 at 10:58 AM.. Reason: Code tags, please...
# 2  
Old 11-21-2009
Not very robust but it might serve your need:
Code:
egrep -o '[^ ]* [^ ]* [^ ]*:[^ ]*|[^<]*@[^>]*' testdata|
while read word; do
  case $word in
   *@*) printf ",$word" ;;
     *) printf "$(date -d "$word" '+%s')";;
   esac
done
echo

# 3  
Old 11-22-2009
CPU & Memory Got a solution
Code:
cat testdata | grep -o '[[:alnum:]+\.\_\-]*@[[:alnum:]+\.\_\-]*' | grep -vw '[[:alnum:]+\.\_\-]*@a-cti.com'  >to
cat testdata | grep -o '[A-Z][a-z][a-z] [0-9][0-9] [[0-9][0-9]]*:[[:digit:]]*' >time
cat testdata | grep -o '[[:alnum:]+\.\_\-]*@a-cti.com' >from

date +%s -f time > Etime

paste Etime from to -d , >Dummy
sed 's/$/,0/' Dummy > FDummy

Hey this worked ....

Hey Scrutinizer can tell me how your script runs ... step by step... I wish learn more but i dont know much... Smilie

---------- Post updated 22-11-09 at 10:23 AM ---------- Previous update was 21-11-09 at 05:37 PM ----------

I used this code

Code:
cat testdata | grep -o '[[:alnum:]+\.\_\-]*@[[:alnum:]+\.\_\-]*' | grep -vw '[[:alnum:]+\.\_\-]*@a-cti.com'  >to
cat testdata | grep -o '[A-Z][a-z][a-z] [0-9][0-9] [[0-9][0-9]]*:[[:digit:]]*' >time
cat testdata | grep -o '[[:alnum:]+\.\_\-]*@a-cti.com' >from

date +%s -f time > Etime

paste Etime from to -d , >Dummy
sed 's/$/,0/' Dummy > FDummy

So my FDummy file looks like this

Code:
1258666080,customerservice@XXX.com,abcde@yahoo.com,0
1258644660,txt@XXX.com,bdefg@jkllo.com,0

so my next challenge is to compare this file with another file called FDummy2 containing same pattern of data ie
EPOCHTIME,FORMADDRESS,TOADDRESS,0

and if any any line of data/string from FDummy is new to FDummy2 I must append/added as new line it to FDummy2. EPOCHTIME could be ignored.


Any ideas. I m stumped and dont have a clue what command to use. Also if u are gonna post a code could also explain how it works. I m like 85% newbie to linux learning as I go.Smilie
Last edited by pludi; 11-21-2009 at 10:59 AM.. Reason: removed links
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Logfile monitoring with logfile replacement

Bonjour, I've wrote a script to monitor a logfile in realtime. It is working almost perfeclty except for two things. The script use the following technique : tail -fn0 $logfile | \ while read line ; do ... some stuff done First one, I'd like a way to end the monitoring script if a... (3 Replies)
Discussion started by: Warluck
3 Replies

2. Shell Programming and Scripting

Logs from logfile

Hi Team, Have to write a shell script to pick only 1 hr logs from the generated logfile and send it to other logfile. Thanks & Regards, Indu (3 Replies)
Discussion started by: indira_s
3 Replies

3. Shell Programming and Scripting

Parsing Logfile

Hi, I need to continuously monitor a logfile to get the log information between a process start and end. the logfile look like this abcdddddddddd cjjckkkkkkkkkkkk abc : Process started aaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbbb abc... (6 Replies)
Discussion started by: Byorg
6 Replies

4. UNIX for Dummies Questions & Answers

writing to a logfile

I cannot get anything to go to my log. This is what i see on my screen when i run my korn shell. cd ok, cwd=/export/home/tsp_inst/TSP pget: /nas4/edata/tsp/rawdata/test2.gz: File exists But i cant get it to write it to my log. cd /this/is/my/newdirectory lftp -e 'pget -c -n 4... (1 Reply)
Discussion started by: tyngsboro
1 Replies

5. Shell Programming and Scripting

looking for string in logfile

I have a shell script that used to look for a particular sting in the last line of a log file. Howeve this string now has moved to the 3rd or 4th line from bottom. Can anyone point me to how i easiest chage this one to look within the last frew lines (5 or so) for the particular string rather... (6 Replies)
Discussion started by: jjlinux
6 Replies

6. UNIX for Dummies Questions & Answers

change value of logfile

Hi, I have a log file like exyymmdd.log and i need to do is read the log file and inside the logfile if i find the word which starts with PQR002106570.book then i need to replace the value of PQRXXX.book(this is unique) with its corresponding title. A list of id(PQRXXX.book) and its title are... (5 Replies)
Discussion started by: umapearl
5 Replies

7. Shell Programming and Scripting

logfile parsing

I thought I was pretty handy with awk until I got this one. :) I'm trying to parse a log file where the events could have different delimiters (2 scripts is ok), the errors are spread over multiple lines, and I"m trying to figure out how to not read the same lines that have already been read. ... (1 Reply)
Discussion started by: linkslice
1 Replies

8. Shell Programming and Scripting

logfile

hi iam new of the ksh script.iwant in formation of how to call in logfile in ksh scripts. if the meaning in ksh. please help me thanks naveen.g (1 Reply)
Discussion started by: naveeng.81
1 Replies

9. Shell Programming and Scripting

last month's logfile

hi friends I need a shell script which will do the following Task Enter the month : if you enter 1 then it ll show you last 1 month's (starting from today).log file in the current directry. if you enter 4 then it ll show you last 4 month's (starting from today).log file in the current... (2 Replies)
Discussion started by: deep_kol
2 Replies

10. Shell Programming and Scripting

sort a logfile

Hi experts, I need help to sort a big logfile. logfile: ------- 67712 dkjd jd jj jjjj ------ kjkj jhjh kkk yggg lll hhh gffgf jj -------- i kkk kllkkl ------- Now I want every think between the "------" in one line. Normaly with paste no problem but you can see that the... (7 Replies)
Discussion started by: joerg
7 Replies
Login or Register to Ask a Question