script to pull info from my email?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting script to pull info from my email?
# 1  
Old 04-01-2009
script to pull info from my email?

Hi, I need help writing a script that would pull info from an email inbox and add it to an Evolution Calendar. I'm pretty sure I can google the commands to add the info to the calendar. The part that I really need help with is getting the info from the email into the command. Basically for work, when I accept an assignment, I get an email from an automated system that provides the job details. I'd like to pull the "who", "when" and "where" info and add that to the calendar. The emails are formated in a consistent fashion so the info I need will always be given in the same way.
Any help would be greatly appreciated as I'm new scripting and commands like 'grep'

Thank you.
# 2  
Old 04-02-2009
Ok so I'm thinking that AWK might be the best way to go but I have run into a problem that I'm sure has a solution. Basically I don't know how to use a string with spaces. I'll elaborate a bit more on what I'd like to do. Basically the body of the emails is similar to this:

Code:
You have been assigned a job starting on 01/23/4567.

The following are the details of the job:

*************
 Job Summary 
*************
Starting on              : 01/23/4567
Place                   : Name Of Place                   
Person                  : Lastname, Firstname


**********
 Job Days 
**********
Place                                             Date     From    To     
--------------------------------------------- -------- ------- -------
Name Of Place                                 01/23/45  8:20AM  3:00PM


Please do not reply to this system generated message.

I can get AWK to give me the name of place with:
Code:
awk '/Place :/ {print $3,$4,$5}' testfile

This gives me the output "Name Of Place" which is good, but the next thing I need to do is get the date and time. This info is on a line further down. I hoped that this command would work, but it doesn't:
Code:
awk '/Name Of Place/ {print $4,$5,$6}' testfile

it returns this output:
Code:
Of Place
01/23/45  8:20AM  3:00PM

What do I need to modify so that it only gives the date and time?
# 3  
Old 04-02-2009

Code:
awk '/^Name Of Place/ {print $(NF-2),$(NF-1),$NF}' testfile

# 4  
Old 04-02-2009
I tried the suggested command and didn't get any output, but it got me on the right track. So, thank you very much.
It was the ^ that I was missing.
Code:
awk '/^Name Of Place/ {print $4,$5,$6}' testfile

This gives me the date and time info.

At least now I know how to get the data I need from a testfile. Next I need to figure out how to get it from my inbox. This shouldn't be too tricky as I'm guessing that there are quite a few posts in this forum that deal with email inboxes.
# 5  
Old 04-08-2009
Hi, I've had some success at writing this script. I'm sure it's not the most graceful and it still needs some work, but, it's a good start. I've had to search quite a bit for things that I'm sure are fairly simple to do. I could use some help adding a bit of functionality that I need in the script.
So far I have the emails coming into a directory and a separate file is made for each one (getmail does this). This script goes through the directory and gets the info I need from the email and adds this to a calendar file(.ics). Right now I have it confined to a single test file until I can have it do everything it needs to do. Once I have it ready to use on all files that come, I'll add code for it to move the parsed email out and work through all the files in the directory until it's empty. The script checks a few line that are in the emails but aren't in the example I gave before.

Heres the script:
Code:
#!/bin/bash

NEWEST=$(ls -lrt | awk '{ f=$NF }; END{ print f }')
TMPNAME=$(awk '/Confirmation No.         :/ {print $4}' $NEWEST)
PLACE=$(awk '/Place                   :/ {print $3,$4,$5,$6,$7}' $NEWEST)
WHO=$(awk '/Person                  :/ {print $4,$5,$3}' $NEWEST)
TITLE=$(awk '/Title                    :/ {print $3,$4,$5,$6,$7}' $NEWEST)
DATE=$(awk "/^$PLACE/ "' {print $(NF-2)}' $NEWEST)
STIME=$(awk "/^$PLACE/ "' {print $(NF-1)}' $NEWEST)
ETIME=$(awk "/^$PLACE/ "' {print $(NF)}' $NEWEST)
DTSTART=$(date --date "$DATE $STIME +5 hours" +"%Y%m%d%H%M%S")
DTEND=$(date --date "$DATE $ETIME +5 hours" +"%Y%m%d%H%M%S")
DSTAMP=$(date --date '+5 hours' +"%Y%m%d%H%M%S")


echo -e "BEGIN:VEVENT\n\
DTSTART:$DTSTART\n\
DTEND:$DTEND\n\
DSTAMP:$DSTAMP\n\
UID:$TMPNAME-Sync-Script\n\
CLASS:PRIVATE\n\
CREATED:$DSTAMP\n\
DESCRIPTION:$TITLE\n\
LAST-MODIFIED:$DSTAMP\n\
LOCATION:$PLACE\n\
SEQUENCE:0\n\
STATUS:CONFIRMED\n\
SUMMARY:$WHO\n\
TRANSP:OPAQUE\n\
BEGIN:VALARM\n\
ACTION:DISPLAY\n\
DESCRIPTION:This is an event reminder\n\
TRIGGER:-P0DT0H30M0S\n\
END:VALARM\n\
END:VEVENT" > $TMPNAME

awk '{if (/END:VCALENDAR/) system("cat '$TMPNAME'"); print}' basic.ics > $TMPNAME.ics

mv $TMPNAME.ics basic.ics
rm $TMPNAME

This work well (mostly) and leaves me with a file that I can use with my calendar (I've decided against evolution). I say this works well because normally the email wil have only one line like this:
Code:
Name Of Place                                 01/23/45  8:20AM  3:00PM

But sometimes there will be more than one, e.g.:
Code:
Name Of Place                                 01/23/45  8:20AM  3:00PM
Name Of Place                                 01/24/56  8:20AM  3:00PM

The script gets the job date, start and end time from this line and it works well when there is only one line. Sometimes (not too often) the job will be on multiple consecutive days. So this line will be repeated with a different date.

How can I have the script add an entry for each line (day)?

Last edited by xinix; 04-08-2009 at 01:22 AM..
# 6  
Old 04-08-2009
Quote:
Originally Posted by xinix
Hi, I've had some success at writing this script. I'm sure it's not the most graceful and it still needs some work, but, it's a good start. I've had to search quite a bit for things that I'm sure are fairly simple to do. I could use some help adding a bit of functionality that I need in the script.
So far I have the emails coming into a directory and a separate file is made for each one (getmail does this). This script goes through the directory and gets the info I need from the email and adds this to a calendar file(.ics). Right now I have it confined to a single test file until I can have it do everything it needs to do. Once I have it ready to use on all files that come, I'll add code for it to move the parsed email out and work through all the files in the directory until it's empty. The script checks a few line that are in the emails but aren't in the example I gave before.

Heres the script:
Code:
#!/bin/bash

NEWEST=$(ls -lrt | awk '{ f=$NF }; END{ print f }')


Why are you reversing the output of ls and then getting the last line instead of just getting the first?

Code:
NEWEST=$( ls -t | head -1 )

Or:

Code:
set -f; set -- $( ls -t )
NEWEST=$1

Quote:
Code:
TMPNAME=$(awk '/Confirmation No.         :/ {print $4}' $NEWEST)
PLACE=$(awk '/Place                   :/ {print $3,$4,$5,$6,$7}' $NEWEST)
WHO=$(awk '/Person                  :/ {print $4,$5,$3}' $NEWEST)
TITLE=$(awk '/Title                    :/ {print $3,$4,$5,$6,$7}' $NEWEST)
DATE=$(awk "/^$PLACE/ "' {print $(NF-2)}' $NEWEST)
STIME=$(awk "/^$PLACE/ "' {print $(NF-1)}' $NEWEST)
ETIME=$(awk "/^$PLACE/ "' {print $(NF)}' $NEWEST)


Why seven calls to awk instead of one?

I don't have time to rewrite the awk code at the momment, but use this for a model:

Code:
eval "$( awk '{ printf "var" ++n "=\"%s\"\n", $1 }' "$NEWEST")"

Quote:
Code:
DTSTART=$(date --date "$DATE $STIME +5 hours" +"%Y%m%d%H%M%S")
DTEND=$(date --date "$DATE $ETIME +5 hours" +"%Y%m%d%H%M%S")
DSTAMP=$(date --date '+5 hours' +"%Y%m%d%H%M%S")

echo -e "BEGIN:VEVENT\n\
DTSTART:$DTSTART\n\
DTEND:$DTEND\n\
DSTAMP:$DSTAMP\n\
...
TRIGGER:-P0DT0H30M0S\n\
END:VALARM\n\
END:VEVENT" > $TMPNAME


Why all the unnecessary \ns and backslashes?

Code:
echo "BEGIN:VEVENT
DTSTART:$DTSTART
DTEND:$DTEND
DSTAMP:$DSTAMP
...
TRIGGER:-P0DT0H30M0S
END:VALARM
END:VEVENT" > $TMPNAME

Quote:
Code:
awk '{if (/END:VCALENDAR/) system("cat '$TMPNAME'"); print}' basic.ics > $TMPNAME.ics

mv $TMPNAME.ics basic.ics
rm $TMPNAME

This work well (mostly) and leaves me with a file that I can use with my calendar (I've decided against evolution). I say this works well because normally the email wil have only one line like this:
Code:
Name Of Place                                 01/23/45  8:20AM  3:00PM

But sometimes there will be more than one, e.g.:
Code:
Name Of Place                                 01/23/45  8:20AM  3:00PM
Name Of Place                                 01/24/56  8:20AM  3:00PM

The script gets the job date, start and end time from this line and it works well when there is only one line. Sometimes (not too often) the job will be on multiple consecutive days. So this line will be repeated with a different date.

How can I have the script add an entry for each line (day)?

How do you add an entry to your calendar?
# 7  
Old 04-09-2009
Thanks for some of the comments.
Quote:
Why are you reversing the output of ls and then getting the last line instead of just getting the first?
Basically it was the first working code I found online. Your suggestion is better.

Quote:
Why seven calls to awk instead of one?
Each one gets a unique piece of info that is used for a specific line in the echo command. If there's a single call to that can do it that would be nice. But I just started learning awk stuff a week ago so I'm using what I can get to work for me (even if it might be a bit barbaric).

Basically:
Code:
PLACE=$(awk '/Place                   :/ {print $3,$4,$5,$6,$7}' $NEWEST)

This command gets me the name of the place. I need to know this first because some of the important information is on a line that starts with that name.

Code:
DATE=$(awk "/^$PLACE/ "' {print $(NF-2)}' $NEWEST)

This line gets the mm/dd/yy date from the source file and another call gets the start hour.
This needs to be converted to a yyyymmdd format and time +5 hours, which is done by this:
Code:
DTSTART=$(date --date "$DATE $STIME +5 hours" +"%Y%m%d%H%M%S")

This is repeated to get the finish time.
I'm sure I could consolidate some of these commads. Maybe once I'm a little more experienced.

The "\n\" are not needed.

Quote:
How do you add an entry to your calendar?
The calendar uses a "filename.ics" file (basically an ical file) that several calendar apps seem to understand. This script adds the block of lines that make up an event entry just before the last line which closes the file.
the block looks something like this:

Code:
BEGIN:VEVENT
DTSTART:20090504170000
DTEND:20090504203000
DSTAMP:20090409021153
UID:34533016-Sync-Script
CLASS:PRIVATE
CREATED:20090409021153
DESCRIPTION:Descripion   
LAST-MODIFIED:20090409021153
LOCATION:Name of Place 
SEQUENCE:0
STATUS:CONFIRMED
SUMMARY:Name of Person
TRANSP:OPAQUE
BEGIN:VALARM
ACTION:DISPLAY
DESCRIPTION:This is an event reminder
TRIGGER:-P0DT0H30M0S
END:VALARM
END:VEVENT

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Generic script to recursively cd into directories and git pull

Hi all, I'm trying to write a script to recursively cd into my Git projects and pull them, and will later expand it to build my projects as well. I'm having a bit of trouble with my current script, as I want to supply a command line argument to tell it which branch to check out. I can hard... (2 Replies)
Discussion started by: Cows
2 Replies

2. Shell Programming and Scripting

Shell script to pull certain fields

I/m a beginner so be easy. I have text files that live on an AIX server. The files come in and I've been charged with writing a shell script to email out that pulls the first date, and the last date of the file. I need to load these 2 dates into 2 separate variables. I can figure out the variables,... (13 Replies)
Discussion started by: mattadams1983
13 Replies

3. Shell Programming and Scripting

Bash Script to pull ipa server name on 500 servers

Hello All, I need help writing a bash script that will run on 500 LINUX servers and do the following: 1. Capture the ipa_server name from /etc/sssd/sssd.conf on a list of 500 servers in the ipahosts file. 2. Write to a file outputing only server name and IPA server name. Root ssh keys... (3 Replies)
Discussion started by: vtowntechy
3 Replies

4. Shell Programming and Scripting

Script to pull hashes out of large text file

I am attempting to write a script that will pull out NTLM hashes from a text file that contains about 500,000 lines of data. Not all accounts contain hashes and I only need the ones that do contain hashes. Here is a sample of what the data looks like: There are thousands of other lines in... (6 Replies)
Discussion started by: chango77747
6 Replies

5. Shell Programming and Scripting

Please do help: Perl Script to pull out rows from a CSV file

I have CSV file that contains data in the format as shown below: ABC, 67, 56, 67, 78, 89, 76, 55 PDR, 85, 83, 83, 72, 82, 89, 83 MPG, 86, 53, 54, 65, 23, 54, 75 .. .. .. .. I want to create a script that will pull out the rows from the above sheet and paste it into another CSV file.... (12 Replies)
Discussion started by: pankajusc
12 Replies

6. Shell Programming and Scripting

Need script to pull multiple field from log file

I am hoping to get some help with a script to pull certain fields from a log file. User update (xx6xxx P) rpt (yy6yyy B) 2010/01/20 21:36:01.298 Remote client forward start streamid 85af 2010/01/20 21:36:01.307 rpt2 (ZZ6ZZZ G) rpt1 (YY6YYY B) urcall (CQCQCQ ) mycall (W1AW) user... (5 Replies)
Discussion started by: TedSD
5 Replies

7. Shell Programming and Scripting

How to pull info under headers in file(awk,grep,while loop)

below is an extract from my file and I am trying to use Awk and grep and a while loop to pull infomation from under neath "HBA WWN=".HBA WWN=" reoccurs all over the file but the 100000c.....number are unique and I want to be able to pull and reference specifi information under this header ever time... (2 Replies)
Discussion started by: kieranfoley
2 Replies

8. Forum Support Area for Unregistered Users & Account Problems

Please list email ids or contact info of members

Hi , Is it possible to list the user's email id for further communication. Thanks, MoonwalaPL (3 Replies)
Discussion started by: moonwalapl
3 Replies

9. UNIX for Dummies Questions & Answers

Pull a file from a remote server through a shell script

Hi, I am writing a shell script to pull a file from a remote server (Let say its a windows based remote server). One of my criteria is to pull a file only if it is not empty. We have done a similar script to push a file from our end to a remote server and before pushing it we check for the... (2 Replies)
Discussion started by: sashankkrk
2 Replies

10. Shell Programming and Scripting

Email the changed info in the file

Hey Guyz Just need some help regarding this.I need to send an email(sendmail) to group of users, when ever the content in a file e.g abc.txt changes.We need to send the changed content in the email.We are using bash here.Thanks for your help Guyz --CoolKid (1 Reply)
Discussion started by: coolkid
1 Replies
Login or Register to Ask a Question