Hi there,

I receive some log files by email every day. I read them with Firefox (MS Windows) and interpret them. The problem is that I'm getting more and more log files. So I wondered if I could find a way to:
1) Configure a mail box on a linux server
2) automatically save every mail as a file
3) parse the files with sed (or similar) to easily extract the error messages
nb: my linux server is just command line (I have no GUI)

I never did neither point 1 nor 2. But if I can deal with them, I think I will be able to work out my sed scripts to do the job for me.

Does anyone knows if it is possible to received email and store them as files? Is there a simple client to install? Do you have any tip for the configuration?

Thanks in advance
Santiago

(1 & 2) Which is the actual Mail server? Does it run on said Linux server? Where do the log files come from? Other Linux servers?

(2) Read up on procmail. It's not necessary, but it can help depending on the situation.

(3) Well, when you get to (3), post specific examples in the "shell scripting" forum.

Good luck.

Hi otheus and thanks for considering my request.

The mail server is simply gmail (more precisly, my own domain managed through google app but I think that makes no difference). Thus, I want to be able to download emails sent to logs@mydomain.com (from pop.gmail.com). I want to download the emails to my linux server for further processing.

The log files are generated by a software running on windows clients. They are sent by the software itself to the specified address. I've not created the software and mail is the only way I have to receive information from the program.

I'm currently trying to get started with procmail.


Hi pludi and thanks as well.
I'm afraid I did not express myself correctly. I don't want to create a mail server. I just want to download emails to a linux computer and parse them to catch error lines.
I have 7 windows computers running a software that sends logs every once in a while (not even punctual). Today, I have to read through numerous mails to check for errors. Tomorrow I'd like those email to be fetched by a debian server of my own and analysed by sed so I don't have to do it myself.
Actually I have two options :
1) Either I tell the windows program to send logs as attachments. Thus I need my debian mail client to be able to extract attachments as files.
2) Either I tell the windows program to send logs in the mail body. Thus I need my debian mail client to be able to turn email into a file.

I will soon try to see how fetchmail works.

1) An additional mail server might not be needed if you can create a new mailbox on the existing mail server and receive those mails using fetchmail
2) Either procmail or a script that splits the Mail file for you

Right, so use a tool like slurp or fetchmail to snatch email from gmail/pop servers. According to the FAQ, you can use fetchmail's preconnect feature to filter the emails it receives through another utility called mailfilter. There's a configuration example here. Most notably, you want something like:
What I'm not sure about is whether you can configure fetchmail/mailfilter to ignore all your other emails without deleting them.

In addition, or independently, you can use Gmail to recognize these emails, put a label on them, and archive them immediately. Then you use your normal email client to simply ignore these or not to download them. You can also have Gmail forward these particular emails to another mail server, from which you can use fetchmail to download.

Once again, I wasn't clear in my request.

I don't need to filter my emails, my personal address is santiago.diez@mydomain.fr. The logs are sent to logs@mydomain.fr. So mails containing logs are perfectly traceable and I know how to ignore them in my gmail interface.

The problem is that I can't avoid to read the logs. I have to read them because they contain information that I need to take actions. And this information is very hard to pull out. If I had to seek for a line that would match the word ERR or bug or wathever, I would do a filter with gmail. Instead, I need to make comparison between lines to detect mistakes and do calculation to extract necessary values to take actions.
Another problem is that the program that sends the logs cuts them every 1000 lines. So I need to join the emails after I receive them in order to be able to compare line 1000 of log #24 with line 1 of log #25.
Also remember that I receive emails from several computers. The email subjects are like "computername: log#98"

To achieve this, I do a stupid thing:
1) I download my emails with Outlook Express
2) I move log mails to a file system directory (so now the email is just a file.eml)
3) I run a ms-dos batch to join mail bodys from the same computer (so I have only one big log file per computer)
4) I open the new log files with MS Excel to parse them and take appropriate action

This takes me time and I have to think of doing it. That's why I thought I could give that job to my debian server because it would not mind doing it repeatedly even every hour. I know I would be able to write a shell script to do 3 & 4 using sed. But I don't know how to retrieve emails and turn them into files so I can sed them.

Sorry guys to take so much time in explaining the problem and thanks for your help.

With gmail you can do this really cool thing: add +something to the email address. So you could send the logs to santiago.diez+logs@mydomain.fr. Now you download the emails with fetchmail/mailfilter, and save the appropriate files to a UNIX folder. (I'm not sure if it saves individual files, or if it uses the "mbox" format.) Then you write an awk or perl script to remove the mail headers, join the lines (as needed).

You could also do step(1) and (2) for now as is, but 2b: ftp/smb copy the files to a Unix box (or a foreign virtual unix environment running on your workstation) and run this script.

If you do it this way, however, there may be issues unique to the MSDOS/eml format, so first chose which method you should do, then (we) can help you with the rest.