Login or Register to Ask a Question and Join Our Community


UNIX for Advanced & Expert Users

cron job to extact lines from files to another file and ftp to new server

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users cron job to extact lines from files to another file and ftp to new server
# 1  
Old 08-10-2012
cron job to extact lines from files to another file and ftp to new server
i have a text file in this format: which creates a new one everyday in the form of filename _zing__r200_2012_8_10_log.txt

Code:
Fri Aug 10 07:29:17 EDT 2012, usera(192.168.0.245) to anotheruser: hey top, this is a private test
Fri Aug 10 07:29:28 EDT 2012, anotheruser(192.168.0.245) to usera: got you top
Fri Aug 10 07:29:33 EDT 2012, anotheruser(192.168.0.245) to usera: returning the test
Fri Aug 10 07:29:58 EDT 2012, anotheruser(192.168.0.245) to usera: this is a test
Fri Aug 10 07:30:05 EDT 2012, anotheruser(192.168.0.245) to all: hey folks
Fri Aug 10 07:30:09 EDT 2012, anotheruser(192.168.0.245) to all: what is going on??
Fri Aug 10 07:30:22 EDT 2012, usera(192.168.0.245) to all: chilling, chilling top
Fri Aug 10 07:30:28 EDT 2012, usera(192.168.0.245) to all: what's up with you  ??
Fri Aug 10 07:30:51 EDT 2012, usera(192.168.0.245) to anotheruser: another tes top before disabling pv
Fri Aug 10 07:30:54 EDT 2012, usera(192.168.0.245) to anotheruser: got me ??
Fri Aug 10 07:31:02 EDT 2012, anotheruser(192.168.0.245) to usera: loud and clear

I would like to run a cron job to extract all the lines wich has the pattern Fri Aug 10 07:30:05 EDT 2012, anotheruser(192.168.0.245) to all: to a new file like 2012_8_10_log.txt which I will either scp or ftp to another server.

my ultimate goal is to take that file, use regualr expression such as

Code:
$patterns = array();
$patterns[0] = '/\,/';
$patterns[1] = '/(\(.*\))/';
$patterns[2] = '/\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}/';
$patterns[3] = '/(?<=all)\:/';

to get each line like this
Code:
Wed Jul 04 08:23:55 EDT 2012|~|anotheruser|~| testeeeeeeeeeeeeeeeeettttttttttttttttttttt, this a line test

and insert in mysql using |~| as delimiter.

any help, tip or pointer ??

thanx
Moderator's Comments:
Mod Comment Please view this code tag video for how to use code tags when posting code and data.
Last edited by Corona688; 08-10-2012 at 11:10 AM..
# 2  
Old 08-10-2012
awk understands tokens, which lets you do things on 'the third column' instead of a big regex with width specifiers, so I'd use it here to make things simpler.

Code:
awk '{ sub(/,/, "|~|", $6); # Change , to |~| in column 6
                  sub(/[(].*/, "", $7); # Delete everything in column 7 after first (
                  $8=""; # Blank to
                  $9="|~|"; # blank username and replace with |~|
                  }' inputfile > outputfile

This User Gave Thanks to Corona688 For This Post:
bkkid
# 3  
Old 08-10-2012
I'll be sure to use [code] next time

The regex was for the php side. sorry

I do not quite get it, how would awk know the 6th column and on?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script that will be a cron job to export rrd files for cacti server

I wrote a quick little script that will eventually end up as a cron job to export rrd files for my cacti server. Here is the script: #!/bin/bash rm -rf /backup/cacti_xml/* cd /var/www/html/rra ls -1 *.rrd | awk '{print "rrdtool dump "$1" > /backup/cacti_xml/"$1".xml"}' | sh -x Is there... (5 Replies)
Discussion started by: TheBigAmbulance
5 Replies

2. Shell Programming and Scripting

Echo print in different lines within email sent by Cron job

Hi all, I think this could have a simple solution, just I canīt get it so far. I have the script below that includes several echo commands in order to show that every part of the script have been executed. A cron job executes this script and when is completed the output is sent by email. ... (4 Replies)
Discussion started by: cgkmal
4 Replies

3. UNIX for Dummies Questions & Answers

Keep Server Alive with Cron Job

Despite my best efforts, my media streaming server still dies sometimes and I am in a random place trying to ssh into the server to restart it on my cell phone after customers start calling.... I tried using google to track down a script that would do the following Every 5 min execute: sudo... (1 Reply)
Discussion started by: ajhalls
1 Replies

4. UNIX for Dummies Questions & Answers

If FTP cron job fails to connect = quit

Im having a problem with my ftp CRON jobs. I have another related (but not the same question in shell scripting). My FTP CRON jobs connect to a server and drop some files off, if for some reason it cannot connect to the server (timesout/password wrong etc) instead of stopping, it tries again... (3 Replies)
Discussion started by: mokachoka
3 Replies

5. Shell Programming and Scripting

Im trying to create a FTP Cron Job

Hello, Im fairly new to this and am hoping for your help in moving forward. 1. I need to carry out a FTP transfer from SERVER1 to SERVER2 at 3am every morning. 2. The FTP is to work on SERVER1 3. There will be many files to transfer (96 files per day) 4. I want to delete the files... (3 Replies)
Discussion started by: nozerf
3 Replies

6. UNIX for Dummies Questions & Answers

Server moved, new IP, broken CRON job

Due to downsizing and attrition, I have inherited SysAdmin tasks, but unfortunately not all the required knowledge and skills came to me along with the assignment -> so I appreciate any advice and help ( be patient with my newbie terms and questions). We moved a central server and changed it's... (1 Reply)
Discussion started by: HikerLT
1 Replies

7. UNIX for Advanced & Expert Users

Cron FTP job

We have a script that we run manually, we want to set it up so that it will run automatically via cron, the problem is during the ftp process nothing is put, the script logs in changes directory fine, but nothing is put to the directory. I have tried it as root, a power user, etc. I have made the... (4 Replies)
Discussion started by: marvin51796
4 Replies

8. UNIX for Dummies Questions & Answers

cron ftp job help

I am new to cron and am trying to set a cron job that will run everyday at 2 to go out to a server (via ftp with user name and password) and get a file and then bring it back to my sever. Below is what I have but it doesn't seem to be working. Any help would be appreciated. cron command - will... (4 Replies)
Discussion started by: bernfuss
4 Replies

9. UNIX for Dummies Questions & Answers

ftp in cron job

I am writing a script that will be placed in a crontab, which basically retrieves a file that is overwritten everyday (with current customer data) and then sent to another company. I was wondering what the best way to ensure that ftp was successful and notification of a success or failure message.... (3 Replies)
Discussion started by: walterja
3 Replies

10. UNIX for Dummies Questions & Answers

CRON Job - saving to a different server?

Thanks for the all the help last evening on CRON jobs. It's now working. Is it possible with cron to save a backup of a file to a different server or location? The idea being that if one host goes arse over tit, I'll have a backup of the DB on another server? Many thanks, Ed Ludlow (3 Replies)
Discussion started by: eludlow
3 Replies
Login or Register to Ask a Question