Sponsored Content
Top Forums Shell Programming and Scripting Create shell script to extract unique information from one file to a new file. Post 302545586 by Mr_47 on Tuesday 9th of August 2011 12:39:09 AM
Old 08-09-2011
Create shell script to extract unique information from one file to a new file.

Hi to all,

I got this content/pattern from file http.log.20110808.gz

Code:
[07/Aug/2011:07:37:39 +0800] mail1 httpd[14646]: Account Notice: close [192.168.10.128] igchung@abc.com 2011/8/7 7:37:36 0:00:03 0 0 1
[07/Aug/2011:07:37:44 +0800] mail1 httpd[14647]: Account Information: login [192.168.10.131:17187] sastria9@abc.com proxy sid=gFp4DLm5HnU
[07/Aug/2011:07:37:44 +0800] mail1 httpd[14648]: Account Notice: close [192.168.10.131] sastria9@abc.com 2011/8/7 7:37:44 0:00:00 0 0 1
[07/Aug/2011:07:37:45 +0800] mail1 httpd[14647]: Account Information: login [192.168.10.131:17194] sastria9@abc.com proxy sid=gSiaecABc/E
[07/Aug/2011:07:38:37 +0800] mail1 httpd[14646]: Account Information: login [192.168.10.129:2063] pntcdor1@abc.com proxy sid=ZGhAdmqmz3k
[07/Aug/2011:07:38:37 +0800] mail1 httpd[14647]: Account Notice: close [192.168.10.129] pntcdor1@abc.com 2011/8/7 7:38:37 0:00:00 0 0 1
[07/Aug/2011:07:38:38 +0800] mail1 httpd[14646]: Account Information: login [192.168.10.129:2071] pntcdor1@abc.com proxy sid=PtwbGuIk+I4
[07/Aug/2011:07:38:48 +0800] mail1 httpd[14646]: Account Information: login [192.168.10.130:14272] visnet@abc.com proxy sid=4W6xBKPXXvk
[07/Aug/2011:07:38:48 +0800] mail1 httpd[14647]: Account Notice: close [192.168.10.130] visnet@abc.com 2011/8/7 7:38:48 0:00:00 0 0 1
[07/Aug/2011:07:38:48 +0800] mail1 httpd[14646]: Account Information: login [192.168.10.130:14279] visnet@abc.com proxy sid=/qenNd/tps8
[07/Aug/2011:07:38:59 +0800] mail1 httpd[14646]: Account Notice: close [192.168.10.130] visnet@abc.com 2011/8/7 7:38:48 0:00:11 0 0 1
[07/Aug/2011:07:39:06 +0800] mail1 httpd[14647]: Account Information: login [192.168.10.130:14367] animan86@abc.com proxy sid=VdYyCOMtPsQ


how can I generate one new file with content as below, from file above?


Last edited by Scott; 08-09-2011 at 03:14 AM.. Reason: Code tags
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Urgent: selecting unique specific content of a file using shell script

Hi, I have a file whose content and format at places is as given below. print coloumn .... coloumn .... coloumn .... skip 1 line print coloumn ... skip 1 line I need to select the following : print coloumn .... coloumn .... coloumn... (2 Replies)
Discussion started by: jisha
2 Replies

2. UNIX for Dummies Questions & Answers

Extract Unique Values from file

Hello all, I have a file with following sample data 2009-08-26 05:32:01.65 spid5 Process ID 86:214 owns resources that are blocking processes on Scheduler 0. 2009-08-26 05:32:01.65 spid5 Process ID 86:214 owns resources that are blocking processes on Scheduler 0. 2009-08-26... (5 Replies)
Discussion started by: simonsimon
5 Replies

3. Shell Programming and Scripting

shell script to sort information in one file

Hi to all, anyway to create shell script to sort informations from one file and create new file with the sorted values? from file 30days.out -bash-3.00# more 30days.out user/str4@kl.com/INBOX user/tg1@johor.com/INBOX user/tg2@kedah.com/INBOX user/tg3@titangroup.com/INBOX... (3 Replies)
Discussion started by: Mr_47
3 Replies

4. Shell Programming and Scripting

Extract UNIque records from File

Hi, I have a file with 20GB Pipe Delimited file where i have too many duplicate records. I need an awk script to extract the unique records from the file and put it into another file. Kindly help. Thanks, Arun (1 Reply)
Discussion started by: Arun Mishra
1 Replies

5. Shell Programming and Scripting

Shell Script to Dynamically Extract file content based on Parameters from a pdf file

Hi Guru's, I am new to shell scripting. I have a unique requirement: The system generates a single pdf(/tmp/ABC.pdf) file with Invoices for Multiple Customers, the format is something like this: Page1 >> Customer 1 >>Invoice1 + invoice 2 >> Page1 end Page2 >> Customer 2 >>Invoice 3 + Invoice 4... (3 Replies)
Discussion started by: DIps
3 Replies

6. Shell Programming and Scripting

Help with shell script to extract certain information

Hi, I have a file which I need to programmatically split into two files. All the information in the file before pattern "STOP HERE" is to be stripped and output into one file while everything after "STOP HERE" is to be output into a separate file. I would appreciate help on how to do... (8 Replies)
Discussion started by: PTL
8 Replies

7. Shell Programming and Scripting

Generate 10000 unique audio file of 2MB each using shell script.

Hi, I want 10000+ unique Audio file of approx 2MB each. How can i generate numerous audio files using shell script. Any tool, command or suggestions are welcome. If i give one audio seed file then can we create numerous unique files with same seed file? Any help is highly appreciable.... (11 Replies)
Discussion started by: sushil.kumar
11 Replies

8. Shell Programming and Scripting

How to create file and file content based existing information?

Hi Gurus, I am SQL developer and new unix user. I need to create some file and file content based on information in two files. I have one file contains basic information below file1 and another exception file file2. the rule is if "zone' and "cd" in file1 exists in file2, then file name is... (13 Replies)
Discussion started by: Torhong
13 Replies

9. UNIX for Beginners Questions & Answers

TCL script to extract the file name and then create two independent list

I am having one problem as stated below Problem Description I am having some "sv" extension files , I am using "glob" to extract the matching files , Now in these matching files , I need to split them and extract the elements and create different lists. For example set files This... (1 Reply)
Discussion started by: kshitij
1 Replies

10. UNIX for Beginners Questions & Answers

awk script to extract transcript information from gff3 file

I need help to extract transcript information from gff3 file. Here is the input Chr01 JGI gene 82773 86941 . - . ID=Potri.001G000900;Name=Potri.001G000900 Chr01 JGI mRNA 82793 86530 . - . ID=PAC:27047814;Name=Potri.001G000900.1;pacid=27047814;longest=1;Parent=Potri.001G000900... (6 Replies)
Discussion started by: Maduranga
6 Replies
ftpmail(1)						      General Commands Manual							ftpmail(1)

NAME
ftpmail - FIFO-based Perl script for sending email based on proftpd TransferLog SYNOPSIS
ftpmail [ --help ] [ --fifo fifo-path ] [ --from email-address ] [ --log xferlog-file ] [ --recipient email-address ] [ --subject email- subject ] [ --smtp-server server-address ] [ --attach-file ] [ --auth smtp-auth-info-file ] [ --ignore-users regex-pattern ] [ --watch- users regex-pattern ] DESCRIPTION
ftpmail is a Perl script designed to read ProFTPD's TransferLog log entries, watching for uploads, and to send an automatic email notifica- tion when uploads occur. To use ftpmail , you configure your proftpd daemon to write its TransferLog to a FIFO; the ftpmail program is a FIFO reading program which then processes those log messages. OPTIONS
--help Display a short usage description, including all available options. --attach-file If used, this will cause a copy of the uploaded file to be included, as an attachment, in the generated email. --auth smtp-auth-info-file Configures the path to a file containing SMTP authentication information. The configured file should look like this: user username password password --fifo fifo-path Indicates the path to the FIFO to which proftpd is writing its TransferLog. That is, this is the path that you used for the TransferLog directive in your proftpd.conf. This parameter is REQUIRED. --from email-address Specifies the email address to use in the From email header. This parameter is REQUIRED. --ignore-users regex-pattern Specifies a Perl regular expression. If the uploading user name matches this regular expression, then an email notification is NOT sent; otherwise, an email is sent. --log xferlog-path Since this script reads the TransferLog using FIFOs, the actual TransferLog file is not written by default. Use this option to write the normal TransferLog file, in addition to watching for uploads. --recipient email-address Specifies an email address to which to send an email notification of the upload. This option can be used multiple times to specify multiple recipients. AT LEAST ONE recipient is REQUIRED. --smtp-server server-address Specifies the SMTP server to which to send the email. This parameter is REQUIRED. --subject subject Specify a custom Subject email header for the email sent. The default Subject is: User '$user' uploaded file '$file' via FTP --watch-users regex-pattern Specifies a Perl regular expression. If the uploading user name matches this regular expression, then an email notification is sent; otherwise, no email is sent. FILES
/var/log/xferlog AUTHORS
ProFTPD is written and maintained by a number of people, full credits can be found on http://www.proftpd.org/credits.html SEE ALSO
proftpd(8), xferlog(5) Full documentation on ProFTPD, including configuration and FAQs, is available at http://www.proftpd.org/ For help/support, try the ProFTPD mailing lists, detailed on http://www.proftpd.org/lists.html Report bugs at http://bugs.proftpd.org/ March 2011 ftpmail(1)
All times are GMT -4. The time now is 12:50 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy