awk: Print fields between two delimiters on separate lines and send to variables Post: 302685003

Sponsored Content

Top Forums Shell Programming and Scripting awk: Print fields between two delimiters on separate lines and send to variables Post 302685003 by tay9000 on Friday 10th of August 2012 01:16:45 PM

08-10-2012

Registered User

Below is my entire script. Yes, I am aware it is highly inefficient but I am a newbie and so far have just been writing very dirty scripts that get the job done. This script is going to go through about 600 files per day. Right now it is just echoing out all of the lines that I need. My idea with the variables was for the script to create a file inside of the $WorkingDir for each email address it finds and then write the lines I am echoing into each appropriate files based on the emails addresses the message is for. I haven't figured out the code to do this yet. I need to figure out how to extract those email addresses first! I did not try after your last post but now since I know how to pipe the email content to awk -f env.awk I think I could get further than before. Thank you. At first I was using the $To variable to get the addresses but then I noticed it was only getting the first line's worth of addresses. And then after banging my head on the keyboard enough, I came here.

Code:

#!/usr/local/bin/bash
# Variables
SpamDir='/home/tay/spam'
CurrentLine='0'
MaxLines=`ls $SpamDir/*.gz | wc -l`
WorkingDir='/tmp/spam-summary'

cd $SpamDir

# Variable Control
function VariableControl() {
CurrentLine=$(expr $CurrentLine + 1)
Mail=`ls *.gz | head -$CurrentLine | tail -1`
#MailConent=`zgrep $Mail`
From=`zgrep $Mail -e 'X-Envelope-From:' | awk '{print $2}' | sed 's/<//g;s/>//g'`
To=`zgrep $Mail -e 'X-Envelope-To:' | awk 'FS=":" {print $2}' | sed 's/<//g;s/>//g;s/,//g'`
Subject=`zgrep $Mail -e 'Subject:' | awk '{$1=""; print $0}'`
Score=`zgrep $Mail -e 'X-Spam-Score:' | awk '{print $2}'`
TimeEpoch=`ls -lh -D %s $Mail | awk '{print $6}'`; TimeHuman=`date -r $TimeEpoch +"%Y-%m-%d %l:%M %p"`
ID=`ls -lh $Mail | awk '{print $9}'`
RunControl
}

# AddQuery
function AddQuery() {
echo "$From	$Subject	$Score	$TimeHuman	$ID"
VariableControl
}

# Run Control
function RunControl() {
if [ $CurrentLine -gt $MaxLines ]
        then
exit
fi
AddQuery
}

VariableControl

Last edited by tay9000; 08-10-2012 at 02:23 PM..

tay9000

View Public Profile for tay9000

Find all posts by tay9000

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

trying to print selected fields of selected lines by AWK

I am trying to print 1st, 2nd, 13th and 14th fields of a file of line numbers from 29 to 10029. I dont know how to put this in one code. Currently I am removing the selected lines by awk 'NR==29,NR==10029' File1 > File2 and then doing awk '{print $1, $2, $13, $14}' File2 > File3 Can...

2. Shell Programming and Scripting

extract nth line of all files and print in output file on separate lines.

Hello UNIX experts, I have 124 text files in a directory. I want to extract the 45678th line of all the files sequentialy by file names. The extracted lines should be printed in the output file on seperate lines. e.g. The input Files are one.txt, two.txt, three.txt, four.txt The cat of four...

3. Shell Programming and Scripting

Compare Tab Separated Field with AWK to all and print lines of unique fields.

Hi. I have a tab separated file that has a couple nearly identical lines. When doing: sort file | uniq > file.new It passes through the nearly identical lines because, well, they still are unique. a) I want to look only at field x for uniqueness and if the content in field x is the...

4. Shell Programming and Scripting

awk print header as text from separate file with getline

I would like to print the output beginning with a header from a seperate file like this: awk 'BEGIN{FS="_";print ((getline < "header.txt")>0)} { if (! ($0 ~ /EL/ ) print }" input.txtWhat am i doing wrong?

5. Shell Programming and Scripting

Print only lines where fields concatenated match strings

Hello everyone, Maybe somebody could help me with an awk script. I have this input (field separator is comma ","): 547894982,M|N|J,U|Q|P,98,101,0,1,1 234900027,M|N|J,U|Q|P,98,101,0,1,1 234900023,M|N|J,U|Q|P,98,54,3,1,1 234900028,M|H|J,S|Q|P,98,101,0,1,1 234900030,M|N|J,U|F|P,98,101,0,1,1...

6. Shell Programming and Scripting

How to print 1st field and last 2 fields together and the rest of the fields after it using awk?

Hi experts, I need to print the first field first then last two fields should come next and then i need to print rest of the fields. Input : a1,abc,jsd,fhf,fkk,b1,b2 a2,acb,dfg,ghj,b3,c4 a3,djf,wdjg,fkg,dff,ggk,d4,d5 Expected output: a1,b1,b2,abc,jsd,fhf,fkk...

7. Shell Programming and Scripting

awk sort based on difference of fields and print all fields

Hi I have a file as below <field1> <field2> <field3> ... <field_num1> <field_num2> Trying to sort based on difference of <field_num1> and <field_num2> in desceding order and print all fields. I tried this and it doesn't sort on the difference field .. Appreciate your help. cat...

8. UNIX for Beginners Questions & Answers

How to count lines of CSV file where 2 fields match variables?

I'm trying to use awk to count the occurrences of two matching fields of a CSV file. For instance, for data that looks like this... Joe,Blue,Yes,No,High Mike,Blue,Yes,Yes,Low Joe,Red,No,No,Low Joe,Red,Yes,Yes,Low I've been trying to use code like this... countvar=`awk ' $2~/$color/...

9. Shell Programming and Scripting

awk to print line is values between two fields in separate file

I am trying to use awk to find all the $3 values in file2 that are between $2 and $3 in file1. If a value in $3 of file2 is between the file1 fields then it is printed along with the $6 value in file1. Both file1 and file2 are tab-delimited as well as the desired output. If there is nothing to...

10. Shell Programming and Scripting

awk to print lines based on text in field and value in two additional fields

In the awk below I am trying to print the entire line, along with the header row, if $2 is SNV or MNV or INDEL. If that condition is met or is true, and $3 is less than or equal to 0.05, then in $7 the sub pattern :GMAF= is found and the value after the = sign is checked. If that value is less than...

LEARN ABOUT DEBIAN

log::agent::driver::mail

Agent::Driver::Mail(3pm)				User Contributed Perl Documentation				  Agent::Driver::Mail(3pm)

NAME

       Log::Agent::Driver::Mail - email driver for Log::Agent

SYNOPSIS

	use Log::Agent;
	require Log::Agent::Driver::Mail;

	my $driver = Log::Agent::Driver::Mail->make(
	    -to      => 'oncall@example.org',
	    -cc      => [ qw( noc@example.org admin@example,net ) ],
	    -subject => "ALERT! ALERT!",
	    -mailer  => [ 'smtp', Server => 'mail.example.net' ]
	);
	logconfig(-driver => $driver);

DESCRIPTION

       This driver maps the logxxx() calls to email messages.  Each call generates a separate email message.  The Mail::Mailer module is required.

CONSTRUCTOR

   make OPTIONS
       The OPTIONS argument is a hash with the following keys:

       -prefix An optional prefix for the message body.

       -to     The destination addresses, may be a scalar containing a valid email address or a reference to an array of addresses.

       -reply_to
	       The reply-to addresses, may be a scalar containing a valid email address or a reference to an array of addresses.

       -from   The source address, must be a scalar containing a valid email address.

       -subject
	       The subject line of the email message.

       -cc     The carbon copy addresses, may be a scalar containing a valid email address or a reference to an array of addresses.

       -bcc    The blind carbon copy addresses, may be a scalar containing a valid email address or a reference to an array of addresses.

       -priority
	       The priority level for the email message.  This is NOT related to the logging priority.

       -mailer A reference to an array containing the optional arguments to Mail::Mailer->new().  Generally, this can be omitted.

NOTES

       Thanks to Shirley Wang for the idea for this module.

AUTHOR

       Mark Rogaski <mrogaski@pobox.com>

LICENSE

       Copyright (C) 2002 Mark Rogaski; all rights reserved.

       See Log::Agent(3) or the README file included with the distribution for license information.

SEE ALSO

       Mail::Mailer, Log::Agent::Driver(3), Log::Agent(3).

perl v5.10.0							    2002-05-12						  Agent::Driver::Mail(3pm)