awk: Print fields between two delimiters on separate lines and send to variables Post: 302685003

Sponsored Content

Top Forums Shell Programming and Scripting awk: Print fields between two delimiters on separate lines and send to variables Post 302685003 by tay9000 on Friday 10th of August 2012 01:16:45 PM

08-10-2012

Registered User

Below is my entire script. Yes, I am aware it is highly inefficient but I am a newbie and so far have just been writing very dirty scripts that get the job done. This script is going to go through about 600 files per day. Right now it is just echoing out all of the lines that I need. My idea with the variables was for the script to create a file inside of the $WorkingDir for each email address it finds and then write the lines I am echoing into each appropriate files based on the emails addresses the message is for. I haven't figured out the code to do this yet. I need to figure out how to extract those email addresses first! I did not try after your last post but now since I know how to pipe the email content to awk -f env.awk I think I could get further than before. Thank you. At first I was using the $To variable to get the addresses but then I noticed it was only getting the first line's worth of addresses. And then after banging my head on the keyboard enough, I came here.

Code:

#!/usr/local/bin/bash
# Variables
SpamDir='/home/tay/spam'
CurrentLine='0'
MaxLines=`ls $SpamDir/*.gz | wc -l`
WorkingDir='/tmp/spam-summary'

cd $SpamDir

# Variable Control
function VariableControl() {
CurrentLine=$(expr $CurrentLine + 1)
Mail=`ls *.gz | head -$CurrentLine | tail -1`
#MailConent=`zgrep $Mail`
From=`zgrep $Mail -e 'X-Envelope-From:' | awk '{print $2}' | sed 's/<//g;s/>//g'`
To=`zgrep $Mail -e 'X-Envelope-To:' | awk 'FS=":" {print $2}' | sed 's/<//g;s/>//g;s/,//g'`
Subject=`zgrep $Mail -e 'Subject:' | awk '{$1=""; print $0}'`
Score=`zgrep $Mail -e 'X-Spam-Score:' | awk '{print $2}'`
TimeEpoch=`ls -lh -D %s $Mail | awk '{print $6}'`; TimeHuman=`date -r $TimeEpoch +"%Y-%m-%d %l:%M %p"`
ID=`ls -lh $Mail | awk '{print $9}'`
RunControl
}

# AddQuery
function AddQuery() {
echo "$From	$Subject	$Score	$TimeHuman	$ID"
VariableControl
}

# Run Control
function RunControl() {
if [ $CurrentLine -gt $MaxLines ]
        then
exit
fi
AddQuery
}

VariableControl

Last edited by tay9000; 08-10-2012 at 02:23 PM..

tay9000

View Public Profile for tay9000

Find all posts by tay9000

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

trying to print selected fields of selected lines by AWK

I am trying to print 1st, 2nd, 13th and 14th fields of a file of line numbers from 29 to 10029. I dont know how to put this in one code. Currently I am removing the selected lines by awk 'NR==29,NR==10029' File1 > File2 and then doing awk '{print $1, $2, $13, $14}' File2 > File3 Can...

2. Shell Programming and Scripting

extract nth line of all files and print in output file on separate lines.

Hello UNIX experts, I have 124 text files in a directory. I want to extract the 45678th line of all the files sequentialy by file names. The extracted lines should be printed in the output file on seperate lines. e.g. The input Files are one.txt, two.txt, three.txt, four.txt The cat of four...

3. Shell Programming and Scripting

Compare Tab Separated Field with AWK to all and print lines of unique fields.

Hi. I have a tab separated file that has a couple nearly identical lines. When doing: sort file | uniq > file.new It passes through the nearly identical lines because, well, they still are unique. a) I want to look only at field x for uniqueness and if the content in field x is the...

4. Shell Programming and Scripting

awk print header as text from separate file with getline

I would like to print the output beginning with a header from a seperate file like this: awk 'BEGIN{FS="_";print ((getline < "header.txt")>0)} { if (! ($0 ~ /EL/ ) print }" input.txtWhat am i doing wrong?

5. Shell Programming and Scripting

Print only lines where fields concatenated match strings

Hello everyone, Maybe somebody could help me with an awk script. I have this input (field separator is comma ","): 547894982,M|N|J,U|Q|P,98,101,0,1,1 234900027,M|N|J,U|Q|P,98,101,0,1,1 234900023,M|N|J,U|Q|P,98,54,3,1,1 234900028,M|H|J,S|Q|P,98,101,0,1,1 234900030,M|N|J,U|F|P,98,101,0,1,1...

6. Shell Programming and Scripting

How to print 1st field and last 2 fields together and the rest of the fields after it using awk?

Hi experts, I need to print the first field first then last two fields should come next and then i need to print rest of the fields. Input : a1,abc,jsd,fhf,fkk,b1,b2 a2,acb,dfg,ghj,b3,c4 a3,djf,wdjg,fkg,dff,ggk,d4,d5 Expected output: a1,b1,b2,abc,jsd,fhf,fkk...

7. Shell Programming and Scripting

awk sort based on difference of fields and print all fields

Hi I have a file as below <field1> <field2> <field3> ... <field_num1> <field_num2> Trying to sort based on difference of <field_num1> and <field_num2> in desceding order and print all fields. I tried this and it doesn't sort on the difference field .. Appreciate your help. cat...

8. UNIX for Beginners Questions & Answers

How to count lines of CSV file where 2 fields match variables?

I'm trying to use awk to count the occurrences of two matching fields of a CSV file. For instance, for data that looks like this... Joe,Blue,Yes,No,High Mike,Blue,Yes,Yes,Low Joe,Red,No,No,Low Joe,Red,Yes,Yes,Low I've been trying to use code like this... countvar=`awk ' $2~/$color/...

9. Shell Programming and Scripting

awk to print line is values between two fields in separate file

I am trying to use awk to find all the $3 values in file2 that are between $2 and $3 in file1. If a value in $3 of file2 is between the file1 fields then it is printed along with the $6 value in file1. Both file1 and file2 are tab-delimited as well as the desired output. If there is nothing to...

10. Shell Programming and Scripting

awk to print lines based on text in field and value in two additional fields

In the awk below I am trying to print the entire line, along with the header row, if $2 is SNV or MNV or INDEL. If that condition is met or is true, and $3 is less than or equal to 0.05, then in $7 the sub pattern :GMAF= is found and the value after the = sign is checked. If that value is less than...

LEARN ABOUT DEBIAN

mail::verify

Mail::Verify(3pm)					User Contributed Perl Documentation					 Mail::Verify(3pm)

NAME

       Mail::Verify - Utility to verify an email address

SYNOPSIS

	   use Mail::Verify;

DESCRIPTION

       "Mail::Verify" provides a function CheckAddress function for verifying email addresses.	First the syntax of the email address is checked,
       then it verifies that there is at least one valid MX server accepting email for the domain.  Using Net::DNS and IO::Socket a list of MX
       records (or, falling back on a hosts A record) are checked to make sure at least one SMTP server is accepting connections.

ERRORS

       Here are a list of return codes and what they mean:

       0      The email address appears to be valid.

       1      No email address was supplied.

       2      There is a syntaxical error in the email address.

       3      There are no DNS entries for the host in question (no MX records or A records).

       4      There are no live SMTP servers accepting connections for this email address.

EXAMPLES

       This example shows obtaining an email address from a form field and verifying it.

	 use CGI qw/:standard/;
	 use Mail::Verify;
	 my $q = new CGI;
	 [...]
	 my $email = $q->param("emailaddr");
	 my $email_ck = Mail::Verify::CheckAddress( $email );
	 if( $email_ck ) {
	     print '<h1>Form input error: Invalid email address.</h1>';
	 }
	 [...]

perl v5.8.8							    2002-06-09							 Mail::Verify(3pm)