awk: Print fields between two delimiters on separate lines and send to variables Post: 302685841

Sponsored Content

Top Forums Shell Programming and Scripting awk: Print fields between two delimiters on separate lines and send to variables Post 302685841 by Corona688 on Monday 13th of August 2012 12:03:42 PM

08-13-2012

Registered User

Slightly updated script which gets rid of some extra spaces:

Code:

#!/bin/sh

SpamDir='./'
WorkingDir='/tmp/spam-summary'

IFS=":"

# Loop on the files directly, instead of doing loops on line numbers
for FILE in ${SpamDir}/*.gz
do
        # Clear out variables
        From=
        To=
        Subject=
        Score=

        # ? What is column 9 on your ls -l ?
        ID=`ls -lh $Mail | awk '{print $9}'`

        # Your time functions look okay
#        TimeEpoch=`ls -lh -D %s "$FILE" | awk '{print $6}'`
#        TimeHuman=`date -r $TimeEpoch +"%Y-%m-%d %l:%M %p"`

        # Decompress file once instead of 9 times
        zcat "$FILE" > /tmp/$$

        # Read and process lines from the decompressed file one by one
        while read LINE
        do
                IFS=":" # Split on : so $1=X-Envelope-From, $2=<spammer@vnyu.com>
                set -- $LINE
                # If line has a : in it, save the header, then get rid of $1
                if [ "$#" -gt 1 ]
                then
                        HEADER="$1"
                        shift
                fi

                # Split on spaces, commas, and <>
                IFS="<>,        " # Not nine spaces -- one tab, eight spaces
                # Split <spammer@vnyu.com>, <whatever@...> into $1=spammer@vnyu.com, $2=whatever@..., etc
                set -- $1
                IFS=" "
                set -- $* # Get rid of extra spaces in input

                case "$HEADER" in
                X-Envelope-From) From="$From $*" ;;
                X-Envelope-To)     To="$To $*" ;;
                Subject)              Subject="$*" ;;
                X-Spam-Score)     Score="$*" ;;
                esac
        done < /tmp/$$

        echo
        echo "Processed $FILE"
        echo "To:$To"
        echo "From:$From"
        echo "Subject:$Subject"
        echo "Score:$Score"
done

rm -f /tmp/$$

Corona688

View Public Profile for Corona688

Visit Corona688's homepage!

Find all posts by Corona688

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

trying to print selected fields of selected lines by AWK

I am trying to print 1st, 2nd, 13th and 14th fields of a file of line numbers from 29 to 10029. I dont know how to put this in one code. Currently I am removing the selected lines by awk 'NR==29,NR==10029' File1 > File2 and then doing awk '{print $1, $2, $13, $14}' File2 > File3 Can...

2. Shell Programming and Scripting

extract nth line of all files and print in output file on separate lines.

Hello UNIX experts, I have 124 text files in a directory. I want to extract the 45678th line of all the files sequentialy by file names. The extracted lines should be printed in the output file on seperate lines. e.g. The input Files are one.txt, two.txt, three.txt, four.txt The cat of four...

3. Shell Programming and Scripting

Compare Tab Separated Field with AWK to all and print lines of unique fields.

Hi. I have a tab separated file that has a couple nearly identical lines. When doing: sort file | uniq > file.new It passes through the nearly identical lines because, well, they still are unique. a) I want to look only at field x for uniqueness and if the content in field x is the...

4. Shell Programming and Scripting

awk print header as text from separate file with getline

I would like to print the output beginning with a header from a seperate file like this: awk 'BEGIN{FS="_";print ((getline < "header.txt")>0)} { if (! ($0 ~ /EL/ ) print }" input.txtWhat am i doing wrong?

5. Shell Programming and Scripting

Print only lines where fields concatenated match strings

Hello everyone, Maybe somebody could help me with an awk script. I have this input (field separator is comma ","): 547894982,M|N|J,U|Q|P,98,101,0,1,1 234900027,M|N|J,U|Q|P,98,101,0,1,1 234900023,M|N|J,U|Q|P,98,54,3,1,1 234900028,M|H|J,S|Q|P,98,101,0,1,1 234900030,M|N|J,U|F|P,98,101,0,1,1...

6. Shell Programming and Scripting

How to print 1st field and last 2 fields together and the rest of the fields after it using awk?

Hi experts, I need to print the first field first then last two fields should come next and then i need to print rest of the fields. Input : a1,abc,jsd,fhf,fkk,b1,b2 a2,acb,dfg,ghj,b3,c4 a3,djf,wdjg,fkg,dff,ggk,d4,d5 Expected output: a1,b1,b2,abc,jsd,fhf,fkk...

7. Shell Programming and Scripting

awk sort based on difference of fields and print all fields

Hi I have a file as below <field1> <field2> <field3> ... <field_num1> <field_num2> Trying to sort based on difference of <field_num1> and <field_num2> in desceding order and print all fields. I tried this and it doesn't sort on the difference field .. Appreciate your help. cat...

8. UNIX for Beginners Questions & Answers

How to count lines of CSV file where 2 fields match variables?

I'm trying to use awk to count the occurrences of two matching fields of a CSV file. For instance, for data that looks like this... Joe,Blue,Yes,No,High Mike,Blue,Yes,Yes,Low Joe,Red,No,No,Low Joe,Red,Yes,Yes,Low I've been trying to use code like this... countvar=`awk ' $2~/$color/...

9. Shell Programming and Scripting

awk to print line is values between two fields in separate file

I am trying to use awk to find all the $3 values in file2 that are between $2 and $3 in file1. If a value in $3 of file2 is between the file1 fields then it is printed along with the $6 value in file1. Both file1 and file2 are tab-delimited as well as the desired output. If there is nothing to...

10. Shell Programming and Scripting

awk to print lines based on text in field and value in two additional fields

In the awk below I am trying to print the entire line, along with the header row, if $2 is SNV or MNV or INDEL. If that condition is met or is true, and $3 is less than or equal to 0.05, then in $7 the sub pattern :GMAF= is found and the value after the = sign is checked. If that value is less than...

LEARN ABOUT SUNOS

echo

echo(1B)					     SunOS/BSD Compatibility Package Commands						  echo(1B)

NAME

       echo - echo arguments to standard output

SYNOPSIS

       /usr/ucb/echo [-n] [argument]

DESCRIPTION

       echo writes its arguments, separated by BLANKs and terminated by a NEWLINE, to the standard output.

       echo  is useful for producing diagnostics in command files and for sending known data into a pipe, and for displaying the contents of envi-
       ronment variables.

       For example, you can use echo to determine how many subdirectories below the root directory (/) is your current directory, as follows:

	 o  echo your current-working-directory's full pathname

	 o  pipe the output through tr to translate the path's embedded slash-characters into space-characters

	 o  pipe that output through wc -w for a count of the names in your path.

	      example% /usr/bin/echo "echo $PWD | tr '/' ' ' | wc -w"

       See tr(1) and wc(1) for their functionality.

       The shells csh(1), ksh(1), and sh(1), each have an echo built-in command, which, by default, will have precedence, and will be  invoked	if
       the  user  calls  echo without a full pathname. /usr/ucb/echo and csh's echo() have an -n option, but do not understand back-slashed escape
       characters. sh's echo(), ksh's echo(), and /usr/bin/echo, on the other hand, understand the  black-slashed  escape  characters,	and  ksh's
       echo() also understands a as the audible bell character; however, these commands do not have an -n option.

OPTIONS

       -n	Do not add the NEWLINE to the output.

ATTRIBUTES

       See attributes(5) for descriptions of the following attributes:

       +-----------------------------+-----------------------------+
       |      ATTRIBUTE TYPE	     |	    ATTRIBUTE VALUE	   |
       +-----------------------------+-----------------------------+
       |Availability		     |SUNWscpu			   |
       +-----------------------------+-----------------------------+

SEE ALSO

       csh(1), echo(1), ksh(1), sh(1), tr(1),  wc(1), attributes(5)

NOTES

       The -n option is a transition aid for BSD applications, and may not be supported in future  releases.

SunOS 5.10							    3 Aug 1994								  echo(1B)