Pulling Data, Then Moving to the Next File


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Pulling Data, Then Moving to the Next File
# 1  
Old 04-24-2014
Pulling Data, Then Moving to the Next File

I'm scanning a list of emails- I need to pull 2 pieces of data, then move to the next file:
Sender's Email Address
Email Date

I need these to be outputted into a single column- separated by a ",". Like this:

Email1's Address, Email1's Date Stamp
Email2's Address, Email2's Date Stamp
....so on

Instead it stacks the results like this:
Email1's Address
Email1's Date Stamp
Email2's Address
Email2's Date Stamp
....so on

How can I achieve the results I want? Thanks!!

Code:
find $SRC -type f -name *.emlx |
while read FILE
    do
        awk '/^From:/ && gsub(/.*<|>.*/,x)' $FILE
        sed -n '/Date/p' $FILE

    done > ~/Desktop/output.txt

# 2  
Old 04-24-2014
Code:
find $SRC -type f -name *.emlx |
while read FILE
    do
        awk '/^From:/ && gsub(/.*<|>.*/,x)' $FILE
        sed -n '/Date/p' $FILE

    done | awk 'ORS=NR%2?", ":RS' > ~/Desktop/output.txt

This User Gave Thanks to pilnet101 For This Post:
# 3  
Old 04-25-2014
This works OK, but the problem is if I run into an error with any one of those extraction programs, the formatting for the rest of the document gets shifted. I tried to remedy this with an if statement entering a placeholder for bad/no matches but it's not bullet proof (shown below).

Any ideas for a stronger formatting structure?

Code:
sed -n '/Date/p' $FILE | egrep -o -m1 "Date:.{17}"
if [ $? -eq 1 ]; then
echo "NODATE"
fi

# 4  
Old 04-28-2014
Can you provide some example data of the bad output when the data is shifted?
# 5  
Old 04-28-2014
Sure, currently my code reads:
Code:
find $SRC -type f -name *.emlx |
while read FILE
do
     sed -n '/Serial/p' $FILE | egrep -o -m1 "Serial#.{14}"
          if [ $? -eq 1 ]; then
          echo "NO SERIAL"
     sed -n '/Date/p' $FILE | egrep -o -m1 "Date:.{17}"
          if [ $? -eq 1 ]; then
          echo "NODATE"
done | awk 'ORS=NR%2?", ":RS' > ~/Desktop/output.txt
fi

The output gets tripped up because it has Serial# twice (even though I thought the -m1 trigger only looks at the first instance) and pushes the data over 1 comma.

The line that USUALLY comes in on the email is:
"Serial#: C02J13XXXXXX"
But the user decided to add data in when they noticed the Serial number was missing, tripping up my filter:
"Serial#: System Serial# (N0 Serial#) 2010 - 13" MB Air 1.86GHZ Intel Core 2 Duo"

As you can see it shifts my data over:

Serial#: C02J13XXXXXX, Date: Thu, 18 Jul 2013
Serial#: C02J1JXXXXXX, Date: Thu, 18 Jul 2013
Serial#: System Seria, Serial#) 2010 - 13" M
Date: Fri, 19 Jul 2013, Serial#: C02HNCZXXXXX
Date: Thu, 04 Jul 2013, Serial#: C02HNCXXXXX
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Pulling information from a data file by date

awk -v now="$(date +%s)" -v tDiff="${USERMINUTES}" ' BEGIN { FS="=" if (!now) now=systime() if (!tDiff) tDiff=60*60 p=1 } /{/ {rec=$0;p=1;next} /}/ && rec && p {print rec ORS $0;next} $1=="entry_time" { if (now-$2>tDiff)p=0 } {rec=rec ORS $0}'... (6 Replies)
Discussion started by: SkySmart
6 Replies

2. Shell Programming and Scripting

BASH- Need help pulling data from .emlx

Hello, fellow computer junkies. First time poster! My boss wrote an application (Mavericks 10.9, Mountain Lion 10.8) that checks a user's security settings. The user runs the application, then it spits out an email that is sent back to our inbox showing the results. On our end, we have a mail rule... (5 Replies)
Discussion started by: sudo
5 Replies

3. Shell Programming and Scripting

Pulling data from xml

Hi there, Please could anyone help with this. I have an xml file that contains repeating values eg <Rule name> AAAAA <Action> BBBBB </Action> <Data> CCCCC </Data> <Type> DDDDD </Type> </Rule name> <Rule name> A1A1A1A1 <Action> B1B1B1B1 </Action> <Data> C1C1C1C </Data> <Type>... (4 Replies)
Discussion started by: ssideel
4 Replies

4. UNIX for Dummies Questions & Answers

Data file moving

Suppose there is a file “Text1.txt” which contains 100 lines. I need to move 1st 25 line into another file “Text2.txt” How we can do it? Suppose there is a file “Text1.txt” in which city: Bangalore is repeating N times. I need to replace Bangalore with Delhi. How we can do... (1 Reply)
Discussion started by: Rajesh1412
1 Replies

5. Shell Programming and Scripting

Moving a column across a delimited data file

Hi, I am trying to move a column from one position to another position in a delimited file. The positions are dynamic in nature and are available by environmental variables. Also the file can have n number of columns. Example: Initial Column Position=1 Final Column Position=3 Delimiter='|' ... (2 Replies)
Discussion started by: ayan153
2 Replies

6. Shell Programming and Scripting

Pulling data by GPS coordinates from text file

Hi there, I'm having a problem trying to extract data from within a text file. I'm trying to extract this manually for a lack of better words. I need any items that fall within latitude 36.5 to 39.5 and long -75.3 to -83.9 I have been doing this using cat neta.txt | grep '!38' and working... (6 Replies)
Discussion started by: Mikey
6 Replies

7. Shell Programming and Scripting

Help with pulling / filtering data from a .csv

Good day Gurus, I have a csv file that contains an inventory of active servers. This csv file contains a well over a hundred systems (IBM, SUN, HP). It also contains those systems details. See below for an example hostA,invver,1.02,20100430 hostA,date,08/30/2010,06:18 hostA,use,"Unknown... (4 Replies)
Discussion started by: LuffyDMonkey
4 Replies

8. Shell Programming and Scripting

SFTP to server, pulling data and removing the data

Hi all, I have the following script, but are not too sure about the syntax to complete the script. In essence, the script must connect to a SFTP server at a client site with username and password located in a file on my server. Then change to the appropriate directory. Pull the data to the... (1 Reply)
Discussion started by: codenjanod
1 Replies

9. Shell Programming and Scripting

Pulling data and following lines from file

I saw a few posts close to what i want to do, but they didn't look like they would work exactly.. or I need to think out of the box on this. I have a file that I keep server stats in for my own performance analysis. this file has the output from many commands in it (uptime, vmstats, ps, swap... (2 Replies)
Discussion started by: MizzGail
2 Replies
Login or Register to Ask a Question