Visit The New, Modern Unix Linux Community


Pulling Data, Then Moving to the Next File


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Pulling Data, Then Moving to the Next File
# 1  
Pulling Data, Then Moving to the Next File

I'm scanning a list of emails- I need to pull 2 pieces of data, then move to the next file:
Sender's Email Address
Email Date

I need these to be outputted into a single column- separated by a ",". Like this:

Email1's Address, Email1's Date Stamp
Email2's Address, Email2's Date Stamp
....so on

Instead it stacks the results like this:
Email1's Address
Email1's Date Stamp
Email2's Address
Email2's Date Stamp
....so on

How can I achieve the results I want? Thanks!!

Code:
find $SRC -type f -name *.emlx |
while read FILE
    do
        awk '/^From:/ && gsub(/.*<|>.*/,x)' $FILE
        sed -n '/Date/p' $FILE

    done > ~/Desktop/output.txt

# 2  
Code:
find $SRC -type f -name *.emlx |
while read FILE
    do
        awk '/^From:/ && gsub(/.*<|>.*/,x)' $FILE
        sed -n '/Date/p' $FILE

    done | awk 'ORS=NR%2?", ":RS' > ~/Desktop/output.txt

This User Gave Thanks to pilnet101 For This Post:
# 3  
This works OK, but the problem is if I run into an error with any one of those extraction programs, the formatting for the rest of the document gets shifted. I tried to remedy this with an if statement entering a placeholder for bad/no matches but it's not bullet proof (shown below).

Any ideas for a stronger formatting structure?

Code:
sed -n '/Date/p' $FILE | egrep -o -m1 "Date:.{17}"
if [ $? -eq 1 ]; then
echo "NODATE"
fi

# 4  
Can you provide some example data of the bad output when the data is shifted?
# 5  
Sure, currently my code reads:
Code:
find $SRC -type f -name *.emlx |
while read FILE
do
     sed -n '/Serial/p' $FILE | egrep -o -m1 "Serial#.{14}"
          if [ $? -eq 1 ]; then
          echo "NO SERIAL"
     sed -n '/Date/p' $FILE | egrep -o -m1 "Date:.{17}"
          if [ $? -eq 1 ]; then
          echo "NODATE"
done | awk 'ORS=NR%2?", ":RS' > ~/Desktop/output.txt
fi

The output gets tripped up because it has Serial# twice (even though I thought the -m1 trigger only looks at the first instance) and pushes the data over 1 comma.

The line that USUALLY comes in on the email is:
"Serial#: C02J13XXXXXX"
But the user decided to add data in when they noticed the Serial number was missing, tripping up my filter:
"Serial#: System Serial# (N0 Serial#) 2010 - 13" MB Air 1.86GHZ Intel Core 2 Duo"

As you can see it shifts my data over:

Serial#: C02J13XXXXXX, Date: Thu, 18 Jul 2013
Serial#: C02J1JXXXXXX, Date: Thu, 18 Jul 2013
Serial#: System Seria, Serial#) 2010 - 13" M
Date: Fri, 19 Jul 2013, Serial#: C02HNCZXXXXX
Date: Thu, 04 Jul 2013, Serial#: C02HNCXXXXX

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #688
Difficulty: Medium
The first computer to implement architectures enabling reverse Polish notation (RPN) was the Apple I computer.
True or False?

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Pulling information from a data file by date

awk -v now="$(date +%s)" -v tDiff="${USERMINUTES}" ' BEGIN { FS="=" if (!now) now=systime() if (!tDiff) tDiff=60*60 p=1 } /{/ {rec=$0;p=1;next} /}/ && rec && p {print rec ORS $0;next} $1=="entry_time" { if (now-$2>tDiff)p=0 } {rec=rec ORS $0}'... (6 Replies)
Discussion started by: SkySmart
6 Replies

2. Shell Programming and Scripting

BASH- Need help pulling data from .emlx

Hello, fellow computer junkies. First time poster! My boss wrote an application (Mavericks 10.9, Mountain Lion 10.8) that checks a user's security settings. The user runs the application, then it spits out an email that is sent back to our inbox showing the results. On our end, we have a mail rule... (5 Replies)
Discussion started by: sudo
5 Replies

3. Shell Programming and Scripting

Pulling data from xml

Hi there, Please could anyone help with this. I have an xml file that contains repeating values eg <Rule name> AAAAA <Action> BBBBB </Action> <Data> CCCCC </Data> <Type> DDDDD </Type> </Rule name> <Rule name> A1A1A1A1 <Action> B1B1B1B1 </Action> <Data> C1C1C1C </Data> <Type>... (4 Replies)
Discussion started by: ssideel
4 Replies

4. UNIX for Dummies Questions & Answers

Data file moving

Suppose there is a file “Text1.txt” which contains 100 lines. I need to move 1st 25 line into another file “Text2.txt” How we can do it? Suppose there is a file “Text1.txt” in which city: Bangalore is repeating N times. I need to replace Bangalore with Delhi. How we can do... (1 Reply)
Discussion started by: Rajesh1412
1 Replies

5. Shell Programming and Scripting

Moving a column across a delimited data file

Hi, I am trying to move a column from one position to another position in a delimited file. The positions are dynamic in nature and are available by environmental variables. Also the file can have n number of columns. Example: Initial Column Position=1 Final Column Position=3 Delimiter='|' ... (2 Replies)
Discussion started by: ayan153
2 Replies

6. Shell Programming and Scripting

Pulling data by GPS coordinates from text file

Hi there, I'm having a problem trying to extract data from within a text file. I'm trying to extract this manually for a lack of better words. I need any items that fall within latitude 36.5 to 39.5 and long -75.3 to -83.9 I have been doing this using cat neta.txt | grep '!38' and working... (6 Replies)
Discussion started by: Mikey
6 Replies

7. Shell Programming and Scripting

Help with pulling / filtering data from a .csv

Good day Gurus, I have a csv file that contains an inventory of active servers. This csv file contains a well over a hundred systems (IBM, SUN, HP). It also contains those systems details. See below for an example hostA,invver,1.02,20100430 hostA,date,08/30/2010,06:18 hostA,use,"Unknown... (4 Replies)
Discussion started by: LuffyDMonkey
4 Replies

8. Shell Programming and Scripting

SFTP to server, pulling data and removing the data

Hi all, I have the following script, but are not too sure about the syntax to complete the script. In essence, the script must connect to a SFTP server at a client site with username and password located in a file on my server. Then change to the appropriate directory. Pull the data to the... (1 Reply)
Discussion started by: codenjanod
1 Replies

9. Shell Programming and Scripting

Pulling data and following lines from file

I saw a few posts close to what i want to do, but they didn't look like they would work exactly.. or I need to think out of the box on this. I have a file that I keep server stats in for my own performance analysis. this file has the output from many commands in it (uptime, vmstats, ps, swap... (2 Replies)
Discussion started by: MizzGail
2 Replies

Featured Tech Videos