11-10-2010
So I ran my script with the suggested perl statement inside the while loop, and even though it completed successfully, it took over 6 hours to run reading line by line in a 90MB log file. I've decided to extract each of the 10 columns out individually like what's below, write them to their own file, and then paste each file (line by line) together into one log file. Doing it that way only took 1 minute to run.
Anyway, this perl statement doesn't seem to work outside of the while loop when there is a null value.
perl -ne '/USERID:\s"(.+?)"/; print $1, "\n"' datafile.log > userid.log
Example from datafile.log:
<null>
<null>
kristin
<null>
<null>
kristin
.......
What shows up in the userid.log:
<null>
<null>
kristin
kristin
kristin
kristin
I don't know why the 4th and 5th value don't show up as null in userid.log. I've been researching all day and trying different things but can't seem to get past the null issue. I also tried that suggested awk statement and got these errors:
awk: syntax error near line 1
awk: bailing out near line 1
Any help is much appreciated!
---------- Post updated at 06:43 PM ---------- Previous update was at 06:16 PM ----------
Sorry forgot to add one thing - when I say null value, I mean that USERID: "blah" doesn't exist on lines 1, 2, 4, and 5 in the datafile.log file. I didn't mean USERID: "". So the datafile.log actually looks like this:
TERMINAL: "blah" USERHOST: "blah"
TERMINAL: "blah" USERHOST: "blah"
TERMINAL: "blah" USERID: "kristin" USERHOST: "blah"
TERMINAL: "blah" USERHOST: "blah"
TERMINAL: "blah" USERHOST: "blah"
TERMINAL: "blah" USERID: "kristin" USERHOST: "blah"
Hope this makes sense.
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
I have a text file with following content (3 lines)
filename : output.txt
first line:12/12/2008
second line:12/12/2008
third line:Y
I would like to know how we can replace 'Y' with 'N' in the 3rd line keeping 1st and 2nd lines same as what it was before.
I tried using cat output.txt... (4 Replies)
Discussion started by: santosham
4 Replies
2. Shell Programming and Scripting
Hi,
I wanted to add specific text to each row in a text file containing three rows. Example:
0 8 7 6 5 5
7 8 9 0 7 9
7 8 9 0 1 2
And I want to add a 21 at the beginning of the first row, and blank spaces at the beginning of the second two rows. To get this:
21 0 8 7 6 5 5
7 8... (4 Replies)
Discussion started by: hertingm
4 Replies
3. Shell Programming and Scripting
Hi all,
I need UNIX command that would give me all files without the string "4R" anywhere in the file. I have about a hundred files in my directory, and I need to list all files wihtout the string "4R" anywhere. Can anyone help me please? Thank you.l (4 Replies)
Discussion started by: risk_sly
4 Replies
4. Shell Programming and Scripting
I am attempting to insert multiple lines of text into a specific place in a text file based on the lines above or below it.
For example, Here is a portion of a zone file.
IN NS ns1.domain.tld.
IN NS ns2.domain.tld.
IN ... (2 Replies)
Discussion started by: cdn_humbucker
2 Replies
5. Shell Programming and Scripting
Hi,
I have the following text file:
8 T1mapping_flip02 ok 128 108 30 1 665000-000008-000001.dcm
9 T1mapping_flip05 ok 128 108 30 1 665000-000009-000001.dcm
10 T1mapping_flip10 ok 128 108 30 1 665000-000010-000001.dcm
11 T1mapping_flip15 ok 128 108 30... (2 Replies)
Discussion started by: goodbenito
2 Replies
6. Shell Programming and Scripting
Hi all,
I have 2 files, One file contain data like this
FHIT
CS
CHRM1
PDE3A
PDE3B
HSP90AA1
PTK2
HTR1A
ESR1
PARP1
PLA2G1B
These names are mentioned in the second file(Please see attached second file) as
# Drug_Target_X_Gene_Name:(Where X can be any number (1-1000) (1 Reply)
Discussion started by: manigrover
1 Replies
7. Shell Programming and Scripting
Hi all,
I have 2 files, One file contain data like this
FHIT
CS
CHRM1
PDE3A
PDE3B
HSP90AA1
PTK2
HTR1A
ESR1
PARP1
PLA2G1B
These names are mentioned in the second file(Please see attached second file) as (7 Replies)
Discussion started by: manigrover
7 Replies
8. UNIX for Dummies Questions & Answers
Hi,
I have log file which rolls out every second which is as this.
HttpGenRequest - -<!--OXi dbPublish--> <created="2014-03-24 23:45:37" lastMsgId="" requestTime="0.0333"> <response request="getOutcomeDetails" code="114" message="Request found no matching data" debug="" provider="undefined"/>... (3 Replies)
Discussion started by: karthikprakash
3 Replies
9. Shell Programming and Scripting
Hi
This is my first post and I'm just a beginner. So please be nice to me.
I have a couple of html files where a pattern beginning with "http://www.site.com" and ending with "/resource.dat" is present on every 241st line. How do I extract this to a new text file?
I have tried sed -n 241,241p... (13 Replies)
Discussion started by: dejavo
13 Replies
10. Shell Programming and Scripting
Hi guys,
I have a text file named file1.txt that is formatted like this:
001 , ID , 20000
002 , Name , Brandon
003 , Phone_Number , 616-234-1999
004 , SSNumber , 234-23-234
005 , Model , Toyota
007 , Engine ,V8
008 , GPS , OFF
and I have file2.txt formatted like this:
... (2 Replies)
Discussion started by: An0mander
2 Replies
LEARN ABOUT PHP
mailparse_uudecode_all
MAILPARSE_UUDECODE_ALL(3) 1 MAILPARSE_UUDECODE_ALL(3)
mailparse_uudecode_all - Scans the data from fp and extract each embedded uuencoded file
SYNOPSIS
array mailparse_uudecode_all (resource $fp)
DESCRIPTION
Scans the data from the given file pointer and extract each embedded uuencoded file into a temporary file.
PARAMETERS
o $fp
- A valid file pointer.
RETURN VALUES
Returns an array of associative arrays listing filename information.
+-------------+-------------------------------------------------+
| | |
| filename | |
| | |
| | Path to the temporary file name created |
| | |
| | |
|origfilename | |
| | |
| | The original filename, for uuencoded parts only |
| | |
+-------------+-------------------------------------------------+
The first filename entry is the message body. The next entries are the decoded uuencoded files.
EXAMPLES
Example #1
mailparse_uudecode_all(3) example
<?php
$text = <<<EOD
To: fred@example.com
hello, this is some text hello.
blah blah blah.
begin 644 test.txt
/=&AI<R!I<R!A('1E<W0*
`
end
EOD;
$fp = tmpfile();
fwrite($fp, $text);
$data = mailparse_uudecode_all($fp);
echo "BODY
";
readfile($data[0]["filename"]);
echo "UUE ({$data[1]['origfilename']})
";
readfile($data[1]["filename"]);
// Clean up
unlink($data[0]["filename"]);
unlink($data[1]["filename"]);
?>
The above example will output:
BODY
To: fred@example.com
hello, this is some text hello.
blah blah blah.
UUE (test.txt)
this is a test
PHP Documentation Group MAILPARSE_UUDECODE_ALL(3)