*log to *.csv in perl


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting *log to *.csv in perl
# 1  
Old 12-19-2012
*log to *.csv in perl

I want to convert the log files to xls. by bash script I converted it to csv, but I need to make some changes in some of them. please dont ask why xls and not csv. I need xls and this is to be a automated process.
I am new to linux, bash and NULL in Perl.
That would be nice if someone help me to convert to xls :-)
this is my code in bash scripting which converts log to csv. Now I want log to xls.

Code:
#!/bin/bash
NAME=evaluation_`date +%Y%m%d`
find /dir -name "*.log" -mtime 0 -exec cat "{}" \; >> $NAME.log
grep -e word1 -e word2 $NAME.log | grep -v word3| awk '{n=split($11,P,"/");print $1 " " $2 ";" $3 ";" $9 ";" $11 ";" P[n]}' > $NAME.csv

sed -i '1iDate;Time;From;To;User' $NAME.csv

rm $NAME.log

echo "done."


Last edited by Scrutinizer; 12-20-2012 at 09:35 AM.. Reason: I want to wrap with code; mod: cleaned up excess font definitions
# 2  
Old 12-19-2012
Robust CSV needs quote doubling and comma-containing-field quoting (and technically, C/R before L/F), but for "well-behaved" data you can just delimit with commas.

No ned for an intermediate file, just pipe. I like "find ...|xargs -r cat|" for more pipeline parallelism and efficiency: not one fork/exec-cat per file.

I usually mtime -1, but only use it for casual things, else I avoid mtime, as the result is not well bounded. I use "find -newer" a marker file, either 'touch' to exact date-time or sleep 1 so any files modified in that second are bounded by last_marker and new_marker.

The grep -v might be earlier to reduce the work for the rest.

We used to call the CSV file something.xls to trigger nice associations. Even URLs that end in .xls like http://haha.hoho.com/cgi-bin/makexls?a=123&z=.xls are welcomed into a spreadsheet app.

There are also off the shelf writers of xls: (code) text2xls.pl (was csv2xls.pl) You don't need to know PERL to use PERL scripts in your shell script -- an executable is an executable; all any scripting language file needs is a #! first line and execute permissions, as described in man execve(): Man Page for execve (linux Section 2) - The UNIX and Linux Forums

Now that excel has xlsx, an xml flavor, I expect more tools will come out to write that, as it is XML (text) not binary. But of course many users still have older spreadsheet apps. You might publish it as a google docs spreadsheet -- no spreadsheet app needed! I just did an election report using it: https://docs.google.com/spreadsheet/ccc?key=0AhFSGPrNgs0mdGdIM1BnRmNNUHpQc2psZ1RHQ3dvMlE#gid=0

Last edited by DGPickett; 12-20-2012 at 10:53 AM..
# 3  
Old 12-20-2012
Thanks for your reply and suggestions.
when I run the script, I get:
Code:
Spreadsheet/WriteExcel.pm did not return a true value at 12.pl line 11.
BEGIN failed--compilation aborted at 12.pl line 11.

and in dir which I run my scripts Ihave also:
- ActivePerl-5.16.1.1601-x86_64-linux-glibc-2.3.5-296175
- Spreadsheet-ParseExcel-0.2602
- Spreadsheet-WriteExcel-2.38
I dont know why still I get the Error.

Could you please help me out in n underestanding way? Smilie

Last edited by Scrutinizer; 12-20-2012 at 09:32 AM.. Reason: code tags
# 4  
Old 12-20-2012
Quote:
Originally Posted by frhling
Thanks for your reply and suggestions.
when I run the script, I get:
Code:
Spreadsheet/WriteExcel.pm did not return a true value at 12.pl line 11.
BEGIN failed--compilation aborted at 12.pl line 11.

and in dir which I run my scripts Ihave also:
- ActivePerl-5.16.1.1601-x86_64-linux-glibc-2.3.5-296175
- Spreadsheet-ParseExcel-0.2602
- Spreadsheet-WriteExcel-2.38
I dont know why still I get the Error.

Could you please help me out in n underestanding way? Smilie
Code:
WriteExcel.pm module should end with '1;'. In fact, Each and every perl modules should be end with an positive value.

Cheers,
RangaSmilie
# 5  
Old 12-20-2012
Makes no sense to me, if you have a late perl version.
http://search.cpan.org/~jmcnamara/Sp...m#REQUIREMENTS
http://search.cpan.org/~jmcnamara/Sp...r_and_perl_5.6

The module comes with many examples: Spreadsheet::WriteExcel::Examples - search.cpan.org

Last edited by DGPickett; 12-20-2012 at 11:23 AM..
# 6  
Old 12-21-2012
Thanks.
When I write '1:' then the problem jumps to the next one.
lets say I have a perl script like this:
Code:
#!/usr/bin/perl -w

use strict;
use Spreadsheet::WriteExcel '1;'
use Text::CSV_XS;

# Check for valid number of arguments
if (($#ARGV < 1) || ($#ARGV > 2)) {
   die("Usage: csv2xls csvfile.txt newfile.xls\n");
};

# Open the Comma Separated Variable file
open (CSVFILE, $ARGV[0]) or die "$ARGV[0]: $!";

# Create a new Excel workbook
my $workbook  = Spreadsheet::WriteExcel->new($ARGV[1]);
my $worksheet = $workbook->add_worksheet();

# Create a new CSV parsing object
my $csv = Text::CSV_XS->new;

# Row and column are zero indexed
my $row = 0;


while (<CSVFILE>) {
    if ($csv->parse($_)) {
        my @Fld = $csv->fields;

        my $col = 0;
        foreach my $token (@Fld) {
            $worksheet->write($row, $col, $token);
            $col++;
        }
        $row++;
    }
    else {
        my $err = $csv->error_input;
        print "Text::CSV_XS parse() failed on argument: ", $err, "\n";
    }
}

How that can be solved?
# 7  
Old 12-21-2012
I got it running with perl 5.6.1 on hp-ux 11.00 with just one more line for my module loc and a simple semicolon:
Code:
#!/usr/bin/perl -w
use strict;
use lib '/myhome/perl/Spreadsheet-WriteExcel-2.38/lib';
use Spreadsheet::WriteExcel ;
 .
 .
 .
 .

(I forget how to use the @INC thing.)
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need to read a csv and create xml - Perl

I have a csv file like below. john,r2,testdomain1,john.r2@hex.com,DOMAINADMIN,testdomain1.dom maxwell,b2, testdomain1,maxwell.b2@hex.com,DOMAINADMIN,testdomain1.dom I would need the perl script to read the above csv and create an xml like below. <Users> ... (1 Reply)
Discussion started by: Tuxidow
1 Replies

2. Shell Programming and Scripting

Sorting the csv file in Perl

Hi All How all are doing today. Just struck in an issue in Perl I have a csv file which contain 32 column, I want to make sorting in that csv file with respect to 26th column. Is it possible to do so without any module being added? Regards Aditya ---------- Post updated at 10:02 AM... (11 Replies)
Discussion started by: adisky123
11 Replies

3. Shell Programming and Scripting

TEXT to CSV using Perl

Hi Folks Need some help with this and my Perl isnt the hottest I also have text::csv installed on my perl install The large text with a few million entries is in a format below example text file Fig Leafs Cake No: 0000001 Author: King s. Record No: 995-34343-232-232... (5 Replies)
Discussion started by: messiah1
5 Replies

4. Shell Programming and Scripting

Perl: module to convert xlsx to csv

Is there any perl module to convert .xlsx file(excel sheet 2007) to a csv file. (1 Reply)
Discussion started by: giridhar276
1 Replies

5. Shell Programming and Scripting

Perl search csv fileA where two strings exist on another csv fileB

Hi I have two csv files, with the following formats: FileA.log: Application, This occured blah Application, That occured blah Application, Also this AnotherLog, Bob did this AnotherLog, Dave did that FileB.log: Uk, London, Application, datetime, LaterDateTime, Today it had'nt... (8 Replies)
Discussion started by: PerlNewbRP
8 Replies

6. Programming

split xlsx to csv using perl

Hi, I am new to perl. I want to split excel 2007 file to csv. every formula should be evaluated on each field. Plz help me (1 Reply)
Discussion started by: malikshahid85
1 Replies

7. UNIX for Dummies Questions & Answers

How to compare csv files using perl

I need to compare 2 csv files and report should containg number of matching lines,different lines ,missing lines in one file using perl. I dont want to use read line by line and scan thru the second file for matching line ,as this logic was so time consuming .Can other ideas .please respond asap... (2 Replies)
Discussion started by: kittu1979
2 Replies

8. UNIX for Dummies Questions & Answers

Compare 2 csv files in perl

need to compare 2 csv files and report should containg number of matching lines,different lines ,missing lines in one file using perl. I dont want to use read line by line and scan thru the second file for matching line ,as this logic was so time consuming .Any ideas.i need the soultion badly .... (2 Replies)
Discussion started by: kittu1979
2 Replies

9. Shell Programming and Scripting

CSV File parse help in Perl

Folks, I have a bit of an issue trying to obtain some data from a csv file using PERL. I can sort the file and remove any duplicates leaving only 4 or 5 rows containing data. My problem is that the data contained in the original file contains a lot more columns and when I try ro run this script... (13 Replies)
Discussion started by: lodey
13 Replies

10. Shell Programming and Scripting

Perl - extracting data from .csv files

PROJECT: Extracting data from an employee timesheet. The timesheets are done in excel (for user ease) and then converted to .csv files that look like this (see color code key below): ,,,,,,,,,,,,,,,,,,, 9/14/2003,<-- Week Ending,,,,,,,,,,,,,,,,,, Craig Brennan,,,,,,,,,,,,,,,,,,,... (3 Replies)
Discussion started by: kregh99
3 Replies
Login or Register to Ask a Question