Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Perl - CAPTURE query

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Perl - CAPTURE query
# 8  
Old 05-05-2010
Hi thanks again,

the file will be something like as the text will be surrounded by quotes;

Code:
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,"1-this is a test",20
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,"2-this is a test",21
...

when I run

Code:
cat file.tmp | perl -pe 's/(([^,]*,){18}\w+)/$1,/'

it doesn't like the double quotes, I would want it to produce

something like

Code:
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,"1-","this is a test",20
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,"2"-"this is a test",21
...



---------- Post updated at 09:58 AM ---------- Previous update was at 08:56 AM ----------

Hopefully final question,

How do I call this from within a script so that it writes into a secondary file

e.g.,

Code:
my $tmpfile2=/tmp/tmpfile2.tmp;
cat /tmp/file.tmp | perl -pe 's/(([^,]*,){18}\w+)/$1,/' >> $tmpfile2

Last edited by Scott; 05-05-2010 at 01:47 PM.. Reason: Code tags please...
# 9  
Old 05-05-2010
Firstly, this -
Quote:
Originally Posted by gefa
...
cat file.tmp | perl -pe 's/(([^,]*,){18}\w+)/$1,/'
...
is eligible for Useless Use of Cat Award

And secondly -

Quote:
...
the file will be something like as the text will be surrounded by quotes;

1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,"1-this is a test",20
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,"2-this is a test",21
...

when I run

cat file.tmp | perl -pe 's/(([^,]*,){18}\w+)/$1,/'

it doesn't like the double quotes, I would want it to produce

something like

1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,"1-","this is a test",20
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,"2"-"this is a test",21
...
It doesn't like the fact that the double quotes aren't mentioned in your regex.

Code:
$
$
$ cat f0
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,"1-this is a test",20
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,"2-this is a test",21
$
$ perl -pe 's/(([^,]*,){18}"\d+-)/$1","/' f0
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,"1-","this is a test",20
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,"2-","this is a test",21
$
$

Quote:
...
How do I call this from within a script so that it writes into a secondary file

e.g.,

my $tmpfile2=/tmp/tmpfile2.tmp;
cat /tmp/file.tmp | perl -pe 's/(([^,]*,){18}\w+)/$1,/' >> $tmpfile2
If, by "a script", you mean "a shell script" then you can simply put whatever you executed on the dollar-prompt inside your shell script.

If it runs on the *nix command line successfully, it should run when put inside a shell script as well.

HTH,
tyler_durden
Last edited by durden_tyler; 05-05-2010 at 01:53 PM..
# 10  
Old 05-05-2010
Quote:
Originally Posted by gefa
Hi thanks again,

the file will be something like as the text will be surrounded by quotes;

Code:
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,"1-this is a test",20
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,"2-this is a test",21
...

when I run

Code:
cat file.tmp | perl -pe 's/(([^,]*,){18}\w+)/$1,/'

it doesn't like the double quotes, I would want it to produce

something like

Code:
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,"1-","this is a test",20
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,"2"-"this is a test",21
...

]
Be aware that Comma Separated Value "CSV" is far more complex in practice than simple test cases often suggest. This applies to Perl, Awk, whatever regex or parsing tool you are using.

If you are absolutely, positively, sure of your CSV format, you can write your own regex against it. Too often though, your CSV format is controlled by others and there are subtleties that will throw off your best laid plans.

Here is a regex I use often for CSV that I control. If I don't, I use Text::CSV cpan library...

That regex can be changed to get the nth field of CSV like so:

Code:
#!/usr/bin/perl
use warnings;
use strict;

while(<DATA>) {
	chomp;
	my $str=$_;
	
	#if you want to deal with all at once:
	my @fields=/(?:^|,)("(?:[^"]+|"")*"|[^,]*)/g;
	my $i=0;
	print "All fields= $str\n";
	foreach my $field (@fields) {
   		print "field $i of $#fields = \'$field\'\n";
   		$i+=1;
   	}	
   	
   	#if you want to deal with field n
   	my $n=18;
   	my $actual=$n+1;  #there is no zeroth match...
   	$str=~/((?:^|,)(?:"(?:[^"]+|"")*"|[^,]*)){$actual}/; 	
   	print "single field, field $n=$1\n\n";
}

__DATA__
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,"1-this is a test",20
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,"2-this is a test",21

The beauty of Perl is the strength of the solutions on CPAN. In the case of CSV, XML, HTML, or other difficult to regex things, use a CPAN tool that has been tested in literally millions of cases so that you are more certain of your solution.

Here is a good overview of parsing CSV with Perl. This is the area of strength of Perl, and it still has its difficulties...
Last edited by drewk; 05-05-2010 at 02:40 PM..
# 11  
Old 05-06-2010
Many thanks for all your help.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Capture output of open pipe to a file in perl

Hi, I am trying to capture the output of the an open pipe in perl. but I am not sure how to do this. can some one please help me do that? Below is the script I am using (2 Replies)
Discussion started by: ahmedwaseem2000
2 Replies

2. Programming

Perl Query

Hi , Im using the below script to find the Good and Bad for the file permission. $rc=`find /etc/security/opasswd -perm 0600 -print -ls`; if($rc == 1) { print "GOOD: AD.1.8.4.1: The file /etc/security/opasswd exists and had permission 0600\n\n"; ... (6 Replies)
Discussion started by: gsiva
6 Replies

3. Shell Programming and Scripting

Capture query returned values in file.

Hi All, I am connecting to Oracle DB from UNIX script. Want to capture all dates between start date and end date and store them in file. Once this is done, want to read dates one by one. How to achive this in UNIX and Oracle? Please let me know if you have any idea on the same. Thanks and... (4 Replies)
Discussion started by: Nagaraja Akkiva
4 Replies

4. Shell Programming and Scripting

Script to capture date/time in seconds in PERL... Cant understand errors

I'm Using this script to find the time of a file. I'm very much new to PERL and found this script posted by some one on this forum. It runs perfectly fine, just that it gives me following errors with the accurate output as well. I jus want the output to be stored in another file so that i can... (0 Replies)
Discussion started by: bankimmehta
0 Replies

5. Shell Programming and Scripting

[Perl] Capture system call error message.

Hi, I googled a bit, but could not find the answer to my problem. But I am sure it is a common issue. I have this code: #!/bin/perl -w #-d use strict; sub remsh_test() { my $host = $_; printf "\n----\n\n"; printf "remsh to $host with system call\n"; my $result = system... (3 Replies)
Discussion started by: ejdv
3 Replies

6. Shell Programming and Scripting

Perl Query Regarding format

Hello people. I have got the following script QM=ARGV; open (CHS_OUT, "echo 'DISPLAY QSTATUS(SYSTEM.CLUSTER.MY.QUEUE) all'|runmqsc $qm|"); while (<CHS_OUT>) { if ( /QUEUE\(/ ) { $QueueName = ValueParser("QUEUE", 6); } if ( /IPPROCS\(/ ) { $InpProcs = ValueParser("IPPROCS", 8); #print... (3 Replies)
Discussion started by: King Nothing
3 Replies

7. UNIX and Linux Applications

Perl - PostGreSql query

Guys, I guess, I posted something in the wrong forum. Here it is - https://www.unix.com/shell-programming-scripting/67395-perl-postgrepsql-question.html Can you please help me with this? Regards, garric (0 Replies)
Discussion started by: garric
0 Replies

8. UNIX for Dummies Questions & Answers

Query in perl

can any1 give me line by line explanation for the following perl script as i dunno perl .. n i have searched in google .. but still thn i wanna confirm my findings fro perl experts :mad: perl -e 'while (<>) { print; $num = 2 if /fail_halt/i; $num = 1 if (/failure/i && ($num < 1)); }... (2 Replies)
Discussion started by: Dana Evans
2 Replies

9. Shell Programming and Scripting

perl query

a file test.dat has the following David Veterinarian John Orthopedist Jeff Dentist perl -p -e "s/\s*(\w+).*/$1/;" test.dat ......will print David Jonh Jeff how does the part in double quotes work out.... (1 Reply)
Discussion started by: bishweshwar
1 Replies

10. Shell Programming and Scripting

lazy capture don't work (regexp in perl)

hello all im trying to capture only the first brackets but no matter what i do i keep capturing from the first brackets to the last one , here what i have : <% if (!Env.strLen(oidInvoice)); szDocumentTitle = Env.formatS("S",Env.getMsg("BP_INVOICE_ENUMERATION_CREATE_TITLE")) %> and... (1 Reply)
Discussion started by: umen
1 Replies
Login or Register to Ask a Question