12-31-2008
Parsing a string in PERL
I have an extractfile (with fields delimited by pipes '|') and I want to prepend a counter based on the below requirements:
- The counter starts at 3.
- The counter increments only if the date (67th field of the extractfile) is different.
Below is what I started off with:
$cnt=2;
$date="dummy";
open( extractFile, "<$extractFile" ) or die "ERROR: Can't open file: $extractFile - $!\n";
open( outputFile, ">$outputFile" ) or die "ERROR: Can't write file: $outputFile - $!\n";
while( $extractLine = <extractFile> ) {
# if the date is the same then prepend the counter without incrementing it
print outputFile "$cnt|$extractLine";
# else if the date is different, then increment the counter and then prepend it
cnt++;
print outputFile "$cnt|$extractLine";
}
I want to avoid using the 'split' by pipes to read in the records because I will be dealing with 800 million record extract. So I want to make this process as efficient as possible.
To clarify, I need help in reading in a certain field (which is a date - 67th field) from a file without using the 'split' command.
Thanks,
- CB
Last edited by ChicagoBlues; 12-31-2008 at 08:24 PM..
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi,
I'm writing a shell script that outputs, among other things, some of the information that is outputted by the mysqladmin status command.
The output of the command looks like this:
Uptime: 816351 Threads: 19 Questions: 80719739 Slow queries: 1419 Opens: 15903523 Flush tables: 1 Open tables:... (6 Replies)
Discussion started by: achieve
6 Replies
2. Shell Programming and Scripting
#! /usr/local/bin/perl -w
$ip = "$ARGV";
$rw = "$ARGV";
$snmpg = "/usr/local/bin/snmpbulkget -v2c -Cn1 -Cn2 -Os -c $rw";
$snmpw = "/usr/local/bin/snmpwalk -Os -c $rw";
$syst=`$snmpg $ip system sysName sysObjectID`;
sysDescr.0 = STRING: Cisco Internetwork Operating System Software... (1 Reply)
Discussion started by: popeye
1 Replies
3. Shell Programming and Scripting
Hey guys,
I have this file generated by me... i want to create some HTML output from it.
The problem is that i am really confused about how do I go about reading the file.
The file is in the following format:
TID1 Name1 ATime=xx AResult=yyy AExpected=yyy BTime=xx BResult=yyy... (8 Replies)
Discussion started by: umar.shaikh
8 Replies
4. Shell Programming and Scripting
I need the perl solution for the following :
$string="I LOVE INDIA"
now, in a new string i need the first character of each word...
that is string2 should be "ILN". (10 Replies)
Discussion started by: vijay_0209
10 Replies
5. Shell Programming and Scripting
Hello All,
I have a case, wherein I have a string of the format
"attr1=value1 attr2=value2 attr3=value3 attr4=value4"
How do I extract the value associated with for a given attributename. For eg. I need to get a value of "value2" when I give an input for attribute name as "attr2". Note, each... (10 Replies)
Discussion started by: msgforsunil
10 Replies
6. Shell Programming and Scripting
Hi,
name=VDSL_TTV_ HN_SUB create coid=MA5603U phone=5678 portpolicy=APortSelectionPolicy rfu10=TTV rfu3=Dot1q sz7_portmode=VDSL2 rfu5=1234 srprofile.sy_profname=$ADSL_TTV_SubProfile1
I have a line like this. Its a single line.I need the output as
name=VDSL_TTV_ HN_SUB create... (1 Reply)
Discussion started by: giri_luck
1 Replies
7. Shell Programming and Scripting
Hi,
name=VDSL_TTV_ HN_SUB create coid=MA5603U phone=5678 portpolicy=APortSelectionPolicy rfu10=TTV rfu3=Dot1q sz7_portmode=VDSL2 rfu5=1234 srprofile.sy_profname=$ADSL_TTV_SubProfile1
I have a line like this. Its a single line.I need the output as
name=VDSL_TTV_ HN_SUB create... (6 Replies)
Discussion started by: giri_luck
6 Replies
8. Shell Programming and Scripting
for i in `cat list`;do lol=`curl -m 2 -s ${i} | grep 'class=info' | cut -d '>' -f14 | cut -d '<' -f1 | sed '/^$/d'`; if ;then echo "$i,$lol" >> dirty; echo "$i,$lol";fi; done
cut: you must specify a list of bytes, characters, or fields
Try `cut --help' for more information.
it gave me that... (0 Replies)
Discussion started by: p33plime
0 Replies
9. Shell Programming and Scripting
Hi everyone,
I am working on fetchmail + procmail to filter mails and I am having problem with parsing a long line in the body of the email.
Could anyone help me construct a reg exp for this string below. It needs to match exactly as this string.
GetRyt... (4 Replies)
Discussion started by: cwiggler
4 Replies
10. UNIX for Beginners Questions & Answers
Hi,
I need some guidance with understanding this Perl script below. I am not the author of the script and the author has not leave any documentation. I supposed it is meant to be 'easy' if you're a Perl or regex guru. I am having problem understanding what regex to use :confused: The script does... (3 Replies)
Discussion started by: newbie_01
3 Replies
LEARN ABOUT DEBIAN
perfmon
PERFMON(4) BSD/i386 Kernel Interfaces Manual PERFMON(4)
NAME
perfmon -- CPU performance-monitoring interface
SYNOPSIS
cpu I586_CPU
cpu I686_CPU
options PERFMON
DESCRIPTION
The perfmon driver provides access to the internal performance-monitoring capabilities of the Intel Pentium and Pentium Pro CPUs. These pro-
cessors implement two internal counters which can be configured to measure a variety of events for either count or duration (in CPU cycles),
as well as a cycle counter which counts clock cycles. The perfmon driver provides a device-style interface to these capabilities.
All access to the performance-monitoring counters is performed through the special device file ``/dev/perfmon''. This device supports a num-
ber of ioctl(2) requests, defined in <machine/perfmon.h> along with the definitions of the various counters for both Pentium and Pentium Pro
processors.
NOTA BENE: The set of available events differs from processor to processor. It is the responsibility of the programmer to ensure that the
event numbers used are the correct ones for the CPU type being measured.
The following ioctl(2) requests are defined:
PMIOSETUP (struct pmc) Set up a counter with parameters and flags defined in the structure. The following fields are defined in struct
pmc:
int pmc_num the number of the counter in question; must be less than NPMC (currently 2).
u_char pmc_event the particular event number to be monitored, as defined in <machine/perfmon.h>.
u_char pmc_unit the unit mask value, specific to the event type (see the Intel documentation).
u_char pmc_flags flags modifying the operation of the counter (see below).
u_char pmc_mask the counter mask value; essentially, this is a threshold used to restrict the count to events lasting more (or
less) than the specified number of clocks.
The following pmc_flags values are defined:
PMCF_USR count events in user mode
PMCF_OS count events in kernel mode
PMCF_E count number of events rather than their duration
PMCF_INV invert the sense of the counter mask comparison
PMIOGET (struct pmc) returns the current configuration of the specified counter.
PMIOSTART
PMIOSTOP (int) starts (stops) the specified counter. Due to hardware deficiencies, counters must be started and stopped in numerical
order. (That is to say, counter 0 can never be stopped without first stopping counter 1.) The driver will not enforce this
restriction (since it may not be present in future CPUs).
PMIORESET (int) reset the specified counter to zero. The counter should be stopped with PMIOSTOP before it is reset. All counters are
automatically reset by PMIOSETUP.
PMIOREAD (struct pmc_data) get the current value of the counter. The pmc_data structure defines two fields:
int pmcd_num the number of the counter to read
quad_t pmcd_value the resulting value as a 64-bit signed integer
In the future, it may be possible to use the RDPMC instruction on Pentium Pro processors to read the counters directly.
PMIOTSTAMP (struct pmc_tstamp) read the time stamp counter. The pmc_tstamp structure defines two fields:
int pmct_rate the approximate rate of the counter, in MHz
quad_t pmct_value the current value of the counter as a 64-bit integer
It is important to note that the counter rate, as provided in the pmct_rate field, is often incorrect because of calibration dif-
ficulties and non-integral clock rates. This field should be considered more of a hint or sanity-check than an actual represen-
tation of the rate of clock ticks.
FILES
/dev/perfmon character device interface to counters
/usr/include/machine/perfmon.h include file with definitions of structures and event types
/usr/share/examples/perfmon sample source code demonstrating use of all the ioctl() commands
SEE ALSO
ioctl(2), hwpmc(4)
Intel Corporation, Pentium Pro Family Developer's Manual, vol. 3, January 1996, Operating System Writer's Manual.
HISTORY
The perfmon device first appeared in FreeBSD 2.2.
AUTHORS
The perfmon driver was written by Garrett A. Wollman, MIT Laboratory for Computer Science.
BSD
March 26, 1996 BSD