Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Complex text parsing with speed/performance problem (awk solution?)
Top Forums Shell Programming and Scripting Complex text parsing with speed/performance problem (awk solution?) Post 302781107 by Michael Stora on Friday 15th of March 2013 02:32:07 PM
Old 03-15-2013
Thanks for giving me some code to work with. Here is an actual data file (with IP obfuscation)

Code:
XXXXXXX,,,,XXXXXXX
XXXXXXX,XXXXXXX,,,
XXXXXXX,XXXXXXX,,,
XXXXXXX,,,,
,,,,
XXXXXXX,,,,
XXXXXXX,XXXXXXX,,,
,,,,
XXXXXXX,,,,
XXXXXXX,XXXXXXX,,,
XXXXXXX,411.7075872,,,
XXXXXXX,388.756628,,,
XXXXXXX,384.2531634,,,
XXXXXXX,188.317418,,,
XXXXXXX,495.1306749,,,
XXXXXXX,495.7313397,,,
XXXXXXX,364.8057139,,,
XXXXXXX,128.1652694,,,
XXXXXXX,78.1880777,,,
XXXXXXX,47.85832595,,,
XXXXXXX,397.106979,,,
XXXXXXX,171.5723148,,,
XXXXXXX,452.5367818,,,
XXXXXXX,334.4613963,,,
XXXXXXX,245.0863368,,,
XXXXXXX,182.0549603,,,
XXXXXXX,495.5126526,,,
XXXXXXX,30.64512099,,,
XXXXXXX,291.9205658,,,
XXXXXXX,221.6485369,,,
XXXXXXX,24.33776897,,,
XXXXXXX,270.5466812,,,
XXXXXXX,32.99794073,,,
XXXXXXX,183.2580134,,,

Mike
 

6 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk parsing problem

I need help with a problem that I have not been able to figure out. I have a file that is about 650K lines. Records are seperated by blank lines, fields seperated by new lines. I was trying to make a report that would add up 2 fields and associate them with a CP. example output would be... (11 Replies)
Discussion started by: timj123
11 Replies

2. Shell Programming and Scripting

Parsing a complex log file

I have a log file that has many SQL statements/queries/blocks and their resultant output (success or failure) added to each of them. I need to pick up all the statements which caused errors and write them to a separate file. On most cases, the SQL statement is a single line, like DROP . And if... (1 Reply)
Discussion started by: exchequer598
1 Replies

3. Shell Programming and Scripting

Difficult problem: Complex text file manipulation in bash script.

I don't know if this is a big issue or not, but I'm having difficulties. I apoligize for the upcoming essay :o. I'm writing a script, similar to a paint program that edits images, but in the form of ANSI block characters. The program so far is working. I managed to save the image into a file,... (14 Replies)
Discussion started by: tinman47
14 Replies

4. Shell Programming and Scripting

Complex awk problem

hello, i have a complex awk problem... i have two tables, one with a value (0 to 1) and it's corresponding p-value, like this: 1. table: ______________________________ value p-value ... ... 0.254 0.003 0.245 0.005 0.233 0.006 ... ... ______________________________ and a... (6 Replies)
Discussion started by: dietmar13
6 Replies

5. Shell Programming and Scripting

Text string parsing in awk

I have a awk script that parses many millions of lines so performance is critical. At one point I am extracting some variables from a space delimited string. alarm = $11; len = split(alarm,a," "); ent = a; chem = a; for (i = 5; i<= len; i++) {chem = chem " " a}It works but is slow. Adding the... (7 Replies)
Discussion started by: Michael Stora
7 Replies

6. Shell Programming and Scripting

awk parsing problem

Hello fellow unix geeks, I am having a small dilemna trying to parse a log file I have. Below is a sample of what it will look like: MY_TOKEN1(group) TOKEN(other)|SSID1 MY_TOKEN2(group, group2)|SSID2 What I need to do is only keep the MY_TOKEN pieces and where there are multiple... (7 Replies)
Discussion started by: dagamier
7 Replies

LEARN ABOUT MOJAVE

datetime::locale::ru5.18

DateTime::Locale::ru(3) 				User Contributed Perl Documentation				   DateTime::Locale::ru(3)

NAME

       DateTime::Locale::ru

SYNOPSIS

	 use DateTime;

	 my $dt = DateTime->now( locale => 'ru' );
	 print $dt->month_name();

DESCRIPTION

       This is the DateTime locale package for Russian.

DATA

       This locale inherits from the DateTime::Locale::root locale.

       It contains the following data.

   Days
       Wide (format)

	 XXXXXXXXXXX
	 XXXXXXX
	 XXXXX
	 XXXXXXX
	 XXXXXXX
	 XXXXXXX
	 XXXXXXXXXXX

       Abbreviated (format)

	 XX
	 XX
	 XX
	 XX
	 XX
	 XX
	 XX

       Narrow (format)

	 X
	 X
	 X
	 X
	 X
	 X
	 X

       Wide (stand-alone)

	 XXXXXXXXXXX
	 XXXXXXX
	 XXXXX
	 XXXXXXX
	 XXXXXXX
	 XXXXXXX
	 XXXXXXXXXXX

       Abbreviated (stand-alone)

	 XX
	 XX
	 XX
	 XX
	 XX
	 XX
	 XX

       Narrow (stand-alone)

	 X
	 X
	 X
	 X
	 X
	 X
	 X

   Months
       Wide (format)

	 XXXXXX
	 XXXXXXX
	 XXXXX
	 XXXXXX
	 XXX
	 XXXX
	 XXXX
	 XXXXXXX
	 XXXXXXXX
	 XXXXXXX
	 XXXXXX
	 XXXXXXX

       Abbreviated (format)

	 XXX.
	 XXXX.
	 XXXXX
	 XXX.
	 XXX
	 XXXX
	 XXXX
	 XXX.
	 XXXX.
	 XXX.
	 XXXX.
	 XXX.

       Narrow (format)

	 X
	 X
	 X
	 X
	 X
	 X
	 X
	 X
	 X
	 X
	 X
	 X

       Wide (stand-alone)

	 XXXXXX
	 XXXXXXX
	 XXXX
	 XXXXXX
	 XXX
	 XXXX
	 XXXX
	 XXXXXX
	 XXXXXXXX
	 XXXXXXX
	 XXXXXX
	 XXXXXXX

       Abbreviated (stand-alone)

	 XXX.
	 XXXX.
	 XXXX
	 XXX.
	 XXX
	 XXXX
	 XXXX
	 XXX.
	 XXXX.
	 XXX.
	 XXXX.
	 XXX.

       Narrow (stand-alone)

	 X
	 X
	 X
	 X
	 X
	 X
	 X
	 X
	 X
	 X
	 X
	 X

   Quarters
       Wide (format)

	 1-X XXXXXXX
	 2-X XXXXXXX
	 3-X XXXXXXX
	 4-X XXXXXXX

       Abbreviated (format)

	 1-X XX.
	 2-X XX.
	 3-X XX.
	 4-X XX.

       Narrow (format)

	 1
	 2
	 3
	 4

       Wide (stand-alone)

	 1-X XXXXXXX
	 2-X XXXXXXX
	 3-X XXXXXXX
	 4-X XXXXXXX

       Abbreviated (stand-alone)

	 1-X XX.
	 2-X XX.
	 3-X XX.
	 4-X XX.

       Narrow (stand-alone)

	 1
	 2
	 3
	 4

   Eras
       Wide

	 XX X.X.
	 X.X.

       Abbreviated

	 XX X.X.
	 X.X.

       Narrow

	 XX X.X.
	 X.X.

   Date Formats
       Full

	  2008-02-05T18:30:30 = XXXXXXX, 5 XXXXXXX 2008 X.
	  1995-12-22T09:05:02 = XXXXXXX, 22 XXXXXXX 1995 X.
	 -0010-09-15T04:44:23 = XXXXXXX, 15 XXXXXXXX -10 X.

       Long

	  2008-02-05T18:30:30 = 5 XXXXXXX 2008 X.
	  1995-12-22T09:05:02 = 22 XXXXXXX 1995 X.
	 -0010-09-15T04:44:23 = 15 XXXXXXXX -10 X.

       Medium

	  2008-02-05T18:30:30 = 05.02.2008
	  1995-12-22T09:05:02 = 22.12.1995
	 -0010-09-15T04:44:23 = 15.09.-010

       Short

	  2008-02-05T18:30:30 = 05.02.08
	  1995-12-22T09:05:02 = 22.12.95
	 -0010-09-15T04:44:23 = 15.09.-10

       Default

	  2008-02-05T18:30:30 = 05.02.2008
	  1995-12-22T09:05:02 = 22.12.1995
	 -0010-09-15T04:44:23 = 15.09.-010

   Time Formats
       Full

	  2008-02-05T18:30:30 = 18:30:30 UTC
	  1995-12-22T09:05:02 = 9:05:02 UTC
	 -0010-09-15T04:44:23 = 4:44:23 UTC

       Long

	  2008-02-05T18:30:30 = 18:30:30 UTC
	  1995-12-22T09:05:02 = 9:05:02 UTC
	 -0010-09-15T04:44:23 = 4:44:23 UTC

       Medium

	  2008-02-05T18:30:30 = 18:30:30
	  1995-12-22T09:05:02 = 9:05:02
	 -0010-09-15T04:44:23 = 4:44:23

       Short

	  2008-02-05T18:30:30 = 18:30
	  1995-12-22T09:05:02 = 9:05
	 -0010-09-15T04:44:23 = 4:44

       Default

	  2008-02-05T18:30:30 = 18:30:30
	  1995-12-22T09:05:02 = 9:05:02
	 -0010-09-15T04:44:23 = 4:44:23

   Datetime Formats
       Full

	  2008-02-05T18:30:30 = XXXXXXX, 5 XXXXXXX 2008 X. 18:30:30 UTC
	  1995-12-22T09:05:02 = XXXXXXX, 22 XXXXXXX 1995 X. 9:05:02 UTC
	 -0010-09-15T04:44:23 = XXXXXXX, 15 XXXXXXXX -10 X. 4:44:23 UTC

       Long

	  2008-02-05T18:30:30 = 5 XXXXXXX 2008 X. 18:30:30 UTC
	  1995-12-22T09:05:02 = 22 XXXXXXX 1995 X. 9:05:02 UTC
	 -0010-09-15T04:44:23 = 15 XXXXXXXX -10 X. 4:44:23 UTC

       Medium

	  2008-02-05T18:30:30 = 05.02.2008 18:30:30
	  1995-12-22T09:05:02 = 22.12.1995 9:05:02
	 -0010-09-15T04:44:23 = 15.09.-010 4:44:23

       Short

	  2008-02-05T18:30:30 = 05.02.08 18:30
	  1995-12-22T09:05:02 = 22.12.95 9:05
	 -0010-09-15T04:44:23 = 15.09.-10 4:44

       Default

	  2008-02-05T18:30:30 = 05.02.2008 18:30:30
	  1995-12-22T09:05:02 = 22.12.1995 9:05:02
	 -0010-09-15T04:44:23 = 15.09.-010 4:44:23

   Available Formats
       d (d)

	  2008-02-05T18:30:30 = 5
	  1995-12-22T09:05:02 = 22
	 -0010-09-15T04:44:23 = 15

       Ed (E d)

	  2008-02-05T18:30:30 = XX 5
	  1995-12-22T09:05:02 = XX 22
	 -0010-09-15T04:44:23 = XX 15

       EEEd (d EEE)

	  2008-02-05T18:30:30 = 5 XX
	  1995-12-22T09:05:02 = 22 XX
	 -0010-09-15T04:44:23 = 15 XX

       H (H)

	  2008-02-05T18:30:30 = 18
	  1995-12-22T09:05:02 = 9
	 -0010-09-15T04:44:23 = 4

       HHmm (HH:mm)

	  2008-02-05T18:30:30 = 18:30
	  1995-12-22T09:05:02 = 09:05
	 -0010-09-15T04:44:23 = 04:44

       HHmmss (HH:mm:ss)

	  2008-02-05T18:30:30 = 18:30:30
	  1995-12-22T09:05:02 = 09:05:02
	 -0010-09-15T04:44:23 = 04:44:23

       Hm (H:mm)

	  2008-02-05T18:30:30 = 18:30
	  1995-12-22T09:05:02 = 9:05
	 -0010-09-15T04:44:23 = 4:44

       hm (h:mm a)

	  2008-02-05T18:30:30 = 6:30 PM
	  1995-12-22T09:05:02 = 9:05 AM
	 -0010-09-15T04:44:23 = 4:44 AM

       Hms (H:mm:ss)

	  2008-02-05T18:30:30 = 18:30:30
	  1995-12-22T09:05:02 = 9:05:02
	 -0010-09-15T04:44:23 = 4:44:23

       hms (h:mm:ss a)

	  2008-02-05T18:30:30 = 6:30:30 PM
	  1995-12-22T09:05:02 = 9:05:02 AM
	 -0010-09-15T04:44:23 = 4:44:23 AM

       M (L)

	  2008-02-05T18:30:30 = 2
	  1995-12-22T09:05:02 = 12
	 -0010-09-15T04:44:23 = 9

       Md (d.M)

	  2008-02-05T18:30:30 = 5.2
	  1995-12-22T09:05:02 = 22.12
	 -0010-09-15T04:44:23 = 15.9

       MEd (E, M-d)

	  2008-02-05T18:30:30 = XX, 2-5
	  1995-12-22T09:05:02 = XX, 12-22
	 -0010-09-15T04:44:23 = XX, 9-15

       MMdd (dd.MM)

	  2008-02-05T18:30:30 = 05.02
	  1995-12-22T09:05:02 = 22.12
	 -0010-09-15T04:44:23 = 15.09

       MMM (LLL)

	  2008-02-05T18:30:30 = XXXX.
	  1995-12-22T09:05:02 = XXX.
	 -0010-09-15T04:44:23 = XXXX.

       MMMd (d MMM)

	  2008-02-05T18:30:30 = 5 XXXX.
	  1995-12-22T09:05:02 = 22 XXX.
	 -0010-09-15T04:44:23 = 15 XXXX.

       MMMEd (E MMM d)

	  2008-02-05T18:30:30 = XX XXXX. 5
	  1995-12-22T09:05:02 = XX XXX. 22
	 -0010-09-15T04:44:23 = XX XXXX. 15

       MMMMd (d MMMM)

	  2008-02-05T18:30:30 = 5 XXXXXXX
	  1995-12-22T09:05:02 = 22 XXXXXXX
	 -0010-09-15T04:44:23 = 15 XXXXXXXX

       MMMMEd (E MMMM d)

	  2008-02-05T18:30:30 = XX XXXXXXX 5
	  1995-12-22T09:05:02 = XX XXXXXXX 22
	 -0010-09-15T04:44:23 = XX XXXXXXXX 15

       mmss (mm:ss)

	  2008-02-05T18:30:30 = 30:30
	  1995-12-22T09:05:02 = 05:02
	 -0010-09-15T04:44:23 = 44:23

       ms (mm:ss)

	  2008-02-05T18:30:30 = 30:30
	  1995-12-22T09:05:02 = 05:02
	 -0010-09-15T04:44:23 = 44:23

       y (y)

	  2008-02-05T18:30:30 = 2008
	  1995-12-22T09:05:02 = 1995
	 -0010-09-15T04:44:23 = -10

       yM (yyyy-M)

	  2008-02-05T18:30:30 = 2008-2
	  1995-12-22T09:05:02 = 1995-12
	 -0010-09-15T04:44:23 = -010-9

       yMEd (EEE, yyyy-M-d)

	  2008-02-05T18:30:30 = XX, 2008-2-5
	  1995-12-22T09:05:02 = XX, 1995-12-22
	 -0010-09-15T04:44:23 = XX, -010-9-15

       yMMM (MMM y)

	  2008-02-05T18:30:30 = XXXX. 2008
	  1995-12-22T09:05:02 = XXX. 1995
	 -0010-09-15T04:44:23 = XXXX. -10

       yMMMEd (E, d MMM y)

	  2008-02-05T18:30:30 = XX, 5 XXXX. 2008
	  1995-12-22T09:05:02 = XX, 22 XXX. 1995
	 -0010-09-15T04:44:23 = XX, 15 XXXX. -10

       yMMMM (MMMM y)

	  2008-02-05T18:30:30 = XXXXXXX 2008
	  1995-12-22T09:05:02 = XXXXXXX 1995
	 -0010-09-15T04:44:23 = XXXXXXXX -10

       yQ (Q y)

	  2008-02-05T18:30:30 = 1 2008
	  1995-12-22T09:05:02 = 4 1995
	 -0010-09-15T04:44:23 = 3 -10

       yQQQ (y QQQ)

	  2008-02-05T18:30:30 = 2008 1-X XX.
	  1995-12-22T09:05:02 = 1995 4-X XX.
	 -0010-09-15T04:44:23 = -10 3-X XX.

       yyMM (MM.yy)

	  2008-02-05T18:30:30 = 02.08
	  1995-12-22T09:05:02 = 12.95
	 -0010-09-15T04:44:23 = 09.-10

       yyMMM (MMM yy)

	  2008-02-05T18:30:30 = XXXX. 08
	  1995-12-22T09:05:02 = XXX. 95
	 -0010-09-15T04:44:23 = XXXX. -10

       yyMMMEEEd (EEE, d MMM yy)

	  2008-02-05T18:30:30 = XX, 5 XXXX. 08
	  1995-12-22T09:05:02 = XX, 22 XXX. 95
	 -0010-09-15T04:44:23 = XX, 15 XXXX. -10

       yyQ (Q yy)

	  2008-02-05T18:30:30 = 1 08
	  1995-12-22T09:05:02 = 4 95
	 -0010-09-15T04:44:23 = 3 -10

       yyyy (y)

	  2008-02-05T18:30:30 = 2008
	  1995-12-22T09:05:02 = 1995
	 -0010-09-15T04:44:23 = -10

       yyyyLLLL (LLLL y)

	  2008-02-05T18:30:30 = XXXXXXX 2008
	  1995-12-22T09:05:02 = XXXXXXX 1995
	 -0010-09-15T04:44:23 = XXXXXXXX -10

       yyyyMM (MM.yyyy)

	  2008-02-05T18:30:30 = 02.2008
	  1995-12-22T09:05:02 = 12.1995
	 -0010-09-15T04:44:23 = 09.-010

       yyyyMMMM (MMMM y)

	  2008-02-05T18:30:30 = XXXXXXX 2008
	  1995-12-22T09:05:02 = XXXXXXX 1995
	 -0010-09-15T04:44:23 = XXXXXXXX -10

       yyyyQQQQ (QQQQ y 'X'.)

	  2008-02-05T18:30:30 = 1-X XXXXXXX 2008 X.
	  1995-12-22T09:05:02 = 4-X XXXXXXX 1995 X.
	 -0010-09-15T04:44:23 = 3-X XXXXXXX -10 X.

   Miscellaneous
       Prefers 24 hour time?

       Yes

       Local first day of the week

       XXXXXXXXXXX

SUPPORT

       See DateTime::Locale.

AUTHOR

       Dave Rolsky <autarch@urth.org>

COPYRIGHT

       Copyright (c) 2008 David Rolsky. All rights reserved. This program is free software; you can redistribute it and/or modify it under the
       same terms as Perl itself.

       This module was generated from data provided by the CLDR project, see the LICENSE.cldr in this distribution for details on the CLDR data's
       license.

perl v5.18.2							    2017-10-06						   DateTime::Locale::ru(3)
All times are GMT -4. The time now is 06:42 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy