Sponsored Content
Top Forums Shell Programming and Scripting Sorting problem: Multiple delimiters, multiple keys Post 302538043 by Ryan. on Monday 11th of July 2011 02:49:21 PM
Old 07-11-2011
Quote:
Originally Posted by alister
Are hours, minutes and seconds all zero padded? For example, 01:02:03 instead of 1:2:3 or 1:02:03? If so, you do not need to modify anything. You can use the default lexicographical sort with the date and time fields as the keys.

Also, you mentioned that hours range betwee 1-23. In case it's relevant, that's only a 23 hour day.

If the source file is 50+ GB, you are going to need a lot of ram. You'll probably need to split the file into smaller chunks, sort them individually, and then merge them with sort -m.

Regards,
Alister
Oddly the hours aren't zero padded but the minutes and seconds are. (I think it's like [1]?[0-9]:[0-5][0-9]:[0-5][0-9] in Regex-speak.)

I'm going to try to figure out how to split it up and then attempt sorting again -- thanks.

Last edited by Ryan.; 07-11-2011 at 04:10 PM.. Reason: Wrong
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk - treat multiple delimiters as one

Is there anyway to get awk to treat multiple delimiters as one? Particularly spaces... (6 Replies)
Discussion started by: peter.herlihy
6 Replies

2. Shell Programming and Scripting

Sorting with multiple numeric keys

Data I want to sort :- 1 10 jj Y 2 100 vv B 19 5 jj A 1 11 hq D 3 8 op X 44 78 ds GG 1 8 hq D and want to sort based on the first 2 columns - which hold numeric values. Am using : cat filename | sort -nk 1,2 But the result is :- 1 10 jj Y 1 11 hq D (1 Reply)
Discussion started by: sinpeak
1 Replies

3. Shell Programming and Scripting

Cutting a file with multiple delimiters into columns

Hi All I have recently had to start using Unix for work and I have hit brick wall with this prob.... I have a file that goes a little something like this.... EUR;EUR;EUR:USD:USD;USD;;;EUR/USD;XAU/AUD;XAU/EUR;XAU/AUD,GBP/BOB,UAD/XAU;;;1.11;2.22;3.33;4.44;5.55;6.66;;; is it possible to... (7 Replies)
Discussion started by: luckycharm
7 Replies

4. Shell Programming and Scripting

AWK with multiple delimiters

I have the following string sample: bla bla bla bla bla I would like to extract the "123" using awk. I thought about awk -F"]" '{ print $1 }' but it doesn't work Any ideas ? (7 Replies)
Discussion started by: gdub
7 Replies

5. Shell Programming and Scripting

Sorting based on multiple delimiters

Hello, I have data where words are separated by a delimiter. In this case "=" The number of delimiters in a line can vary from 4to 8. The norm is 4. Is it possible to have a script where the file could be separated starting with highest number of delimiters and ending with the lowest An... (8 Replies)
Discussion started by: gimley
8 Replies

6. Shell Programming and Scripting

treating multiple delimiters[solved]

Hi, I need to display the last column value in the below o/p. sam2 PS 03/10/11 0 441 Unable to get o/p with this awk code awk -F"+" '{ print $4 }' pwdchk.txt I need to display 441(in this eg.) and also accept it as a variable to treat it with if condition and take a decision.... (1 Reply)
Discussion started by: sam_bd
1 Replies

7. Shell Programming and Scripting

awk multiple delimiters

Hi Folks, This is the first time I ever encountered this situation My input file is of this kind cat input.txt 1 PAIXAF 0 1 1 -9 0 0 0 1 2 0 2 1 2 1 7 PAIXEM 0 7 1 -9 1 0 2 0 1 2 2 1 0 2 9 PAKZXY 0 2 1 -9 2 0 1 1 1 0 1 2 0 1 Till the sixth column (which is -9), I want my columns to... (4 Replies)
Discussion started by: jacobs.smith
4 Replies

8. Shell Programming and Scripting

Editing phone number with multiple delimiters

Hello all I have a data base of information that is formatted like so: JSD4863 XXX-XX-XXXX DOE, JOHN C JR-II BISS CPSC BS INFO TECH 412/779-9445 I need the last four digits of the phone number. However, many lines contain 'garbage data' that I'm not interested in. So i used a 'for loop'... (7 Replies)
Discussion started by: smartSometimes
7 Replies

9. UNIX for Beginners Questions & Answers

How to append the multiple Delimiters up to requirement?

HI All, How to append the multiple delimiters to at end the file up to 69 fields. FinalDelimiter Count is 69 recrod Delimeter count is 10 so 69-10=59 this script will add upto 59 Delimiters to that records. this script will check each and every record in a file and append the delimiters... (4 Replies)
Discussion started by: vinod.peddiredd
4 Replies

10. Shell Programming and Scripting

Getting fields from a file having multiple delimiters

Hi All, I have a file with a single row having the following text ABC.ABC.ABC,Database,New123,DBNAME,F,ABC.ABC.ABC_APP,"@FUNCTION1("ENT1") ,@FUNCTION2("ENT2")",R, I want an output in the following format ABC.ABC.ABC DBNAME ABC.ABC.ABC_APP '@FUNCTION1("ENT1")... (3 Replies)
Discussion started by: dev.devil.1983
3 Replies
Duration(3)						User Contributed Perl Documentation					       Duration(3)

NAME
Time::Duration - rounded or exact English expression of durations SYNOPSIS
Example use in a program that ends by noting its runtime: my $start_time = time(); use Time::Duration; # then things that take all that time, and then ends: print "Runtime ", duration(time() - $start_time), ". "; Example use in a program that reports age of a file: use Time::Duration; my $file = 'that_file'; my $age = $^T - (stat($file))[9]; # 9 = modtime print "$file was modified ", ago($age); DESCRIPTION
This module provides functions for expressing durations in rounded or exact terms. In the first example in the Synopsis, using duration($interval_seconds): If the "time() - $start_time" is 3 seconds, this prints "Runtime: 3 seconds.". If it's 0 seconds, it's "Runtime: 0 seconds.". If it's 1 second, it's "Runtime: 1 second.". If it's 125 seconds, you get "Runtime: 2 minutes and 5 seconds.". If it's 3820 seconds (which is exactly 1h, 3m, 40s), you get it rounded to fit within two expressed units: "Runtime: 1 hour and 4 minutes.". Using duration_exact instead would return "Runtime: 1 hour, 3 minutes, and 40 seconds". In the second example in the Synopsis, using ago($interval_seconds): If the $age is 3 seconds, this prints "file was modified 3 seconds ago". If it's 0 seconds, it's "file was modified just now", as a special case. If it's 1 second, it's "from 1 second ago". If it's 125 seconds, you get "file was modified 2 minutes and 5 seconds ago". If it's 3820 seconds (which is exactly 1h, 3m, 40s), you get it rounded to fit within two expressed units: "file was modified 1 hour and 4 minutes ago". Using ago_exact instead would return "file was modified 1 hour, 3 minutes, and 40 seconds ago". And if the file's modtime is, surprisingly, three seconds into the future, $age is -3, and you'll get the equally and appropriately surprising "file was modified 3 seconds from now." FUNCTIONS
This module provides all the following functions, which are all exported by default when you call "use Time::Duration;". duration($seconds) duration($seconds, $precision) Returns English text expressing the approximate time duration of abs($seconds), with at most "$precision || 2" expressed units. (That is, duration($seconds) is the same as duration($seconds,2).) For example, duration(120) or duration(-120) is "2 minutes". And duration(0) is "0 seconds". The precision figure means that no more than that many units will be used in expressing the time duration. For example, 31,629,659 seconds is a duration of exactly 1 year, 1 day, 2 hours, and 59 seconds (assuming 1 year = exactly 365 days, as we do assume in this module). However, if you wanted an approximation of this to at most two expressed (i.e., nonzero) units, it would round it and truncate it to "1 year and 1 day". Max of 3 expressed units would get you "1 year, 1 day, and 2 hours". Max of 4 expressed units would get you "1 year, 1 day, 2 hours, and 59 seconds", which happens to be exactly true. Max of 5 (or more) expressed units would get you the same, since there are only four nonzero units possible in for that duration. duration_exact($seconds) Same as duration($seconds), except that the returned value is an exact (unrounded) expression of $seconds. For example, duration_exact(31629659) returns "1 year, 1 day, 2 hours, and 59 seconds later", which is exactly true. ago($seconds) ago($seconds, $precision) For a positive value of seconds, this prints the same as "duration($seconds, [$precision]) . ' ago'". For example, ago(120) is "2 minutes ago". For a negative value of seconds, this prints the same as "duration($seconds, [$precision]) . ' from now'". For example, ago(-120) is "2 minutes from now". As a special case, ago(0) returns "right now". ago_exact($seconds) Same as ago($seconds), except that the returned value is an exact (unrounded) expression of $seconds. from_now($seconds) from_now($seconds, $precision) from_now_exact($seconds) The same as ago(-$seconds), ago(-$seconds, $precision), ago_exact(-$seconds). For example, from_now(120) is "2 minutes from now". later($seconds) later($seconds, $precision) For a positive value of seconds, this prints the same as "duration($seconds, [$precision]) . ' later'". For example, ago(120) is "2 minutes later". For a negative value of seconds, this prints the same as "duration($seconds, [$precision]) . ' earlier'". For example, later(-120) is "2 minutes earlier". As a special case, later(0) returns "right then". later_exact($seconds) Same as later($seconds), except that the returned value is an exact (unrounded) expression of $seconds. earlier($seconds) earlier($seconds, $precision) earlier_exact($seconds) The same as later(-$seconds), later(-$seconds, $precision), later_exact(-$seconds). For example, earlier(120) is "2 minutes earlier". concise( function( ... ) ) Concise takes the string output of one of the above functions and makes it more concise. For example, "ago(4567)" returns "1 hour and 16 minutes ago", but "concise(ago(4567))" returns "1h16m ago". I18N/L10N NOTES Little of the internals of this module are English-specific. See source and/or contact me if you're interested in making a localized version for some other language than English. BACKSTORY
I wrote the basic "ago()" function for use in Infobot ("http://www.infobot.org"), because I was tired of this sort of response from the Purl Infobot: me> Purl, seen Woozle? <Purl> Woozle was last seen on #perl 20 days, 7 hours, 32 minutes and 40 seconds ago, saying: Wuzzle! I figured if it was 20 days ago, I don't care about the seconds. So once I had written "ago()", I abstracted the code a bit and got all the other functions. CAVEAT
This module calls a durational "year" an interval of exactly 365 days of exactly 24 hours each, with no provision for leap years or monkey business with 23/25 hour days (much less leap seconds!). But since the main work of this module is approximation, that shouldn't be a great problem for most purposes. SEE ALSO
Date::Interval, which is similarly named, but does something rather different. Star Trek: The Next Generation (1987-1994), where the character Data would express time durations like "1 year, 20 days, 22 hours, 59 minutes, and 35 seconds" instead of rounding to "1 year and 21 days". This is because no-one ever told him to use Time::Duration. COPYRIGHT AND DISCLAIMER
Copyright 2006, Sean M. Burke "sburke@cpan.org", all rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. This program is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. AUTHOR
Current maintainer Avi Finkel, "avi@finkel.org"; Original author Sean M. Burke, "sburke@cpan.org" perl v5.12.1 2007-08-19 Duration(3)
All times are GMT -4. The time now is 12:42 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy