Split and name files as 15 minute periods


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Split and name files as 15 minute periods
# 8  
Old 06-23-2011
Quote:
Originally Posted by ksexton
This looks great. I do get an error when I run it:
> ./script.pl < test.csv
Month '-1' out of range 0..11 at ./script.pl line 13

If I change m-1 to be just m in line 13 I get an error with the day
> ./script.pl < test.csv
Day '' out of range 1..31 at ./script.pl line 13
Your input data's probably not as described. Please post a sample of it.
Quote:
I am actually ok with the first piece of code that uses the fixed size split. I can delete the first 'broken' row from each file. The effect will be minimal for what I need to do.
If y'say so. Your file names will have nothing to do with their contents unless by sheer coincidence.
# 9  
Old 06-23-2011
Ok...Here is some sample data

405038002621334|918602232979||59.161.255.135|59.161.255.67|INTERNET|402|40128|1|1|05/01/2011 00:02:30
405030003291388|919033259968||59.161.255.142|59.161.255.67|INTERNET|402|40128|1|2|05/01/2011 00:02:30
405030000889166|919033829018|1234567898|59.161.255.139|59.161.255.67|INTERNET|402|40128|1|0|05/01/2011 00:02:30
# 10  
Old 06-23-2011
No wonder, my script was expecting the date at the beginning of the line.

Code:
#!/usr/bin/perl
use Time::Local;
use POSIX qw(mktime strftime);

my $start=0, $end=0, $FMT="%Y%m%d.%H%M";

while($line=<STDIN>)
{
        # Split each line apart into the default $_ / @_ variable
        split("\\|",$line);
        # Last token is date.  Split apart date and time into two vars.
        ($mdy, $hms)=split(" ", @_[$#_]);
        #Split apart MM/DD/YYYY on /
        ($m, $d, $y)=split("/", $mdy);
        # Split apart HH:MM:SS on :
        ($hour, $min, $sec)=split(":", $hms);

        # Create a timestamp in epoch seconds from it.  $m-1 because Perl counts months 0-11
        $ldate=timelocal($sec, $min, $hour, $d, $m-1,$y);
        $min=$min-($min%15);    # chunks of 15 minutes

        # Check which date range it's in and change the output file as appropriate
        if($ldate >= $end)
        {
                $start=timelocal($sec,$min,$hour,$d,$m-1,$y);
                $end=$start + (15*60);
                $fname=sprintf("%s-%s.csv",
                        strftime("$FMT", localtime($start)),
                        strftime("$FMT", localtime($end)));

                FILE && close(FILE);
                open(FILE, ">$fname");
        }

        print FILE "$line";
}

FILE && close(FILE);
exit 0;

# 11  
Old 06-23-2011
Excellent.

Thanks
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Automate splitting of files , scp files as each split completes and combine files on target server

i use the split command to split a one terabyte backup file into 10 chunks of 100 GB each. The files are split one after the other. While the files is being split, I will like to scp the files one after the other as soon as the previous one completes, from server A to Server B. Then on server B ,... (2 Replies)
Discussion started by: malaika
2 Replies

2. UNIX for Beginners Questions & Answers

Split and Rename Split Files

Hello, I need to split a file by number of records and rename each split file with actual filename pre-pended with 3 digit split number. What I have tried is the below command with 2 digit numeric value split -l 3 -d abc.txt F (# Will Produce split Files as F00 F01 F02) How to produce... (19 Replies)
Discussion started by: techedipro
19 Replies

3. UNIX for Advanced & Expert Users

Find files modified in previous minute only

Hi, How can I get files which are modified only in last minute ? it should not display 2 minutes back filels -la -rw-rw-r-- 1 stuser st 51 Dec 3 09:22 a.csv -rw-rw-r-- 1 stiser st 50 Dec 3 09:25 b.csv -rw-rw-r-- 1 stuser st 53 Dec 3 09:33 c.csv When I run command at 9:34am then I... (7 Replies)
Discussion started by: sbjv
7 Replies

4. Post Here to Contact Site Administrators and Moderators

Copy files from ServerA to ServerB and sleep for minute

I will get files into ServerA and sleep for 1 minute and go on.. I need a script to copy files from ServerA to ServerB without missing files or copying half files then sleep for minute. For next minute new files has to copied and old files has to be removed from ServerB (1 Reply)
Discussion started by: Srilatha Punna
1 Replies

5. Shell Programming and Scripting

Periods turn into spaces for some reason

Hey all, I've come across a problem that I can't solve. Its as simple as it can get. I want to substitute spaces in a string with dots e.g. i am a string i.am.a.string I've tried examples from the web that should work but they don't. A period seems to become a space for some reason :s ... (5 Replies)
Discussion started by: basherlemon
5 Replies

6. Shell Programming and Scripting

Take minute per minute from a log awk

Hi, I've been trying to develop a script that performs the parsing of a log every 1 minute and then generating some statistics. I'm fairly new to programming and this is why I come to ask if I can lend a hand. this is my log: xxxx 16/04/2012 17:00:52 - xxxx714 - E234 - Time= 119 ms.... (8 Replies)
Discussion started by: jockx
8 Replies

7. UNIX for Advanced & Expert Users

get the file extension having multiple periods

How to get the file extension having multiple periods suppose i have a file samplewprk.txt.dat I need to retrieve txt.dat from the file. I worked all iam getting is either txt or pyd Could anyone help me in providing the solution? Thanks, in advance (3 Replies)
Discussion started by: chinku
3 Replies

8. UNIX for Dummies Questions & Answers

backslashing periods

Hello, I have a script which configures a system, the configuration is currently manual and error prone. I am writting a script, which currently uses hard-coded values. I don't know how to take an IP, e.g. 123.456.789.111, and backslash the periods, so I can pass it to an `exec perl... (0 Replies)
Discussion started by: Bloke
0 Replies

9. UNIX for Dummies Questions & Answers

split files into specified number of output files

Hi everyone, I have some large text files that I need to split into a specific number of files of equal size. As far as I know (and I don't really know that much :)) the split command only lets you specify the number of lines or bytes. The files are all of a different size, so the number of... (4 Replies)
Discussion started by: Migrainegirl
4 Replies
Login or Register to Ask a Question