Shell Programming and Scripting

I have log file whose size is 4 GB , i would like to split it to 1 gb each ,Can any one tell me the syntax of csplit comand for that.

I am using Sun0S 5.8

Based on your description, it would seem that split is more appropriate. Csplit splits a file based on context whereas split split it based on linecount or size.

> split -b 1024m <file> <split-name>

> split -b 1024m reallybigfile smallfile
should give you files smallfileaa, smallfileab, smallfileac and smallfilead

I just saw another post of yours that concerned a problem opening large files. Based on that, I'm going to assume that you will process the log file in these split pieces. If so, you need to split it based on line count.

You can use split or csplit. To determine the line count for the split files, first determine how many lines are in the logfile. You could form an expression to have the shell do the calculation and give you the final number but I won't bother with that here since I don't know which shell you use and you only asked for syntax.

> wc -l logfile
Whatever number is returned, divide by 4 and round up.

Let's say that logfile has 7607255 lines. Diving and rounding gives you approximately 1901814 lines per split-file.

Using split:
> split -1901814 biglogfile splitfile
will give you splitfileaa, splitfileab, splitfileac and splitfilead.


Using csplit:
> csplit -k -f csplitfile biglogfile 1901814 {2}
will give you csplitfile00, csplitfile01, csplitfile02 and csplitfile03.

In this case, I don't think the -k is really necessary but it doesn't hurt to include it just in case some error is encountered. On my system, the first file generated by csplit actually has 1 less line in it.


Whichever method you use, unless there is some anomaly with your log file, the file sizes should be pretty close to 1GB in size.

Thank you all for this Info. -- It is working.

I need some more info on rebuilding my Perl binary.
In my dev i am able to read a 4gb file using perl script --but same script failed in Prod.

When I am typing perl -V in development Env (SAME OS as Prod but different Perl build options and version ) it show -information like

Compile-time options:
USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES
also showing ----

uselargefiles=define
config_args=-Duse64bitint
--- ----

But when i am typing the same perl -V in production --the use LARGE FILE options is not showing ?

I know if i will rebuild with -DLARGE_FILE options or uselargefiles=define then i can open a file of 4 GB,But not sure how to set this parameter in perl and where to set ?

Aalso if i need to rebuild the perl binary -will the existing setup be disturbed ???
Any idea will help me.