There was a minor error in my code which caused the MAXLINES variable to be a string and not a number.
Unfortunately, your input data was so perfectly matched to my bug that it worked anyway, so I didn't notice!
All your code needs is two keystrokes +0. I'd also add a bit of error checking.
Code:
#!/bin/sh
if [ "$#" -lt 5 ] || [ ! -f "$1" ] || [ ! -d "$3" ]
then
echo "usage: $0 inputfile numfiles opdir fileext region" >&2
exit 1
fi
awk -v NFILES="$2" -v FNAME="$3/$5_%d.$4" '
# Do not print the first file -- just count lines
NR==FNR { next }
# First line of the second read through the file.
FNR==1 { HEADER=$0
MAXLINES=sprintf("%d", (NR-1)/NFILES)+0;
LINES=MAXLINES
next }
# skip to the next file and print header if exceeded maxlines
(LINES >= MAXLINES) {
LINES=0; FILE++;
print HEADER > sprintf(FNAME,FILE); }
# Print all lines into the current file
{ print > sprintf(FNAME, FILE); LINES++ }
# Yes, we give awk the same file twice. On the first read, it just counts
# lines. On the second, it decides which lines go into what file.
' "$1" "$1"
This time I tested it all-out, making files with 150,000 lines, 100,009 lines, etc, etc. It splits as you wanted.
Hello..
Iam in need to urgent help with the below.
Have data-file with 40,567
and need to split them into multiple files with smaller line-count.
Iam aware of "split" command with -l option which allows you to specify the no of lines in smaller files ,with the target file-name pattern... (1 Reply)
hi all
im new to this forum..excuse me if anythng wrong.
I have a file containing 600 MB data in that. when i do parse the data in perl program im getting out of memory error.
so iam planning to split the file into smaller files and process one by one.
can any one tell me what is the code... (1 Reply)
Hello
We have a text file with 400,000 lines and need to split into multiple files each with 5000 lines ( will result in 80 files)
Got an idea of using head and tail commands to do that with a loop but looked not efficient.
Please advise the simple and yet effective way to do it.
TIA... (3 Replies)
Hi
I have a big verilog file with multiple modules. Each module begin with the code word 'module <module-name>(ports,...)'
and end with the
'endmodule' keyword.
Could you please suggest the best way to split each of these modules into multiple files?
Thank you for the help.
Example of... (7 Replies)
Hi Everyone,
I am using a centos 5.2 server as an sflow log collector on my network. Currently I am using inmons free sflowtool to collect the packets sent by my switches. I have a bash script running on an infinate loop to stop and start the log collection at set intervals - currently one... (2 Replies)
Hi,
I have a big text file with m columns and n rows. The format is like:
STF123450001000200030004STF123450005000600070008STF123450009001000110012
STF234560345002208330154STF234590705620600070080STF234567804094562357688
STF356780001000200030004STF356780005000600070080STF356780800094562657687... (2 Replies)
I will simplify the explaination a bit, I need to parse through a 87m file -
I have a single text file in the form of :
<NAME>house........
SOMETEXT
SOMETEXT
SOMETEXT
.
.
.
.
</script>
MORETEXT
MORETEXT
.
.
. (6 Replies)
Hi All,
I am new to Scripting language.
I want to split a file and create several subfiles using Perl script.
Example :
File format :
Sourcename ID Date Nbr
SU IMYFDJ 9/17/2012 5552159976555
SU BWZMIG 9/14/2012 1952257857887
AR PEHQDF 11/26/2012 ... (13 Replies)
Hi Everyone,
I'm new here and I was checking this old post:
/shell-programming-and-scripting/180669-splitting-file-into-several-smaller-files-using-perl.html
(cannot paste link because of lack of points)
I need to do something like this but understand very little of perl.
I also check... (4 Replies)
Hello,
I have some large text files that look like,
putrescine
Mrv1583 01041713302D
6 5 0 0 0 0 999 V2000
2.0928 -0.2063 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0
5.6650 0.2063 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0
3.5217 ... (3 Replies)
Discussion started by: LMHmedchem
3 Replies
LEARN ABOUT HPUX
split
split(1) General Commands Manual split(1)NAME
split - split a file into pieces
SYNOPSIS
line_count] suffix_length] [file [name]]
nsuffix_length] [file [name]]
Obsolescent
n] [file [name]]
DESCRIPTION
reads file and writes it in pieces (default 1000 lines) onto a set of output files. The name of the first output file is name with
appended, and so on lexicographically, up to (only ASCII letters are used, a maximum of 676 files). If no output name is given, is the
default.
If no input file is given, or if is given instead, the standard input file is used.
Options
recognizes the following command-line options and arguments:
The input file is split into pieces
line_count lines in size.
suffix_length letters are used to form the suffix of the output filenames. This option allows creation of more than 676 output
files. The output file names created cannot exceed the maximum file name length allowed in the directory containing
the files.
The input file is split into pieces
n bytes in size.
The input file is split into pieces
n x 1024 bytes in size. No space separates the n from the
The input file is split into pieces
n x 1048576 bytes in size. No space separates the n from the
The input file is split into pieces
n lines in size. This option is obsolescent and is equivalent to using the option.
EXTERNAL INFLUENCES
Environment Variables
determines the locale for the interpretation of text as single- and/or multi-byte characters.
determines the language in which messages are displayed.
If or is not specified in the environment or is set to the empty string, the value of is used as a default for each unspecified or empty
variable. If is not specified or is set to the empty string, a default of "C" (see lang(5)) is used instead of
If any internationalization variable contains an invalid setting, behaves as if all internationalization variables are set to "C". See
environ(5).
International Code Set Support
Single- and multi-byte character code sets are supported.
SEE ALSO csplit(1).
STANDARDS CONFORMANCE split(1)
03-30-2012
