There was a minor error in my code which caused the MAXLINES variable to be a string and not a number.
Unfortunately, your input data was so perfectly matched to my bug that it worked anyway, so I didn't notice!
All your code needs is two keystrokes +0. I'd also add a bit of error checking.
Code:
#!/bin/sh
if [ "$#" -lt 5 ] || [ ! -f "$1" ] || [ ! -d "$3" ]
then
echo "usage: $0 inputfile numfiles opdir fileext region" >&2
exit 1
fi
awk -v NFILES="$2" -v FNAME="$3/$5_%d.$4" '
# Do not print the first file -- just count lines
NR==FNR { next }
# First line of the second read through the file.
FNR==1 { HEADER=$0
MAXLINES=sprintf("%d", (NR-1)/NFILES)+0;
LINES=MAXLINES
next }
# skip to the next file and print header if exceeded maxlines
(LINES >= MAXLINES) {
LINES=0; FILE++;
print HEADER > sprintf(FNAME,FILE); }
# Print all lines into the current file
{ print > sprintf(FNAME, FILE); LINES++ }
# Yes, we give awk the same file twice. On the first read, it just counts
# lines. On the second, it decides which lines go into what file.
' "$1" "$1"
This time I tested it all-out, making files with 150,000 lines, 100,009 lines, etc, etc. It splits as you wanted.
Hello..
Iam in need to urgent help with the below.
Have data-file with 40,567
and need to split them into multiple files with smaller line-count.
Iam aware of "split" command with -l option which allows you to specify the no of lines in smaller files ,with the target file-name pattern... (1 Reply)
hi all
im new to this forum..excuse me if anythng wrong.
I have a file containing 600 MB data in that. when i do parse the data in perl program im getting out of memory error.
so iam planning to split the file into smaller files and process one by one.
can any one tell me what is the code... (1 Reply)
Hello
We have a text file with 400,000 lines and need to split into multiple files each with 5000 lines ( will result in 80 files)
Got an idea of using head and tail commands to do that with a loop but looked not efficient.
Please advise the simple and yet effective way to do it.
TIA... (3 Replies)
Hi
I have a big verilog file with multiple modules. Each module begin with the code word 'module <module-name>(ports,...)'
and end with the
'endmodule' keyword.
Could you please suggest the best way to split each of these modules into multiple files?
Thank you for the help.
Example of... (7 Replies)
Hi Everyone,
I am using a centos 5.2 server as an sflow log collector on my network. Currently I am using inmons free sflowtool to collect the packets sent by my switches. I have a bash script running on an infinate loop to stop and start the log collection at set intervals - currently one... (2 Replies)
Hi,
I have a big text file with m columns and n rows. The format is like:
STF123450001000200030004STF123450005000600070008STF123450009001000110012
STF234560345002208330154STF234590705620600070080STF234567804094562357688
STF356780001000200030004STF356780005000600070080STF356780800094562657687... (2 Replies)
I will simplify the explaination a bit, I need to parse through a 87m file -
I have a single text file in the form of :
<NAME>house........
SOMETEXT
SOMETEXT
SOMETEXT
.
.
.
.
</script>
MORETEXT
MORETEXT
.
.
. (6 Replies)
Hi All,
I am new to Scripting language.
I want to split a file and create several subfiles using Perl script.
Example :
File format :
Sourcename ID Date Nbr
SU IMYFDJ 9/17/2012 5552159976555
SU BWZMIG 9/14/2012 1952257857887
AR PEHQDF 11/26/2012 ... (13 Replies)
Hi Everyone,
I'm new here and I was checking this old post:
/shell-programming-and-scripting/180669-splitting-file-into-several-smaller-files-using-perl.html
(cannot paste link because of lack of points)
I need to do something like this but understand very little of perl.
I also check... (4 Replies)
Hello,
I have some large text files that look like,
putrescine
Mrv1583 01041713302D
6 5 0 0 0 0 999 V2000
2.0928 -0.2063 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0
5.6650 0.2063 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0
3.5217 ... (3 Replies)
Discussion started by: LMHmedchem
3 Replies
LEARN ABOUT SUSE
head
HEAD(1) User Commands HEAD(1)NAME
head - output the first part of files
SYNOPSIS
head [OPTION]... [FILE]...
DESCRIPTION
Print the first 10 lines of each FILE to standard output. With more than one FILE, precede each with a header giving the file name. With
no FILE, or when FILE is -, read standard input.
Mandatory arguments to long options are mandatory for short options too.
-c, --bytes=[-]N
print the first N bytes of each file; with the leading `-', print all but the last N bytes of each file
-n, --lines=[-]N
print the first N lines instead of the first 10; with the leading `-', print all but the last N lines of each file
-q, --quiet, --silent
never print headers giving file names
-v, --verbose
always print headers giving file names
--help display this help and exit
--version
output version information and exit
N may have a multiplier suffix: b 512, kB 1000, K 1024, MB 1000*1000, M 1024*1024, GB 1000*1000*1000, G 1024*1024*1024, and so on for T, P,
E, Z, Y.
AUTHOR
Written by David MacKenzie and Jim Meyering.
REPORTING BUGS
Report head bugs to bug-coreutils@gnu.org
GNU coreutils home page: <http://www.gnu.org/software/coreutils/>
General help using GNU software: <http://www.gnu.org/gethelp/>
COPYRIGHT
Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
SEE ALSO
The full documentation for head is maintained as a Texinfo manual. If the info and head programs are properly installed at your site, the
command
info coreutils 'head invocation'
should give you access to the complete manual.
GNU coreutils 7.1 July 2010 HEAD(1)
03-30-2012
