Welcome to the forums. The following AWK one-liner will do what you ask. One file per month based on the date in the fifth pipe-delimited column. The filename will be in the form YYYYMM:
Should you prefer the files to be named with just the month, MM:
Regards,
Alister
---------- Post updated at 11:13 AM ---------- Previous update was at 10:55 AM ----------
Hi,
I need to split xml-files with sizes greater than 2 gb into smaler chunks. As I dont want to end up with billions of files, I want those splitted files to have configurable sizes like 250 MB. Each file should be well formed having an exact copy of the header (and footer as the closing of the... (0 Replies)
hi all
i have a some huge html files (500MB to 1GB). Each file has multiple
<html></html> tags
<html>
.................
....................
....................
</html>
<html>
.................
....................
....................
</html>
<html>
.................... (5 Replies)
Hi ,
i have files coming in my system which are very huge in MB and GBs, all these files are in a single line, there is no newline character.
I need to get only last 700 bytes of these files, of this i am splitting the files by "split -b 700 filename" but this gives all the splitted... (2 Replies)
Hi,
I have an input file like:
111
abcdefgh
asdfghjk
dfghjkl
222
aaaaaaa
bbbbbb
333
djfhfgjktitjhgfkg
444
djdhfjkhfjkghjkfg
hsbfjksdbhjkgherjklg
fjkhfjklsahjgh
fkrjkgnj
I want to read this input file and make separate output files with the header as numric value like "111"... (9 Replies)
Hi
extending to one of my previous posted query ....
I am using
nawk -v invar1="$aa" '{print > ("ABS\_"((/\|/)?"A\_":"B\_")invar1"\_NETWORKID.txt")}' spfile.txt
to get 2 different files based on split condition i.e. "|"
Similar to invar1 variable in nawk I also need one more variable... (18 Replies)
I’m new to Linux script and not sure how to filter out bad records from huge flat files (over 1.3GB each). The delimiter is a semi colon “;”
Here is the sample of 5 lines in the file:
Name1;phone1;address1;city1;state1;zipcode1
Name2;phone2;address2;city2;state2;zipcode2;comment... (7 Replies)
Hi,
I have a huge file with a single line.
But I want to break that line into lines of with each line having five columns.
My file is like this:
code:
"hi","there","how","are","you?","It","was","great","working","with","you.","hope","to","work","you."
I want it like this:
code:... (1 Reply)
I was given a data file that I need to split into multiple lines/records based on a key word. The problem is that it is 2.5GB or bigger and everything I try in perl or sed causes a Segmentation fault. Can someone give me some other ideas.
The data is of the form:... (5 Replies)
Hi all,
I hope you are well. I am very happy to see your contribution. I am eager to become part of it.
I have the following question. I have two huge files to compare (almost 3GB each). The files are simulation outputs. The format of the files are as below
For clear picture, please see... (9 Replies)
Hi,
I have a Huge 7 GB file which has around 1 million records, i want to split this file into 4 files to contain around 250k messages each.
Please help me as Split command cannot work here as it might miss tags..
Format of the file is as below
<!--###### ###### START-->... (6 Replies)
Discussion started by: KishM
6 Replies
LEARN ABOUT OPENDARWIN
split
SPLIT(1) BSD General Commands Manual SPLIT(1)NAME
split -- split a file into pieces
SYNOPSIS
split [-a suffix_length] [-b byte_count[k|m]] [-l line_count] [-p pattern] [file [name]]
DESCRIPTION
The split utility reads the given file and breaks it up into files of 1000 lines each. If file is a single dash ('-') or absent, split reads
from the standard input.
The options are as follows:
-a Use suffix_length letters to form the suffix of the file name.
-b Create smaller files byte_count bytes in length. If ``k'' is appended to the number, the file is split into byte_count kilobyte
pieces. If ``m'' is appended to the number, the file is split into byte_count megabyte pieces.
-l Create smaller files n lines in length.
-p pattern
The file is split whenever an input line matches pattern, which is interpreted as an extended regular expression. The matching line
will be the first line of the next output file. This option is incompatible with the -b and -l options.
If additional arguments are specified, the first is used as the name of the input file which is to be split. If a second additional argument
is specified, it is used as a prefix for the names of the files into which the file is split. In this case, each file into which the file is
split is named by the prefix followed by a lexically ordered suffix using suffix_length characters in the range ``a-z''. If -a is not speci-
fied, two letters are used as the suffix.
If the name argument is not specified, the file is split into lexically ordered files named with prefixes in the range of ``x-z'' and with
suffixes as above.
SEE ALSO csplit(1), re_format(7)STANDARDS
The split utility conforms to IEEE Std 1003.1-2001 (``POSIX.1'').
HISTORY
A split command appeared in Version 3 AT&T UNIX.
BUGS
For historical reasons, if you specify name, split can only create 676 separate files. The default naming convention allows 2028 separate
files. The -a option can be used to work around this limitation.
The maximum line length for matching patterns is 65536.
BSD April 16, 1994 BSD