07-12-2013
Hi Vidyadhar,
Thanks for the reply, this works but when I do not have the header or trailor record in the input file, even then it strips the first and the last line. I wanted to split the files based on the TAGS. Only for lines which starts with a substring say "D", i want to split the file.
I thought of the idea of doing it with temp file but since I am dealing with huge files upto 30GB, taking it to a temp will take more time.
Is there any other solution for this.
Thanks,
Ajay
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi,
I am new to scripting. I need a script to sort and the records in a file and then split them into different files.
For example, the file is:
H1......................
H2......................
D2....................
D2....................
H1........................... (15 Replies)
Discussion started by: Sunitha_edi82
15 Replies
2. UNIX for Dummies Questions & Answers
Hi all,
I would like to extract records of a file based on a condition. The file contains 47 fields, and I would like to extract only those records that match a certain value in one of the columns, e.g.
COL1 COL2 COL3 ............... COL47
1 XX 45 ... (4 Replies)
Discussion started by: risk_sly
4 Replies
3. Shell Programming and Scripting
Hi All,
I have two files say file1 and file2.
I want to check the number of records in file1 and if its atleast 2 (i.e., 2 or greater than 2 ) then I have to check records in file2 .If records in file2 is atleast 1 (i.e. if its not empty ) i have to set some conditions .
Could you pls... (3 Replies)
Discussion started by: mavesum
3 Replies
4. UNIX for Dummies Questions & Answers
Hello,
Each record has a lenght of 7 characters
I have 2 types of records 010 and 011
There is no character of end of line.
For example my file is like that :
010hello 010bonjour011both 011sisters
I would like to have 2 files
010.txt (2 records)
hello
bonjour
and
... (1 Reply)
Discussion started by: jeuffeu
1 Replies
5. Shell Programming and Scripting
Hi,
I need to execute parallel with while loop.
Input File(source_file.csv) contains filenames the below
source_file.csv file contains
Customer1.txt
Product1.txt
Sales.txt
Emp.txt
Dept.txt
Based on the number of rows that file I want to run the script ‘n' times.
while... (2 Replies)
Discussion started by: onesuri
2 Replies
6. UNIX for Advanced & Expert Users
HI,
I have a pipe delimiter file , I have to search for second field pattern, if the second field does not contain a '-' , I need to start capturing the record from this line till I find another second field with '-' value.
Below is the sample data
SOURCE DATA
ABC|ABC_702148-PARAM... (3 Replies)
Discussion started by: mora
3 Replies
7. Shell Programming and Scripting
Hi All,
I am trying to split a record into multiple records based on a value.
Input.txt
"A",1,0,10
"B",2,0,10,15,20
"C",3,11,14,16,19,21,23
"D",1,0,5
My desired output is:
"A",1,0,10
"B",2,0,10
"B",2,15,20
"C",3,11,14
"C",3,16,19
"C",3,21,23 (4 Replies)
Discussion started by: kmsekhar
4 Replies
8. Shell Programming and Scripting
Hello,
I want to filter records of a file if they fall in range associated with a second file. First the chr number (2nd col of 1st file and 1st col of 2nd file) needs to be matched. Then if the 3rd col of the first file falls within any of the ranges specified by the 2nd and 3rd cols , then... (4 Replies)
Discussion started by: ritakadm
4 Replies
9. Shell Programming and Scripting
Hello All,
I have a large file, more than 50,000 lines, and I want to split it in even 5000 records. Which I can do using
sed '1d;$d;' <filename> | awk 'NR%5000==1{x="F"++i;}{print > x}'Now I need to add one more condition that is not to break the file at 5000th record if the 5000th record... (20 Replies)
Discussion started by: ibmtech
20 Replies
10. Shell Programming and Scripting
Hi All,
I have one file containing thousands of table names in single column. Now I want that file split into multiple files e.g one file containing table names starting from A, other containing all tables starting from B...and so on..till Z.
I tried below but it did not work.
for i in... (6 Replies)
Discussion started by: shekhar_4_u
6 Replies
LEARN ABOUT FREEBSD
split
SPLIT(1) BSD General Commands Manual SPLIT(1)
NAME
split -- split a file into pieces
SYNOPSIS
split -d [-l line_count] [-a suffix_length] [file [prefix]]
split -d -b byte_count[K|k|M|m|G|g] [-a suffix_length] [file [prefix]]
split -d -n chunk_count [-a suffix_length] [file [prefix]]
split -d -p pattern [-a suffix_length] [file [prefix]]
DESCRIPTION
The split utility reads the given file and breaks it up into files of 1000 lines each (if no options are specified), leaving the file
unchanged. If file is a single dash ('-') or absent, split reads from the standard input.
The options are as follows:
-a suffix_length
Use suffix_length letters to form the suffix of the file name.
-b byte_count[K|k|M|m|G|g]
Create split files byte_count bytes in length. If k or K is appended to the number, the file is split into byte_count kilobyte
pieces. If m or M is appended to the number, the file is split into byte_count megabyte pieces. If g or G is appended to the num-
ber, the file is split into byte_count gigabyte pieces.
-d Use a numeric suffix instead of a alphabetic suffix.
-l line_count
Create split files line_count lines in length.
-n chunk_count
Split file into chunk_count smaller files.
-p pattern
The file is split whenever an input line matches pattern, which is interpreted as an extended regular expression. The matching line
will be the first line of the next output file. This option is incompatible with the -b and -l options.
If additional arguments are specified, the first is used as the name of the input file which is to be split. If a second additional argument
is specified, it is used as a prefix for the names of the files into which the file is split. In this case, each file into which the file is
split is named by the prefix followed by a lexically ordered suffix using suffix_length characters in the range ``a-z''. If -a is not speci-
fied, two letters are used as the suffix.
If the prefix argument is not specified, the file is split into lexically ordered files named with the prefix ``x'' and with suffixes as
above.
ENVIRONMENT
The LANG, LC_ALL, LC_CTYPE and LC_COLLATE environment variables affect the execution of split as described in environ(7).
EXIT STATUS
The split utility exits 0 on success, and >0 if an error occurs.
SEE ALSO
csplit(1), re_format(7)
STANDARDS
The split utility conforms to IEEE Std 1003.1-2001 (``POSIX.1'').
HISTORY
A split command appeared in Version 3 AT&T UNIX.
BUGS
The maximum line length for matching patterns is 65536.
BSD
May 9, 2013 BSD