Sponsored Content
Full Discussion: Split file using awk
Top Forums Shell Programming and Scripting Split file using awk Post 53641 by pvar on Wednesday 21st of July 2004 09:16:50 AM
Old 07-21-2004
zazzybob's script works fine now.

Ygor's script still have an issue. It creates seperate output files now but it puts lines starting with DOC & END also in the output files. I just need the data between those lines.

Thanks agin for the help.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Split a file with no pattern -- Split, Csplit, Awk

I have gone through all the threads in the forum and tested out different things. I am trying to split a 3GB file into multiple files. Some files are even larger than this. For example: split -l 3000000 filename.txt This is very slow and it splits the file with 3 million records in each... (10 Replies)
Discussion started by: madhunk
10 Replies

2. Shell Programming and Scripting

split file with awk

I did a lot of search on this forum on spiting file; found a lot, but my requirement is a bit different, please guide. Master file: x:start:5 line1:23 line2:12 2:90 x:end:5 x:start:2 45:56 22:90 x:end:2 x:start:3 line1:23 line2:12 x:end:3 x:start:2 line5:23 (1 Reply)
Discussion started by: uwork72
1 Replies

3. Shell Programming and Scripting

awk - split file

How can I split a text file (in awk) in n others with number of record given in input? Thanks (6 Replies)
Discussion started by: pinguc
6 Replies

4. Shell Programming and Scripting

split a file using awk

Hi , I just need to split a file and outputfiles are redirected to gzip file need: Input file - A.gz content of A.gz is 100|sfdds|dffdds|200112|sdfdf 100|sfdds|dffdds|200112|sdfdf 100|sfdds|dffdds|200112|sdfdf 100|sfdds|dffdds|200212|sdfdf 100|sfdds|dffdds|200212|sdfdf... (3 Replies)
Discussion started by: mohan_xunil
3 Replies

5. Shell Programming and Scripting

How to split a file using AWK?

Hello, I have a file like the following: david,a,b,c,20,r thomas,a,b,c,30,r willaiam,a,b,c,80,r barbara,a,b,c,100,r I would like to split the file into other files using a condition for the contents of column 5. The condition should be a if the contents of column 5 is in a range... (4 Replies)
Discussion started by: keenboy100
4 Replies

6. Shell Programming and Scripting

AWK File Split

Hi All, Input.txt XYZONEABC                  CZXTWOJJJ KKKSIXOOO asdfhajlsdhfajs asdfasfasdf Output Files: ONE.txt XYZONEABC                 TWO.txt CZXTWOJJJ SIX.txt KKKSIXOOO I had a script (2 Replies)
Discussion started by: kmsekhar
2 Replies

7. Shell Programming and Scripting

Split a file with awk

Hi! I have a file like this: a,b,c,12,d,e a,b,c,13,d,e a,b,c,14,d,e a,b,c,15,d,e a,b,c,16,d,e a,b,c,17,d,e I need to split that file in two: If field 4 is equal or higher than 14 that row goes to one file and if it is equal or higher than 15 to another. Can anyone point me in the... (2 Replies)
Discussion started by: Tr0cken
2 Replies

8. Shell Programming and Scripting

Split File by Pattern with File Names in Source File... Awk?

Hi all, I'm pretty new to Shell scripting and I need some help to split a source text file into multiple files. The source has a row with pattern where the file needs to be split, and the pattern row also contains the file name of the destination for that specific piece. Here is an example: ... (2 Replies)
Discussion started by: cul8er
2 Replies

9. Shell Programming and Scripting

awk file split

Hi all, First of all I' like to mention that I'm pretty new to unix scripting. :( I'm trying to split an large xml with awk and rename it based on the values of two attributes. Example XML <RECORD> <element1>11</element1> <element2>22</element2> <element3>33</element3>... (18 Replies)
Discussion started by: f0usk4s
18 Replies

10. UNIX for Beginners Questions & Answers

Split file using awk

I need to split the incoming source file in to multiple files using awk. Split position is (6,13) : 8 positions All the records that are greater than 20170101 and less than or equal to 20181231 should go in a split file with file name as source... (11 Replies)
Discussion started by: rosebud123
11 Replies
PT-FIFO-SPLIT(1p)					User Contributed Perl Documentation					 PT-FIFO-SPLIT(1p)

NAME
pt-fifo-split - Split files and pipe lines to a fifo without really splitting. SYNOPSIS
Usage: pt-fifo-split [options] [FILE ...] pt-fifo-split splits FILE and pipes lines to a fifo. With no FILE, or when FILE is -, read standard input. Read hugefile.txt in chunks of a million lines without physically splitting it: pt-fifo-split --lines 1000000 hugefile.txt while [ -e /tmp/pt-fifo-split ]; do cat /tmp/pt-fifo-split; done RISKS
The following section is included to inform users about the potential risks, whether known or unknown, of using this tool. The two main categories of risks are those created by the nature of the tool (e.g. read-only tools vs. read-write tools) and those created by bugs. pt-fifo-split creates and/or deletes the "--fifo" file. Otherwise, no other files are modified, and it merely reads lines from the file given on the command-line. It should be very low-risk. At the time of this release, we know of no bugs that could cause serious harm to users. The authoritative source for updated information is always the online issue tracking system. Issues that affect this tool will be marked as such. You can see a list of such issues at the following URL: http://www.percona.com/bugs/pt-fifo-split <http://www.percona.com/bugs/pt-fifo-split>. See also "BUGS" for more information on filing bugs and getting help. DESCRIPTION
pt-fifo-split lets you read from a file as though it contains only some of the lines in the file. When you read from it again, it contains the next set of lines; when you have gone all the way through it, the file disappears. This works only on Unix-like operating systems. You can specify multiple files on the command line. If you don't specify any, or if you use the special filename "-", lines are read from standard input. OPTIONS
This tool accepts additional command-line arguments. Refer to the "SYNOPSIS" and usage information for details. --config type: Array Read this comma-separated list of config files; if specified, this must be the first option on the command line. --fifo type: string; default: /tmp/pt-fifo-split The name of the fifo from which the lines can be read. --force Remove the fifo if it exists already, then create it again. --help Show help and exit. --lines type: int; default: 1000 The number of lines to read in each chunk. --offset type: int; default: 0 Begin at the Nth line. If the argument is 0, all lines are printed to the fifo. If 1, then beginning at the first line, lines are printed (exactly the same as 0). If 2, the first line is skipped, and the 2nd and subsequent lines are printed to the fifo. --pid type: string Create the given PID file. The file contains the process ID of the script. The PID file is removed when the script exits. Before starting, the script checks if the PID file already exists. If it does not, then the script creates and writes its own PID to it. If it does, then the script checks the following: if the file contains a PID and a process is running with that PID, then the script dies; or, if there is no process running with that PID, then the script overwrites the file with its own PID and starts; else, if the file contains no PID, then the script dies. --statistics Print out statistics between chunks. The statistics are the number of chunks, the number of lines, elapsed time, and lines per second overall and during the last chunk. --version Show version and exit. ENVIRONMENT
The environment variable "PTDEBUG" enables verbose debugging output to STDERR. To enable debugging and capture all output to a file, run the tool like: PTDEBUG=1 pt-fifo-split ... > FILE 2>&1 Be careful: debugging output is voluminous and can generate several megabytes of output. SYSTEM REQUIREMENTS
You need Perl, DBI, DBD::mysql, and some core packages that ought to be installed in any reasonably new version of Perl. BUGS
For a list of known bugs, see http://www.percona.com/bugs/pt-fifo-split <http://www.percona.com/bugs/pt-fifo-split>. Please report bugs at https://bugs.launchpad.net/percona-toolkit <https://bugs.launchpad.net/percona-toolkit>. Include the following information in your bug report: o Complete command-line used to run the tool o Tool "--version" o MySQL version of all servers involved o Output from the tool including STDERR o Input files (log/dump/config files, etc.) If possible, include debugging output by running the tool with "PTDEBUG"; see "ENVIRONMENT". DOWNLOADING
Visit http://www.percona.com/software/percona-toolkit/ <http://www.percona.com/software/percona-toolkit/> to download the latest release of Percona Toolkit. Or, get the latest release from the command line: wget percona.com/get/percona-toolkit.tar.gz wget percona.com/get/percona-toolkit.rpm wget percona.com/get/percona-toolkit.deb You can also get individual tools from the latest release: wget percona.com/get/TOOL Replace "TOOL" with the name of any tool. AUTHORS
Baron Schwartz ABOUT PERCONA TOOLKIT
This tool is part of Percona Toolkit, a collection of advanced command-line tools developed by Percona for MySQL support and consulting. Percona Toolkit was forked from two projects in June, 2011: Maatkit and Aspersa. Those projects were created by Baron Schwartz and developed primarily by him and Daniel Nichter, both of whom are employed by Percona. Visit <http://www.percona.com/software/> for more software developed by Percona. COPYRIGHT, LICENSE, AND WARRANTY This program is copyright 2007-2011 Baron Schwartz, 2011-2012 Percona Inc. Feedback and improvements are welcome. THIS PROGRAM IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, version 2; OR the Perl Artistic License. On UNIX and similar systems, you can issue `man perlgpl' or `man perlartistic' to read these licenses. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA. VERSION
pt-fifo-split 2.1.2 perl v5.14.2 2012-06-15 PT-FIFO-SPLIT(1p)
All times are GMT -4. The time now is 08:10 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy