Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Split a huge 7 GB File Based on Pattern into 4 files Post 302837131 by krishmaths on Thursday 25th of July 2013 07:31:45 AM
Old 07-25-2013
Try breaking down the problem by having a smaller file to test the command and then apply the logic to the actual file.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split a file based on a pattern

Dear all, I have a large file which is composed of 8000 frames, what i would like to do is split the file into 8000 single files names file.pdb.1, file.pdb.2 etc etc each frame in the large file is seperated by a "ENDMDL" flag so my thinking is to use this flag a a point to split the files... (4 Replies)
Discussion started by: Mish_99
4 Replies

2. Shell Programming and Scripting

Split a file into multiple files based on the input pattern

I have a file with lines something like. ...... 123_start ...... ....... 123_end .... ..... 456_start ...... ..... 456_end .... ..... 789_start .... .... 789_end (6 Replies)
Discussion started by: abinash
6 Replies

3. Shell Programming and Scripting

Help- counting delimiter in a huge file and split data into 2 files

I’m new to Linux script and not sure how to filter out bad records from huge flat files (over 1.3GB each). The delimiter is a semi colon “;” Here is the sample of 5 lines in the file: Name1;phone1;address1;city1;state1;zipcode1 Name2;phone2;address2;city2;state2;zipcode2;comment... (7 Replies)
Discussion started by: lv99
7 Replies

4. Shell Programming and Scripting

split XML file into multiple files based on pattern

Hello, I am using awk to split a file into multiple files using command: nawk '{ if ( $1 == "<process" ) { n=split($2, arr, "\""); file=arr } print > file }' processes.xml <process name="Process1.process"> ... (3 Replies)
Discussion started by: chiru_h
3 Replies

5. Shell Programming and Scripting

Split a file based on pattern and size

Hello, I have a large file (2GB) that I would like to split based on pattern and size. I've used the following command to split the file (token is "HELLO") awk '/HELLO/{i++}{print > "file"i}' input.txt and the output is similar to the following (i included filesize in KB): 10 ... (2 Replies)
Discussion started by: jl487
2 Replies

6. Shell Programming and Scripting

Split the file based on pattern

Hi , I have huge files around 400 mb, which has clob data and have diffeent scenarios: I am trying to pass scenario number as parameter and and get required modified file based on the scenario number and criteria. Scenario 1: file name : scenario_1.txt ... (2 Replies)
Discussion started by: sol_nov
2 Replies

7. Shell Programming and Scripting

Help needed - Split large file into smaller files based on pattern match

Help needed urgently please. I have a large file - a few hundred thousand lines. Sample CP START ACCOUNT 1234556 name 1 CP END ACCOUNT CP START ACCOUNT 2224444 name 1 CP END ACCOUNT CP START ACCOUNT 333344444 name 1 CP END ACCOUNT I need to split this file each time "CP START... (7 Replies)
Discussion started by: frustrated1
7 Replies

8. Shell Programming and Scripting

Split Large Files Based On Row Pattern..

Hi all. I've tried searching the web but could not find similar problem to mine. I have one large file to be splitted into several files based on the matching pattern found in each row. For example, let's say the file content: ... (13 Replies)
Discussion started by: aimy
13 Replies

9. Shell Programming and Scripting

How to split a file based on pattern line number?

Hi i have requirement like below M <form_name> sdasadasdMklkM D ...... D ..... M form_name> sdasadasdMklkM D ...... D ..... D ...... D ..... M form_name> sdasadasdMklkM D ...... M form_name> sdasadasdMklkM i want split file based on line number by finding... (10 Replies)
Discussion started by: bhaskar v
10 Replies

10. UNIX for Advanced & Expert Users

Split one file to many based on pattern

Hello All, I have records in a file in a pattern A,B,B,B,B,K,A,B,B,K Is there any command or simple logic I can pull out records into multiple files based on A record? I want output as File1: A,B,B,B,B,K File2: A,B,B,K (9 Replies)
Discussion started by: deal1dealer
9 Replies
APPLY(1)						    BSD General Commands Manual 						  APPLY(1)

NAME
apply -- apply a command to a set of arguments SYNOPSIS
apply [-a c] [-d] [-#] command argument ... DESCRIPTION
The apply utility runs the named command on each argument argument in turn. Character sequences of the form ``%d'' in command, where 'd' is a digit from 1 to 9, are replaced by the d'th following unused argument. In this case, the largest digit number of arguments are discarded for each execution of command. The options are as follows: -# Normally arguments are taken singly; the optional number -# specifies the number of arguments to be passed to command. If the number is zero, command is run, without arguments, once for each argument. If any sequences of ``%d'' occur in command, the -# option is ignored. -a c The use of the character '%' as a magic character may be changed with the -a option. -d Display the commands that would have been executed, but do not actually execute them. ENVIRONMENT
The following environment variable affects the execution of apply: SHELL Pathname of shell to use. If this variable is not defined, the Bourne shell is used. FILES
/bin/sh default shell EXAMPLES
apply echo * is similar to ls(1); apply -2 cmp a1 b1 a2 b2 a3 b3 compares the `a' files to the `b' files; apply -0 who 1 2 3 4 5 runs who(1) 5 times; and apply 'ln %1 /usr/joe' * links all files in the current directory to the directory /usr/joe. HISTORY
The apply command appeared in 4.2BSD. AUTHORS
Rob Pike BUGS
Shell metacharacters in command may have bizarre effects; it is best to enclose complicated commands in single quotes (''). The apply utility does not recognize multibyte characters. BSD
December 13, 2006 BSD
All times are GMT -4. The time now is 06:28 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy