Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: bash: How to split up a file based on another?
Top Forums Shell Programming and Scripting bash: How to split up a file based on another? Post 302563826 by radoulov on Wednesday 12th of October 2011 08:40:02 AM
Old 10-12-2011
Code:
awk 'NR == FNR {
  p || p = 1
  for (i = p; i < $2 + p; i++) {
    if (i > p) {
      nr[i]  = "FILE" c
      cmd[i] = "vgextend" FS $1 
      }
    else {  
      nr[i]  = "FILE" ++c
      cmd[i] = "vgcreate" FS $1
      }
    }    
  p = i; next    
  }
{ 
  print cmd[FNR], $0 > nr[FNR] 
  }' SPLIT_BY.txt  FILE.txt

You may hit the limit of open files with some awk implementations,
the below code keeps one file open for writing at a time.

Code:
awk 'NR == FNR {
  p || p = 1
  for (i = p; i < $2 + p; i++) {
    if (i > p) {
      nr[i]  = "FILE" c
      cmd[i] = "vgextend" FS $1 
      }
    else {  
      nr[i]  = "FILE" ++c
      cmd[i] = "vgcreate" FS $1
      }
    }    
  p = i; next    
  }
{ 
  print cmd[FNR], $0 > nr[FNR]
  t[nr[FNR]]++ || close(nr[FNR - 1])
  }' SPLIT_BY.txt  FILE.txt

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split a file based on a pattern

Dear all, I have a large file which is composed of 8000 frames, what i would like to do is split the file into 8000 single files names file.pdb.1, file.pdb.2 etc etc each frame in the large file is seperated by a "ENDMDL" flag so my thinking is to use this flag a a point to split the files... (4 Replies)
Discussion started by: Mish_99
4 Replies

2. Shell Programming and Scripting

Split file based on field

Hi I have a large file 2.6 million records and I am trying to split the file based on last column. I am doing awk -F"|" '{ print > $NF }' filename1 After around 1000 splits it gives me a error awk: can't open file 3332332423 input record number 1068, file filename1 source... (6 Replies)
Discussion started by: s_adu
6 Replies

3. Shell Programming and Scripting

Split the file based on date value

Hi frnds, I have flat file as . Say : output-file1.txt Output-file2.txt (1 Reply)
Discussion started by: Gopal_Engg
1 Replies

4. Shell Programming and Scripting

How to split file based on subtitle

Hi, unix Gurus, I want to split file based on sub_title. for example: original file fruit apple watermelon meat pork fish beef expected result file file1 fruit apple watermelon file2 meat pork fish beef. (4 Replies)
Discussion started by: ken002
4 Replies

5. Shell Programming and Scripting

Split file based on size

Hi Friends, Below is my requirement. I have a file with the below structure. 0001A1.... 0001B1.. .... 0001L1 0002A1 0002B1 ...... 0002L1 .. the first 4 characters are the sequence numbers for a record, A record will start with A1 and end with L1 with same sequence number. Now the... (2 Replies)
Discussion started by: diva_thilak
2 Replies

6. Shell Programming and Scripting

Split the file based on pattern

Hi , I have huge files around 400 mb, which has clob data and have diffeent scenarios: I am trying to pass scenario number as parameter and and get required modified file based on the scenario number and criteria. Scenario 1: file name : scenario_1.txt ... (2 Replies)
Discussion started by: sol_nov
2 Replies

7. Shell Programming and Scripting

Split file based on records

I have to split a file based on number of lines and the below command works fine: split -l 2 Inputfile -d OutputfileMy input file contains header, detail and trailor info as below: H D D D D TMy split files for the above command contains: First File: H DSecond File: ... (11 Replies)
Discussion started by: Ajay Venkatesan
11 Replies

8. Shell Programming and Scripting

Split File based on different conditions

I need to split the file Conditions: Ignore any record that either starts with 1 or 9 Split the file at position 404 , if position 404 is abc or def then write all the records in a file > File 1 , the remaining records should go in to a file > File 2 Further I want to split the... (7 Replies)
Discussion started by: protech
7 Replies

9. UNIX for Advanced & Expert Users

Split one file to many based on pattern

Hello All, I have records in a file in a pattern A,B,B,B,B,K,A,B,B,K Is there any command or simple logic I can pull out records into multiple files based on A record? I want output as File1: A,B,B,B,B,K File2: A,B,B,K (9 Replies)
Discussion started by: deal1dealer
9 Replies

10. Shell Programming and Scripting

Split the File based on Size

I have a file that is about 7 GB in size. The requirement is I should split the file equally in such a way that the size of the split files is less than 2Gb. If the file is less than 2gb, than nothing needs to be done. ( need to done using shell script) Thanks, (4 Replies)
Discussion started by: rudoraj
4 Replies

LEARN ABOUT CENTOS

gensprep

gensprep(8)							 ICU 50.1.2 Manual						       gensprep(8)

NAME

       gensprep - compile StringPrep data from files filtered by filterRFC3454.pl

SYNOPSIS

       gensprep [ -h, -?, --help ] [ -v, --verbose ] [ -c, --copyright ] [ -s, --sourcedir source ] [ -d, --destdir destination ]

DESCRIPTION

       gensprep reads filtered RFC 3454 files and compiles their information into a binary form.  The resulting file, <name>.icu, can then be read
       directly by ICU, or used by pkgdata(8) for incorporation into a larger archive or library.

       The files read by gensprep are described in the FILES section.

OPTIONS

       -h, -?, --help
	      Print help about usage and exit.

       -v, --verbose
	      Display extra informative messages during execution.

       -c, --copyright
	      Include a copyright notice into the binary data.

       -s, --sourcedir source
	      Set the source directory to source.  The default source directory is specified by the environment variable ICU_DATA.

       -d, --destdir destination
	      Set the destination directory to destination.  The default destination directory is specified by the environment variable ICU_DATA.

ENVIRONMENT

       ICU_DATA  Specifies the directory containing ICU data. Defaults to /usr/share/icu/50.1.2/.  Some tools in ICU depend on the presence of the
		 trailing slash. It is thus important to make sure that it is present if ICU_DATA is set.

FILES

       The  following files are read by gensprep and are looked for in the source /misc for rfc3454_*.txt files and in source /unidata for Normal-
       izationCorrections.txt.

       rfc3453_A_1.txt	   Contains the list of unassigned codepoints in Unicode version 3.2.0....

       rfc3454_B_1.txt	   Contains the list of code points that are commonly mapped to nothing....

       rfc3454_B_2.txt	   Contains the list of mappings for casefolding of  code points when Normalization form NFKC is specified....

       rfc3454_C_X.txt	   Contains the list of code points that are prohibited for IDNA.

       NormalizationCorrections.txt
			   Contains the list of code points whose normalization has changed since Unicode Version 3.2.0.

VERSION

       50.1.2

COPYRIGHT

       Copyright (C) 2000-2002 IBM, Inc. and others.

SEE ALSO

       pkgdata(8)

ICU MANPAGE
							   18 March 2003						       gensprep(8)
All times are GMT -4. The time now is 05:45 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy