Split and Rename Split Files Post: 303015036

Sponsored Content

Top Forums UNIX for Beginners Questions & Answers Split and Rename Split Files Post 303015036 by Don Cragun on Monday 26th of March 2018 07:53:58 PM

03-26-2018

Registered User

Does the following help?

Code:

#!/usr/bin/ksh
# Usage: utlity_name Chunk_Size Filename...
# where:	Chunk_Size is the number of lines to write in each output file.
#
#		Filename is the filename (not pathname) of a file to be split
#		into files each of which contains no more than Chunk_Size lines
#		with a name that is a leading zero-filled 3 digit decimal
#		sequence number of the file being split followed by an
#		underscore follwed by Filename.

# Get the chunk size from the 1st operand.
ChunkSize=$1

# Discard the 1st command-line operand leaving just the filenames as the
# remaining operands.
shift 1

# Invoke awk to process the Filename operands given on the command line:
awk -v s="$ChunkSize" '				# Set s to the chunk size.
FNR == 1 {					# When we find the 1st line in
						# an input file...
	i = 0					#   reset the output file
						#   counter for this input file.
}
FNR%s == 1 {					# When we find the 1st input
						# line in a chunk size set of
						# lines in an input file...
	close(fn)				#   close the previous output
						#   file (if there was one)
						#   and...
	fn = sprintf("%03d_%s", ++i ,FILENAME)	#   set the output filename for
						#   the next output file.
}
{						# For every line read from an
						# input file...
	print > fn				#   write the input line into
						#   the current output file.
}' "$@"						# Mark end of awk script and
						# pass in the remaining
						# command-line arguments as the
						# names of the files to be
						# split.

This User Gave Thanks to Don Cragun For This Post:

Don Cragun

View Public Profile for Don Cragun

Find all posts by Don Cragun

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Split a file with no pattern -- Split, Csplit, Awk

I have gone through all the threads in the forum and tested out different things. I am trying to split a 3GB file into multiple files. Some files are even larger than this. For example: split -l 3000000 filename.txt This is very slow and it splits the file with 3 million records in each...

2. Shell Programming and Scripting

split and rename the file

Hi All, I have a requirement .I want to split a file and the split files should have certain names. Currently when i use the split command split -1000 testdata testdata_ Then the output is testdata_aa testdata_bb testdata_cc and so on. But i want the output as testdata1.snd...

3. Shell Programming and Scripting

awk split and rename files

I have a file test1.html like below: <dctm_topnav_en_US> <html> ..... </html> <dctm_topnav_en_CA> <html> ..... </html> <dctm_topnav_en_FR> <html> ..... </html> I need to use awk to split this into three file names like en_US.html , en_CA.html, en_FR.html each having content between...

4. UNIX for Dummies Questions & Answers

Split then rename

Hi everyone, I am trying to write an if statement that will split a file if it is over 1 million records/lines into files with say 900,000 records and then rename those files without the aaa, aab, aac format that splitting normally does and into a specific naming convention. For instance, if...

5. UNIX for Dummies Questions & Answers

Split and Rename files using Terminal and bin/bash

I have a file named Me_thread_spell.txt that I want to split into smaller files. I want it to be split in each place there is a ;;;. For example, blah blah blah ;;; blah bhlah hlabl awasnceuir asenduhfoijhacseiodnbfxasd;;; oabwcuhaweoir;;; This full file would be three separate files...

6. Shell Programming and Scripting

awk to split one field and print the last two fields within the split part.

Hello; I have a file consists of 4 columns separated by tab. The problem is the third fields. Some of the them are very long but can be split by the vertical bar "|". Also some of them do not contain the string "UniProt", but I could ignore it at this moment, and sort the file afterwards. Here is...

7. UNIX for Dummies Questions & Answers

Split and Rename multiple files

8. Shell Programming and Scripting

Split and rename files

Hello, Need to split files into n number of files and rename the files Example: Input: transaction.txt.1aa transaction.txt.1ab ...... Output: transaction.txt.1 transaction.txt.2 transaction.txt.3

9. Shell Programming and Scripting

awk : split file and rename and save in path according to content

Hello, I'm using Windows 7 ; sed, awk and gnuwin32 are installed. I have a big text file I need to manipulate. In short, I will have to split it in thousands of short files, then rename and save in a folder which name is based upon filename. Here is a snippet of my big input.txt file (this...

10. UNIX for Beginners Questions & Answers

Automate splitting of files , scp files as each split completes and combine files on target server

i use the split command to split a one terabyte backup file into 10 chunks of 100 GB each. The files are split one after the other. While the files is being split, I will like to scp the files one after the other as soon as the previous one completes, from server A to Server B. Then on server B ,...

LEARN ABOUT OPENSOLARIS

split

split(1)							   User Commands							  split(1)

NAME

       split - split a file into pieces

SYNOPSIS

       split [-linecount | -l linecount] [-a suffixlength]
	    [file [name]]

       split [-b n | nk | nm] [-a suffixlength] [file [name]]

DESCRIPTION

       The  split  utility reads file and writes it in linecount-line pieces into a set of output-files. The name of the first output-file is name
       with aa appended, and so on lexicographically, up to zz (a maximum of 676 files). The maximum length of name is 2 characters less than  the
       maximum	filename length allowed by the filesystem. See statvfs(2). If no output name is given, x is used as the default (output-files will
       be called xaa, xab, and so forth).

OPTIONS

       The following options are supported:

       -linecount | -l linecount

	   Number of lines in each piece. Defaults to 1000 lines.

       -a suffixlength

	   Uses suffixlength letters to form the suffix portion of the filenames of the split file. If -a is not  specified,  the  default  suffix
	   length  is  2. If the sum of the name operand and the suffixlength option-argument would create a filename exceeding NAME_MAX bytes, an
	   error will result; split will exit with a diagnostic message and no files will be created.

       -b n

	   Splits a file into pieces n bytes in size.

       -b nk

	   Splits a file into pieces n*1024 bytes in size.

       -b nm

	   Splits a file into pieces n*1048576 bytes in size.

OPERANDS

       The following operands are supported:

       file    The path name of the ordinary file to be split. If no input file is given or file is -, the standard input will be used.

       name    The prefix to be used for each of the files resulting from the split operation. If no name argument is given, x will be used as the
	       prefix  of  the	output	files.	The  combined  length of the basename of prefix and suffixlength cannot exceed NAME_MAX bytes. See
	       OPTIONS.

USAGE

       See largefile(5) for the description of the behavior of split when encountering files greater than or equal to 2 Gbyte ( 2^31 bytes).

ENVIRONMENT VARIABLES

       See environ(5) for descriptions of the following environment variables that affect the execution of split: LANG, LC_ALL, LC_CTYPE,  LC_MES-
       SAGES, and NLSPATH.

EXIT STATUS

       The following exit values are returned:

       0     Successful completion.

       >0    An error occurred.

ATTRIBUTES

       See attributes(5) for descriptions of the following attributes:

       +-----------------------------+-----------------------------+
       |      ATTRIBUTE TYPE	     |	    ATTRIBUTE VALUE	   |
       +-----------------------------+-----------------------------+
       |Availability		     |SUNWesu			   |
       +-----------------------------+-----------------------------+
       |CSI			     |Enabled			   |
       +-----------------------------+-----------------------------+
       |Interface Stability	     |Committed 		   |
       +-----------------------------+-----------------------------+
       |Standard		     |See  standards(5).	   |
       +-----------------------------+-----------------------------+

SEE ALSO

       csplit(1), statvfs(2), attributes(5), environ(5), largefile(5), standards(5)

SunOS 5.11							    16 Apr 1999 							  split(1)

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Split a file with no pattern -- Split, Csplit, Awk

Discussion started by: madhunk

2. Shell Programming and Scripting

split and rename the file

Discussion started by: dnat

3. Shell Programming and Scripting

awk split and rename files

Discussion started by: vijay52

4. UNIX for Dummies Questions & Answers

Split then rename

Discussion started by: coach5779