Split the all files in to 8 parts in a folder Post: 303040866

Sponsored Content

Top Forums UNIX for Beginners Questions & Answers Split the all files in to 8 parts in a folder Post 303040866 by Raghuram717 on Friday 8th of November 2019 04:00:48 AM

11-08-2019

Registered User

Split the all files in to 8 parts in a folder

Hi,

I have different files and i need to split the files in that folder split in to 8 parts with equal number of lines....!
any fastest way of doing this in awk.

for an example i have a file called "BillingDetails_BaseFile.csv" with total line count 65536 and i need to split in to 8 parts with each file having 8192 (8192*8=65536). if 65538,last two lines can be ignored. i had a logic for this but its taking long time for this.
any fastest way in awk is appreciated.

my code:

Code:

cd /tmp/data/
linecount=$(wc -l BillingDetails_BaseFile.csv|cut -d" " -f1)
echo "linecount=$linecount"
b=$(expr  ${linecount} / ${Split_Count} )
echo "spliting lines per file will be ${b}"
split -l${b} -d BillingDetails_BaseFile.csv BillingDetails_
chmod 777 *

c=$(expr  ${Split_Count} - 1  )
for (( i=${c}; i>=0; i-- ))
do
  a=$({ printf "%02d\n" $i ; })
  b=$(expr  ${i} + 1  )
  b=$({ printf "%02d\n" $b ; })
  mv BillingDetails_$a BillingDetails_$b
done

any fastest way just specifying the folder name it should go each file and look for line count and split the 8 parts with equal number of lines.
Any help in awk is appreciated.

Raghuram717

View Public Profile for Raghuram717

Find all posts by Raghuram717

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split folder into dvd size

Hello everyone, I have a big folder in a linux server that contains dozen of big files (total folder size ~ 50 GB) I have a couple of files of 2.5 GB and some others from 100 MB to 1 GB. (that said it's obvious that it's impossible having two 2.5 GB files in one dvd) The purpose is to...

2. Shell Programming and Scripting

Using bash to separate files files based on parts of a filename

Hey guys, Sorry for the basic question but I have a lot of files that I want to separate into groups based on filenames which I can then cat together. Eg I have: (a_b_c.txt) WB34_2_SLA8.txt WB34_1_SLA8.txt WB34_1_DB10.txt WB34_2_DB10.txt WB34_1_SLA8.txt WB34_2_SLA8.txt 77_1_SLA8.txt...

3. Shell Programming and Scripting

Incrementing parts of ten digits number by parts

I have number in file which contains date and serial number: 2013101000. The last two digits are serial number (00). So maximum of serial number is 100. After reaching 100 it becomes 00 with incrementing 10 which is day with max 31. after reaching 31 it becomes 00 and increments 10...

4. Shell Programming and Scripting

Split file into n parts.

Hi all: I have a 5-column tab-separated file. The only thing that I want to do with it is to split it. However, I want to split it with a 80/20 proportion -- randomized, if possible. I know that something like : awk '{print $0 ""> "file" NR}' RS='' input-file will work, but it only...

5. UNIX for Dummies Questions & Answers

How To Split A File In Two Rar Parts?

I Am connected to Whatbox.ca Seed Box Via SSH!! i have a file named avicii.mp3. I Want to split it into two rar parts as Apart1.rar and Apart2.rar So That When i Download Both the parts to My PC And Extract Them They Come out As Whole Avicii.mp3. There is also one more problem!! When I Rar A...

6. UNIX for Dummies Questions & Answers

Split a file into parts only if the first field is different

Hi, I have a file like this: aaa 123 aaa 223 aaa 225 bbb 332 bbb 423 bbb 6755 bbb 324 ccc 112 ccc 234 ccc 897 Which I need to split into several files, something like split -l 3 but the way that the lines with the same names would only go into one file:

7. Shell Programming and Scripting

Split a folder with huge number of files in n folders

We have a folder XYZ with large number of files (>350,000). how can i split the folder and create say 10 of them XYZ1 to XYZ10 with 35,000 files each. (doesnt matter which files go where).

8. Shell Programming and Scripting

Split line in 4 parts

Hi Guys, I have file A.txt 1 2 3 4 5 6 7 8 9 10 11 Want Output :- 1 2 3

9. Shell Programming and Scripting

Shell scripting for moving folder specific files into target directory of that country folder.

I need help to write shell script to copy files from one server to another server. Source Directory UAE(inside i have another folder Misc with files inside UAE folder).I have to copy this to another server UAE folder( Files should be copied to UAE folder and Misc files should be copied in target...

10. Shell Programming and Scripting

Request for Shell script to move files from Subfolder to Parent folder and delete sub folder

Hi Team, I am new to shell script and there is a requirement where files should be moved from Subfolder to parent folder. Eg: parent folder --> /Interface/data/test/IN Sub folder -->/Interface/data/test/IN/Invoice20180607233338 Subfolder will be always with timestamp...

LEARN ABOUT OPENDARWIN

split

SPLIT(1)						    BSD General Commands Manual 						  SPLIT(1)

NAME

     split -- split a file into pieces

SYNOPSIS

     split [-a suffix_length] [-b byte_count[k|m]] [-l line_count] [-p pattern] [file [name]]

DESCRIPTION

     The split utility reads the given file and breaks it up into files of 1000 lines each.  If file is a single dash ('-') or absent, split reads
     from the standard input.

     The options are as follows:

     -a      Use suffix_length letters to form the suffix of the file name.

     -b      Create smaller files byte_count bytes in length.  If ``k'' is appended to the number, the file is split into byte_count kilobyte
	     pieces.  If ``m'' is appended to the number, the file is split into byte_count megabyte pieces.

     -l      Create smaller files n lines in length.

     -p pattern
	     The file is split whenever an input line matches pattern, which is interpreted as an extended regular expression.	The matching line
	     will be the first line of the next output file.  This option is incompatible with the -b and -l options.

     If additional arguments are specified, the first is used as the name of the input file which is to be split.  If a second additional argument
     is specified, it is used as a prefix for the names of the files into which the file is split.  In this case, each file into which the file is
     split is named by the prefix followed by a lexically ordered suffix using suffix_length characters in the range ``a-z''.  If -a is not speci-
     fied, two letters are used as the suffix.

     If the name argument is not specified, the file is split into lexically ordered files named with prefixes in the range of ``x-z'' and with
     suffixes as above.

SEE ALSO

     csplit(1), re_format(7)

STANDARDS

     The split utility conforms to IEEE Std 1003.1-2001 (``POSIX.1'').

HISTORY

     A split command appeared in Version 3 AT&T UNIX.

BUGS

     For historical reasons, if you specify name, split can only create 676 separate files.  The default naming convention allows 2028 separate
     files.  The -a option can be used to work around this limitation.

     The maximum line length for matching patterns is 65536.

BSD
								  April 16, 1994							       BSD

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split folder into dvd size

Discussion started by: cabrao

2. Shell Programming and Scripting

Using bash to separate files files based on parts of a filename

Discussion started by: Breentax

3. Shell Programming and Scripting

Incrementing parts of ten digits number by parts

Discussion started by: Natalie

4. Shell Programming and Scripting

Split file into n parts.

Discussion started by: owwow14