Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: split files based on size
Top Forums UNIX for Dummies Questions & Answers split files based on size Post 302334555 by pludi on Thursday 16th of July 2009 01:41:33 AM
Old 07-16-2009
Luckily I had some large files around:
Code:
> find . -type f -size +5M -print 2>/dev/null | while read file; do
size=$( ls -l $file | awk '{print $5/2}' );
echo split -b $size $file ${file}_ ;
done
split -b 536870912 ./test/copytrunc/xdev/size1G.log ./test/copytrunc/xdev/size1G.log_
split -b 536870912 ./test/copy/local/size1G.log ./test/copy/local/size1G.log_
split -b 536870912 ./test/copy/xdev/size1G.log ./test/copy/xdev/size1G.log_
split -b 536870912 ./test/move/xdev/size1G.log ./test/move/xdev/size1G.log_

Will split the files in 2, naming the parts $file_aa, $file_ab
To disable the safety, remove the echo
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split file based on size

Hi Friends, Below is my requirement. I have a file with the below structure. 0001A1.... 0001B1.. .... 0001L1 0002A1 0002B1 ...... 0002L1 .. the first 4 characters are the sequence numbers for a record, A record will start with A1 and end with L1 with same sequence number. Now the... (2 Replies)
Discussion started by: diva_thilak
2 Replies

2. Shell Programming and Scripting

Split a file based on pattern and size

Hello, I have a large file (2GB) that I would like to split based on pattern and size. I've used the following command to split the file (token is "HELLO") awk '/HELLO/{i++}{print > "file"i}' input.txt and the output is similar to the following (i included filesize in KB): 10 ... (2 Replies)
Discussion started by: jl487
2 Replies

3. Shell Programming and Scripting

Split a file into multiple files based on field value

Hi, I've one requirement. I have to split one comma delimited file into multiple files based on one of the column values. How can I achieve this Unix Here is the sample data. In this case I have split the files based on date column(c4) Input file c1,c2,c3,c4,c5... (1 Reply)
Discussion started by: manasvi24
1 Replies

4. Shell Programming and Scripting

Split file based on file size in Korn script

I need to split a file if it is over 2GB in size (or any size), preferably split on the lines. I have figured out how to get the file size using awk, and I can split the file based on the number of lines (which I got with wc -l) but I can't figure out how to connect them together in the script. ... (6 Replies)
Discussion started by: ssemple2000
6 Replies

5. Shell Programming and Scripting

Split a file in more files based on score content

Dear All, I have the following file tabulated: ID distanceTSS score 8434 571269 10 10122 393912 9 7652 6 10 4863 1451 9 8419 39 2 9363 564 21 9333 7714 22 9638 8334 9 1638 1231 11 10701 918 1000 6587 32056 111 What I would like to do is the following, create 100 new files based... (5 Replies)
Discussion started by: paolo.kunder
5 Replies

6. Shell Programming and Scripting

Split Large Files Based On Row Pattern..

Hi all. I've tried searching the web but could not find similar problem to mine. I have one large file to be splitted into several files based on the matching pattern found in each row. For example, let's say the file content: ... (13 Replies)
Discussion started by: aimy
13 Replies

7. Shell Programming and Scripting

Split the File based on Size

I have a file that is about 7 GB in size. The requirement is I should split the file equally in such a way that the size of the split files is less than 2Gb. If the file is less than 2gb, than nothing needs to be done. ( need to done using shell script) Thanks, (4 Replies)
Discussion started by: rudoraj
4 Replies

8. UNIX for Dummies Questions & Answers

Split files based on user input

Hi All, I have a UNIX script which reads "PxyType" (read PxyType) as input from user and a file (eg : "File.json") with the list all PxyType's. Based on the user input for "PxyType" in "File.json", I want to redirect each matched line to a different file ("File1,2,3,..json"). Can you... (7 Replies)
Discussion started by: Deena1984
7 Replies

9. Shell Programming and Scripting

Split files based on row delimiter count

I have a huge file (around 4-5 GB containing 20 million rows) which has text like: <EOFD>11<EOFD>22<EORD>2<EOFD>2222<EOFD>3333<EORD>3<EOFD>44<EOFD>55<EORD>66<EOFD>888<EOFD>9999<EORD> Actually above is an extracted file from a Sql Server with each field delimited by <EOFD> and each row ends... (8 Replies)
Discussion started by: amvip
8 Replies

10. UNIX for Beginners Questions & Answers

File Size Split up based on Month

Hi, I have a directory in Unix and there are folders available in the directory. Files are created on different month and now i have a requirement to calculate size of the folder on month basis. Is there any Unix command to check this please?? Thanks (6 Replies)
Discussion started by: Nivas
6 Replies

LEARN ABOUT DEBIAN

qpid-python-test

qpid-python-test(1)						   User Commands					       qpid-python-test(1)

NAME

       qpid-python-test - run tests of the python QPID library for a broker

SYNOPSIS

       qpid-python-test [options] PATTERN ...

DESCRIPTION

       Run tests matching the specified PATTERNs.

OPTIONS

       -h, --help
	      show this help message and exit

       -l, --list
	      list tests instead of executing them

       -b BROKER, --broker=BROKER
	      run tests against BROKER (default localhost)

       -f FILE, --log-file=FILE
	      log output to FILE

       -v LEVEL, --log-level=LEVEL
	      only display log messages of LEVEL or higher severity: DEBUG, WARN, ERROR (default WARN)

       -c CATEGORY, --log-category=CATEGORY
	      log only categories matching CATEGORY pattern

       -m MODULES, --module=MODULES
	      add module to test search path

       -i IGNORE, --ignore=IGNORE
	      ignore tests matching IGNORE pattern

       -I IFILE, --ignore-file=IFILE
	      ignore tests matching patterns in IFILE

       -H, --halt-on-error
	      halt if an error is encountered

       -t, --time
	      report timing information on test run

       -D DEFINE, --define=DEFINE
	      define test parameters

SEE ALSO

       For more information on qpid-python-test please check the QPID wiki at http://qpid.apache.org.

Apache QPID							   October 2011 					       qpid-python-test(1)
All times are GMT -4. The time now is 07:03 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy