Sponsored Content
Top Forums Shell Programming and Scripting Split file based on file size in Korn script Post 302661661 by ssemple2000 on Monday 25th of June 2012 02:23:46 PM
Old 06-25-2012
Split file based on file size in Korn script

I need to split a file if it is over 2GB in size (or any size), preferably split on the lines. I have figured out how to get the file size using awk, and I can split the file based on the number of lines (which I got with wc -l) but I can't figure out how to connect them together in the script.

So the command
Code:
ls -l>mylist.txt

gives me the file listing in a file, and the command
Code:
awk < mylist.txt '{if ($5>75000000) print $5 " " $NF}'

gives me a list of all the files that are larger that the size with their sizes, and the command
Code:
wc -l myfile.txt

gives me the number of lines in the file (call it 50000), and if I manually put them together, the command
Code:
split -l 25000 myfile.txt myfile.txt

gives me two files, myfile.txtaa and myfile.txtab, each with 25000 lines.

The problem is how to get them together in one script....
Thank you.

Moderator's Comments:
Mod Comment Video tutorial on how to use code tags in The UNIX and Linux Forums.

Last edited by radoulov; 06-25-2012 at 06:35 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk script to split a file based on the condition

I have the file with the records like 4234234 US phone 3244234 US cup 2342342 CA phone 8947234 US phone 2389472 CA cup 2348972 US maps 3894234 CA phone I want the records with (US,phone) as record to be in one file, (Us, cup) in another file and (CA,cup) to be in another I mean all... (12 Replies)
Discussion started by: superprogrammer
12 Replies

2. Programming

create a spool file based on values passed from korn shell to sql script

this is my issue. 4 parameters are passed from korn shell to sql script. parameter_1= varchar2 datatype or no value entered my user. parameter_2= number datatype or no value entered my user. parameter_3= number datatype or no value entered my user. parameter_4= number datatype or no... (5 Replies)
Discussion started by: megha2525
5 Replies

3. Shell Programming and Scripting

Split file based on size

Hi Friends, Below is my requirement. I have a file with the below structure. 0001A1.... 0001B1.. .... 0001L1 0002A1 0002B1 ...... 0002L1 .. the first 4 characters are the sequence numbers for a record, A record will start with A1 and end with L1 with same sequence number. Now the... (2 Replies)
Discussion started by: diva_thilak
2 Replies

4. Shell Programming and Scripting

Split a file into multiple files based on first two digits of file.

Hi , I do have a fixedwidth flatfile that has data for 10 different datasets each identified by the first two digits in the flatfile. 01 in the first two digit position refers to Set A 02 in the first two digit position refers to Set B and so on I want to genrate 10 different files from my... (6 Replies)
Discussion started by: okkadu
6 Replies

5. Shell Programming and Scripting

Split a file based on pattern and size

Hello, I have a large file (2GB) that I would like to split based on pattern and size. I've used the following command to split the file (token is "HELLO") awk '/HELLO/{i++}{print > "file"i}' input.txt and the output is similar to the following (i included filesize in KB): 10 ... (2 Replies)
Discussion started by: jl487
2 Replies

6. Shell Programming and Scripting

How to split file into multiple files using awk based on 1 field in the file?

Good day all I need some helps, say that I have data like below, each field separated by a tab DATE NAME ADDRESS 15/7/2012 LX a.b.c 15/7/2012 LX1 a.b.c 16/7/2012 AB a.b.c 16/7/2012 AB2 a.b.c 15/7/2012 LX2 a.b.c... (2 Replies)
Discussion started by: alexyyw
2 Replies

7. Shell Programming and Scripting

awk script to split file into multiple files based on many columns

So I have a space delimited file that I'd like to split into multiple files based on multiple column values. This is what my data looks like 1bc9A02 1 10 1000 FTDLNLVQALRQFLWSFRLPGEAQKIDRMMEAFAQRYCQCNNGVFQSTDTCYVLSFAIIMLNTSLHNPNVKDKPTVERFIAMNRGINDGGDLPEELLRNLYESIKNEPFKIPELEHHHHHH 1ku1A02 1 10... (9 Replies)
Discussion started by: viored
9 Replies

8. Shell Programming and Scripting

Split the File based on Size

I have a file that is about 7 GB in size. The requirement is I should split the file equally in such a way that the size of the split files is less than 2Gb. If the file is less than 2gb, than nothing needs to be done. ( need to done using shell script) Thanks, (4 Replies)
Discussion started by: rudoraj
4 Replies

9. Answers to Frequently Asked Questions

How to split a dat file based on another file ni UNIX?

i have two files , one is var.txt and another res.dat file var.txt contains informaton like below date,request,sales,item 20171015,1,123456,216 20171015,1,123456,217 20171015,2,345678,214 20171015,3,456789,218 and res.dat contains is a one huge file contains information like... (1 Reply)
Discussion started by: pogo
1 Replies

10. UNIX for Beginners Questions & Answers

File Size Split up based on Month

Hi, I have a directory in Unix and there are folders available in the directory. Files are created on different month and now i have a requirement to calculate size of the folder on month basis. Is there any Unix command to check this please?? Thanks (6 Replies)
Discussion started by: Nivas
6 Replies
GAPPLETVIEWER(1)							GNU							  GAPPLETVIEWER(1)

NAME
gappletviewer - Load and runs an applet SYNOPSIS
appletviewer [OPTION]... URL... appletviewer [OPTION]... -code CODE appletviewer [OPTION]... -plugin INPUT,OUTPUT DESCRIPTION
The appletviewer tool loads and runs an applet. Use the first form to test applets specified by tag. The URL should resolve to an HTML document from which the appletviewer will extract applet tags. The APPLET, EMBED and OBJECT tags are supported. If a given document contains multiple applet tags, all the applets will be loaded, with each applet appearing in its own window. Likewise, when multiple URLs are specified, each applet tag instance is given its own window. If a given document contains no recognized tags the appletviewer does nothing. appletviewer http://www.gnu.org/software/classpath/ Use the second form to test an applet in development. This form allows applet tag attributes to be supplied on the command line. Only one applet may be specified using the -code option. The -code option overrides the URL form -- any URLs specified will be ignored. appletviewer -code Test.class -param datafile,data.txt gcjwebplugin uses the third form to communicate with the appletviewer through named pipes. OPTIONS
URL OPTIONS -debug This option is not yet implemented but is provided for compatibility. -encoding CHARSET Use this option to specify an alternate character encoding for the specified HTML page. APPLET TAG OPTIONS -code CODE Use the -code option to specify the value of the applet tag CODE attribute. -codebase CODEBASE Use the -codebase option to specify the value of the applet tag CODEBASE attribute. -archive ARCHIVE Use the -archive option to specify the value of the applet tag ARCHIVE attribute. -width WIDTH Use the -width option to specify the value of the applet tag WIDTH attribute. -height HEIGHT Use the -height option to specify the value of the applet tag HEIGHT attribute. -param NAME,VALUE Use the -param option to specify values for the NAME and VALUE attributes of an applet PARAM tag. PLUGIN OPTION -plugin INPUT,OUTPUT gcjwebplugin uses the -plugin option to specify the named pipe the appletviewer should use for receiving commands (INPUT) and the one it should use for sending commands to gcjwebplugin (OUTPUT). DEBUGGING OPTION -verbose Use the -verbose option to have the appletviewer print debugging messages. STANDARD OPTIONS -help Use the -help option to have the appletviewer print a usage message, then exit. -version Use the -version option to have the appletviewer print its version, then exit. -JOPTION Use the -J option to pass OPTION to the virtual machine that will run the appletviewer. Unlike other options, there must not be a space between the -J and OPTION. BUGS
SEE ALSO
AUTHOR
0.98 2010-07-05 GAPPLETVIEWER(1)
All times are GMT -4. The time now is 09:11 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy