Sponsored Content
Top Forums Shell Programming and Scripting Breaking large file into small files Post 302937447 by gandolf989 on Thursday 5th of March 2015 12:46:05 PM
Old 03-05-2015
Quote:
Originally Posted by RavinderSingh13
Hello emily/gandolf989,

If we need to split a file according to lines let's say 10'000 lines per file then following may help.
Code:
awk '{FILENAME="file"int((NR-1)/10000);print >> FILENAME}' Input_file

Thanks,
R. Singh
There seems to be a magic awk command for almost every problem.

---------- Post updated at 12:46 PM ---------- Previous update was at 12:44 PM ----------

Quote:
Originally Posted by rbatte1
Well, gandolf989, that appears to be a poor suggestion given that split will do this all in a single operation....
csplit might be a far better tool, I just haven't used it. There is certainly a size file where head/tail just won't work, but for many files it might work well enough.
This User Gave Thanks to gandolf989 For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Splitting large file into small files

Hi, I need to split a large file into small files based on a string. At different palces in the large I have the string ^Job. I need to split the file into different files starting from ^Job to the last character before the next ^Job. Also all the small files should be automatically named.... (4 Replies)
Discussion started by: dncs
4 Replies

2. Shell Programming and Scripting

Split a file into 16 small files

Hi I want to split a file that has 'n' number of records into 16 small files. Can some one suggest me how to do this using Unix script? Thanks rrkk (10 Replies)
Discussion started by: rrkks
10 Replies

3. Shell Programming and Scripting

Split large file and add header and footer to each small files

I have one large file, after every 200 line i have to split the file and the add header and footer to each small file? It is possible to add different header and footer to each file? (7 Replies)
Discussion started by: ashish4422
7 Replies

4. Shell Programming and Scripting

script to splite large file to number of small files

Dear All, Could you please help me to split a file contain around 240,000,000 line to 4 files all equally likely , note that we need to maintain that the end of each file should started by start flage (MSISDN) and ended by end flag (End), also the number of the line between the... (10 Replies)
Discussion started by: ahmed.gad
10 Replies

5. Shell Programming and Scripting

Breaking one file into many files based on first column?

Hi, I have a file that looks like this (tab deliminited). MAT1 YKR2 3 MAT1 YMR1 2 MAT1 YFG2 2 MAT2 YLM4 4 MAT2 YHL2 1 BAR1 YKR2 3 BAR1 YFR1 4 BAR1 YMR1 1 What I want to do is break this file down into multiple files. So the result will look like this: File 1... (2 Replies)
Discussion started by: kylle345
2 Replies

6. Shell Programming and Scripting

Breaking the files as 10k recs. per file

Hi, I have a code as given below Set -A _Category="A\ B\ C" for _cat in ${_Category} do sed -e "s:<TABLE_NAME>:${_cat}:g" \ -e "s:<date>:${_dt}:g" \ ${_home}/skl/sq1.sql >> ${_dest}/del_${_dt}.sql fi ... (4 Replies)
Discussion started by: mr_manii
4 Replies

7. UNIX for Advanced & Expert Users

Splitting a file into small files

Hi Folks, Please help me in solving the problem. I want to write script in order to split a file into small pieces and send it automatically through mail. Ex. The file name is CALM*.txt . It is around 50 MB. I want to split the file into 20 MB 2-3 smaller files and send (like uuencode) it... (6 Replies)
Discussion started by: piyushbhashkar
6 Replies

8. UNIX for Dummies Questions & Answers

Breaking a fasta formatted file into multiple files containing each gene separately

Hey, I've been trying to break a massive fasta formatted file into files containing each gene separately. Could anyone help me? I've tried to use the following code but i've recieved errors every time: for i in *.rtf.out do awk '/^>/{f=++d".fasta"} {print > $i.out}' $i done (1 Reply)
Discussion started by: Ann Mc Cartney
1 Replies

9. Shell Programming and Scripting

Split a large array into small chunks

Hi, I need to split a large array "@sharedArray" into 10 small arrays. The arrays should be like @sharedArray1,@sharedArray2,@sharedArray3...so on.. Can anyone help me with the logic to do so :(:confused: (6 Replies)
Discussion started by: rkrish
6 Replies

10. UNIX for Beginners Questions & Answers

Split large file into 24 small files on one hour basis

I Have a large file with 24hrs log in the below format.i need to split the large file in to 24 small files on one hour based.i.e ex:from 09:55 to 10:55,10:55-11:55 can any one help me on this.! ... (20 Replies)
Discussion started by: Raghuram717
20 Replies
ROOTTAIL(1)						      General Commands Manual						       ROOTTAIL(1)

NAME
root-tail - print text directly to X11 root window SYNOPSIS
root-tail [-g|--geometry GEOMETRY] [-fn|--font FONTSPEC] [--color color] [--reload SEC COMMAND] [--shade] [--outline] [--minspace] [--noflicker] [-f|--fork] [--reverse] [--whole] [--partial] [--update] [--cont STRING] [--wordwrap] [--justify] [--noinitial] [--frame] [-id ID] [-i|--interval SECONDS] [-V] file1[,color[,desc]] [file2[,color[,desc]]] DESCRIPTION
Displays a given file anywhere on your X11 root window, i.e. it is kind of tail -f for multiple files using your desktop background as out- put window. All non-option arguments on the command line are files to be logged. A null desc (example: "/var/log/messages,red,") will prevent the printing of a description and the []'s. General Options --color COLOR Use COLOR as default. --font | -fn FONTSPEC Use font FONTSPEC. This can be either a fixed width font like -fn fixed or any font using -fn '-*-*-*-*-*-*-*-*-*-*-*-*-*-*' with the appropriate fields filled out (see xfontsel). Specifying a different FONTSPEC before each filename will cause each file to be displayed in a different font. -f | --fork Forks to the background. --reverse Display the files in reverse order, with the newest lines at the top. --whole Only display whole lines. If the last line of a file doesn't yet end with a newline character then wait until it does before dis- playing it. This is the default if more than one file is being displayed. --partial This is the opposite of the --whole option (see above). It displays lines even if they don't yet have a newline at the end. This is the default if only one file is being displayed. --update Update partial lines 'in place' if they are still on the screen when they are updated in their files. Using --update automatically turns on --partial. --cont STRING When partial lines are broken into two lines in the display, prefix the 2nd line with STRING. Defaults to "|| ". Specify the "--whole" argument to ensure partial lines are never displayed, or specify "--update" to attempt to "repair" broken lines in-place. --cont-color COLOR Use COLOR when displaying the continuation string (as optionally specified with the --cont option above). --wordwrap The default behaviour is to fit as much as possible onto each line of output, even if this means splitting a word between one line and the next. The --wordwrap argument splits lines at spaces if possible. --justify After wrapping long lines, attempt to justify the text to produce a smooth right-hand margin. Implies --wordwrap. --reload SEC COMMAND Re-display the file(s) and run COMMAND every SEC seconds. The default is to never re-display the file(s). --shade Add black shading to the font. --outline Add a black outline to the font (making redraws quite a bit slower). --minspace Use minimum linespace even when using shading or outlining. This might result in leftover pixels (dependign on font and logfile con- tent). --noflicker Use slower but flicker-free update. --noinitial Don't display the end of the file(s) initially. -id ID Use the given window ID for output instead of the root window. -i | --interval SECONDS Use the specified sleeping interval between checks instead of the default 2.4 seconds. Fractional values are OK. -V Print version information. --frame Draw a frame around the selected area. This is useful when trying to find the perfect geometry. EXAMPLE
root-tail -g 800x250+100+50 -font 10x20 /var/log/messages,green -font 12x24 /var/log/secure,red,'ALERT' BUGS
Some desktop environments open a virtual root window and make it difficult to share it. If you cannot see anything after starting root- tail, try to find a setting "allow programs on desktop" or similar, or manually specify a window id. Should you happen to find any bugs please fix them and send me a diff. NOTE: This program was modified by Marc Lehmann <pcg@goof.com>, who couldn't reach the original author. Please direct bug-reports etc. to pcg@goof.com. http://root-tail.plan9.de/ 4.2 Berkeley Distribution 2004-03-27 ROOTTAIL(1)
All times are GMT -4. The time now is 09:55 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy