Segment a big file into smaller ones


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Segment a big file into smaller ones
# 1  
Old 12-15-2010
Segment a big file into smaller ones

Greeting to all.

I have big text file that I would like to segment into many smaller files. Each file should be maximum 10 000 lines.

The file is called time.txt. after the execution of the file I would like to have.

time_01.txt, time_02, txt, ...,time_n.txt

Can anybody help.

Br.
# 2  
Old 12-15-2010
use split command in UNIX.

you can read about it using man command.

Code:
bash3.0$ man split

see also csplit


Last edited by ahmad.diab; 12-15-2010 at 12:05 PM..
# 3  
Old 12-15-2010
Code:
 
typeset -i total_lines
typeset -i no_of_files
typeset -i file_no
typeset -i start
typeset -i end
total_lines=$(sed -n '$=' time.txt)
no_of_files=$total_lines/10000+1
file_no=1
end=0
while [ "$file_no" -lt "$no_of_files" ]
do
   start=$end+1
   end=$end+9999
   sed -n "${start},${end}p" time.txt > "time_${file_no}.txt"
done

This User Gave Thanks to anurag.singh For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Splitting a text file into smaller files with awk, how to create a different name for each new file

Hello, I have some large text files that look like, putrescine Mrv1583 01041713302D 6 5 0 0 0 0 999 V2000 2.0928 -0.2063 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0 5.6650 0.2063 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0 3.5217 ... (3 Replies)
Discussion started by: LMHmedchem
3 Replies

2. Programming

Data segment or Text segment

Hi, Whether the following piece of code is placed in the read-only memory of code (text) segment or data segment? char *a = "Hello"; I am getting two different answers while searching in google :( that's why the confusion is (7 Replies)
Discussion started by: royalibrahim
7 Replies

3. Shell Programming and Scripting

parsing data from a big file using keys from another smaller file

Hi, I have 2 files format of file 1 is: a1 b2 a2 c2 d1 f3 format of file 2 is (tab delimited): a1 1.2 0.5 0.06 0.7 0.9 1 0.023 a3 0.91 0.007 0.12 0.34 0.45 1 0.7 a2 1.05 2.3 0.25 1 0.9 0.3 0.091 b1 1 5.4 0.3 9.2 0.3 0.2 0.1 b2 3 5 7 0.9 1 9 0 1 b3 0.001 1 2.3 4.6 8.9 10 0 1 0... (10 Replies)
Discussion started by: Lucky Ali
10 Replies

4. Shell Programming and Scripting

Lookup two values per line (from a second file) and write the smaller value to another file

Hello Unix Gurus, Please let me know if this is hard to understand and I apologize for my inability to explain better. I have a file "Foo" with the following structure CHR_A BP_A SNP_A CHR_B BP_B SNP_B R2 1 ... (3 Replies)
Discussion started by: genehunter
3 Replies

5. UNIX for Dummies Questions & Answers

How big is too big a config.log file?

I have a 5000 line config.log file with several "maybe" errors. Any reccomendations on finding solvable problems? (2 Replies)
Discussion started by: NeedLotsofHelp
2 Replies

6. Shell Programming and Scripting

perl help to split big verilog file into smaller ones for each module

Hi I have a big verilog file with multiple modules. Each module begin with the code word 'module <module-name>(ports,...)' and end with the 'endmodule' keyword. Could you please suggest the best way to split each of these modules into multiple files? Thank you for the help. Example of... (7 Replies)
Discussion started by: return_user
7 Replies

7. Programming

a large file causes segment, how to overcome?

When printing a large file, segment occured. but part of it can be printed normally. There seems no mistake in usage. How to solve the problem. Actually, the file is not very very large. Only 300-400 A4 pages. Thanks (1 Reply)
Discussion started by: cdbug
1 Replies

8. Shell Programming and Scripting

Grepping for filenames containing value in specific segment within file

I am trying to grep for filenames containing a specific value within a particular segment. The lines containing the segment I'm looking through reads like "HL^1^^1^1", "10^9^9^0", and "HL^11^4^8^1". I would like to find the data that contains only the number nine after the third caret where the... (4 Replies)
Discussion started by: HLee1981
4 Replies

9. UNIX for Dummies Questions & Answers

How to view a big file(143M big)

1 . Thanks everyone who read the post first. 2 . I have a log file which size is 143M , I can not use vi open it .I can not use xedit open it too. How to view it ? If I want to view 200-300 ,how can I implement it 3 . Thanks (3 Replies)
Discussion started by: chenhao_no1
3 Replies
Login or Register to Ask a Question