How to cut a big file into small ones?


 
Thread Tools Search this Thread
# 1  
How to cut a big file into small ones?

Hello all,
Currently I have a txt file named as a.txt with the content as:
Code:
f e100
aa bb
cc dd
ee ff
f e222
aa dd
ff gg
f e987
dd aa
f e2222
gg ff
gg aa
dd ff
ee ee

While, for some reason I want to cut a.txt into small ones, e.g. f1.txt, f2.txt, f3.txt and f4.txt. The routine is to cut a.txt at lines starting with "f e*". Then the four small files will look like, e.g. f1.txt as:
Code:
aa bb 
cc dd
ee ff

f2.txt as:
Code:
aa dd
ff gg

f3.txt as:
Code:
dd aa

f4.txt as:
Code:
gg ff
gg aa
dd ff
ee ee

Any help would be appreciated. Thx in advance.
L

Moderator's Comments:
Mod Comment Please use code tags next time for your code and data.
# 2  
Code:
awk '/^f e.*/{++c; F="f"c".txt"; getline; } { print $0 > F; }' a.txt

# 3  
How about a BASH script then?
Code:
#!/bin/bash

c=0
while read line
do
        if [[ "$line" =~ "^f e.*" ]]
        then
                c=$(( c + 1 ))
        else
                echo "$line" >> f${c}.txt
        fi
done < a.txt

# 4  
well, your script work for small file. if a.txt is a huge one then i got an error as:
awk makes too many open files
it succeed in extracting the first 17 files. i guess there is a output number limit. any tips? thx

---------- Post updated at 12:06 PM ---------- Previous update was at 11:59 AM ----------

---------- Post updated at 12:08 PM ---------- Previous update was at 12:06 PM ----------

thx for prompt reply. but it seems the loop does not jump out. Sorry for being a noob. I seldom program in bash
# 5  
Code:
awk '/^f e.*/{close(F);++c; F="f"c".txt"; next } { print $0 > F; }' a.txt

These 2 Users Gave Thanks to vgersh99 For This Post:
# 6  
Quote:
Originally Posted by locohd
thx for prompt reply. but it seems the loop does not jump out. Sorry for being a noob. I seldom program in bash
What do you mean by loop does not jump out?

1. Put the script in a file:
Code:
cat > cut_file.sh
#!/bin/bash

c=0
while read line
do
        if [[ "$line" =~ "^f e.*" ]]
        then
                c=$(( c + 1 ))
        else
                echo "$line" >> f${c}.txt
        fi
done < a.txt

2. Give execute permission:
Code:
chmod +x cut_file.sh

3. And run the script:
Code:
./cut_file.sh

This User Gave Thanks to Yoda For This Post:
# 7  
Hi.

See man csplit ... cheers, drl
 

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Science: Mathematics
Difficulty: Medium
You can square root a negative number with an imaginary number "i".
True or False?

5 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How big is too big a config.log file?

I have a 5000 line config.log file with several "maybe" errors. Any reccomendations on finding solvable problems? (2 Replies)
Discussion started by: NeedLotsofHelp
2 Replies

2. Shell Programming and Scripting

How to cut some data from big file

How to cut data from big file my file around 30 gb I tried "head -50022172 filename > newfile.txt ,and tail -5454283 newfile.txt. It's slowy. afer that I tried sed -n '46467831,50022172p' filename > newfile.txt ,also slow Please recommend me , faster command to cut some data from... (4 Replies)
Discussion started by: almanto
4 Replies

3. Shell Programming and Scripting

Cut big text file into 2

I have a big text file. I want to cut it into 2 pieces at known point or I know the pattern of the contents from where it can separate the files. Is there any quick command/solution? (4 Replies)
Discussion started by: sandy221
4 Replies

4. Shell Programming and Scripting

bash script working for small size files but not for big size files.

Hi, I have one file stat. Stat file contents are as follows: for example. H50768020040913,00260100,507680,13,0000000643,0000000643,00000,0000 H50769520040808,00260100,507695,13,0000000000,0000000000,00000,0000 H50770620040611,00260100,507706,13,0000000000,0000000000,00000,0000 Now i... (1 Reply)
Discussion started by: davidpreml
1 Replies

5. UNIX for Dummies Questions & Answers

How to view a big file(143M big)

1 . Thanks everyone who read the post first. 2 . I have a log file which size is 143M , I can not use vi open it .I can not use xedit open it too. How to view it ? If I want to view 200-300 ,how can I implement it 3 . Thanks (3 Replies)
Discussion started by: chenhao_no1
3 Replies

Featured Tech Videos