Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Split a files into many files when condition

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Split a files into many files when condition
# 8  
Old 04-26-2010
Quote:
Originally Posted by agn
To remove empty lines:

Code:
$ awk '/^\+/ { f=++i".txt" } !/^$/{ print >> f }' buf

Thanks, but seems cannot remove the empty line in the beginging and ending of each file (not middle empty lines).
assume 1.txt, yours output is
Code:
+++
a
b
c

but if in between b and c got few empty lines, those middle empty lines cannot be removed.
Thanks
# 9  
Old 04-26-2010
Try this:
Code:
awk '/^\+/{c++; s=""}
!NF && c {s=s?s ORS s:ORS;next}
{print s $0 > "file" c ".txt"; s=""}' file.txt

# 10  
Old 04-26-2010
only bash
Code:
while read L
do	[ -z "$L" ] && continue
	[ "$L" = "+++" ] && ((i++))
	echo "$L" >> file$i.txt
done < file.txt

# 11  
Old 04-26-2010
Smilie Thanks Franklin52, also Frans
i learnt a lot about awk from you.

Thanks

---------- Post updated at 06:52 AM ---------- Previous update was at 06:45 AM ----------

Smilie i got another question, to see whehter can do together.
right now, the file name is "file1.txt", "file2.txt", and "file3.txt".
instead of fix the file name into [1-3].txt, wheter i can have the file name like:
file1.txt to a.txt
file2.txt to d
file3.txt to asdf fefe

means, the file name is each file's 2nd row value. Please advice.

Thanks
# 12  
Old 04-26-2010
Try:
Code:
awk '/^\+/{p=$0;getline;f=$0;$0=p RS $0;s=""}
!NF && f {s=s?s ORS s:ORS;next}
{print s $0 > f; s=""}' file.txt

# 13  
Old 04-26-2010
modified to match your needs
Code:
while read L
do
    [ -z "$L" ] && continue
    [ "$L" = "+++" ] && { read L; F="$L.txt"; echo "+++" > "$F"; }
    echo "$L" >> "$F"
done < file.txt

Last edited by frans; 04-26-2010 at 11:00 AM.. Reason: removed extra ".txt"
# 14  
Old 04-26-2010
Thanks Franklin52 and Frans, your guys are SmilieSmilieSmilie

---------- Post updated at 08:01 AM ---------- Previous update was at 07:50 AM ----------
Quote:
Originally Posted by Franklin52
Try:
Code:
awk '/^\+/{p=$0;getline;f=$0;$0=p RS $0;s=""}
!NF && f {s=s?s ORS s:ORS;next}
{print s $0 > f; s=""}' file.txt

Hi Frank, in your awk, which part is the one "2nd row value to be the file name", if it is the 3rd row, or other row. (assume the row has the value)
Thanks

i tried many ways to modify your awk, but fail to get other rows value as the file name. Smilie please advice.
Last edited by jimmy_y; 04-26-2010 at 10:51 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Automate splitting of files , scp files as each split completes and combine files on target server

i use the split command to split a one terabyte backup file into 10 chunks of 100 GB each. The files are split one after the other. While the files is being split, I will like to scp the files one after the other as soon as the previous one completes, from server A to Server B. Then on server B ,... (2 Replies)
Discussion started by: malaika
2 Replies

2. UNIX for Beginners Questions & Answers

Compare between two files with condition

Hello there. I am trying to compare two files. File1 Austria Mobile 1 United Kingdom Mobile 1 ... File2 Austria Mobile Vien 2 Austria Mobile Ostr 0 United Kingdom Mobile Dev 0.7 United Kingdom Mobile OST 1.5 What i want to do is to compare both files and... (12 Replies)
Discussion started by: dragonfly85
12 Replies

3. UNIX for Beginners Questions & Answers

Split and Rename Split Files

Hello, I need to split a file by number of records and rename each split file with actual filename pre-pended with 3 digit split number. What I have tried is the below command with 2 digit numeric value split -l 3 -d abc.txt F (# Will Produce split Files as F00 F01 F02) How to produce... (19 Replies)
Discussion started by: techedipro
19 Replies

4. Shell Programming and Scripting

Split files

Hi , I have 100 records in a.txt file Need to split the a.txt file in to 5 files 1ST File: ex: My file name should be a1.txt - line count in file should be 1 to 15 2ND File: ex: My file name should be a2.txt - line count in file should be 16 to 40 3ND File: ex: My file name... (1 Reply)
Discussion started by: satish1222
1 Replies

5. Shell Programming and Scripting

Merge two files by condition

I have two files as below A file /* comment for id1 */ "id1" = "x1" /* comment for id2 */ "id2" = "x2" /* comment for id3 */ "id3" = "x3" B file /* comment for id1 */ "id1" = "y1" /* comment for id2 */ "id2" = "x2" (22 Replies)
Discussion started by: mikezang
22 Replies

6. Shell Programming and Scripting

Merging two files with condition

I have two files of the type 111 222 10 112 223 20 113 224 30 114 225 20 and 111 222 9 444 555 8 113 224 32 666 777 25 I want to merge files based on 1 and 2nd column. if 1st and 2nd column are unique in file 1 and 2 keep... (3 Replies)
Discussion started by: digipak
3 Replies

7. Shell Programming and Scripting

How to compare 2 files & get only few columns based on a condition related to both files?

Hiiiii friends I have 2 files which contains huge data & few lines of it are as shown below File1: b.dat(which has 21 columns) SSR 1976 8 12 13 10 44.00 39.0700 70.7800 7.0 0 0.00 0 2.78 0.00 0.00 0 0.00 2.78 0 NULL ISC 1976 8 12 22 32 37.39 36.2942 70.7338... (6 Replies)
Discussion started by: reva
6 Replies

8. UNIX for Dummies Questions & Answers

split files into specified number of output files

Hi everyone, I have some large text files that I need to split into a specific number of files of equal size. As far as I know (and I don't really know that much :)) the split command only lets you specify the number of lines or bytes. The files are all of a different size, so the number of... (4 Replies)
Discussion started by: Migrainegirl
4 Replies

9. Shell Programming and Scripting

List files with a certain condition

Guys help me out here.... I have the following var: date_system=20080507 And I have a folder with many files with the following pattern: test_%date%_reference, like test_20080201_tokyo.csv test_20080306_ny.csv and so on... What I need is: 1) list all files and them compare the... (1 Reply)
Discussion started by: Rafael.Buria
1 Replies

10. Shell Programming and Scripting

remove some files on a condition..

Hi.. when I do a ls -lt, I get a listing of about 200 files.. These are trace files and some of it I might not need.. To be clear, say in a given week , I might not need files that have been traced between 11 and 11:30 am on a particular day. How can I delete based on this condition ? Thanks,... (4 Replies)
Discussion started by: ST2000
4 Replies
Login or Register to Ask a Question