Split and Rename files using Terminal and bin/bash


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Split and Rename files using Terminal and bin/bash
# 1  
Old 12-01-2010
Split and Rename files using Terminal and bin/bash

I have a file named Me_thread_spell.txt that I want to split into smaller files. I want it to be split in each place there is a ;;;. For example,

blah blah blah ;;;
blah bhlah hlabl
awasnceuir
asenduhfoijhacseiodnbfxasd;;;
oabwcuhaweoir;;;

This full file would be three separate files (separated into files by the ;;Smilie.

Then I want to rename the resulting files with "comment" and then a series of numbers, such as comment1.txt, comment2.txt, comment3.txt... etc. up to comment2500.txt.

I've been working on splitting and renaming in Terminal and this is all I have come up with so far (below), and it doesn't split the files correctly and it doesn't rename the files. Please help!

split -p ';;;' Me_thread_spell.txt FILE && for file in FILE*; do mv $comment "$(sed '0-9')".txt; done

I'm pretty much lost at this point. I am running Mac OS X with bin/bash. I've been told a script would be easier, but I am not sure how to make one.
# 2  
Old 12-01-2010
Try csplit - not I'm not completely clear on your requirements but this is close:
Code:
csplit -f comment -n 4 Me_thread_spell.txt '/;;;/'

This will create comment0001 -> comment9999, one file for each ;;; string in the file.
IF you need to have the new files named commentnnnn.txt add this:
Code:
ls comment* > t.lis
while read fname 
do
  mv $fname ${fname}.txt
done < t.lis

edit note corrections in red

This will only work if there are no pre-existing files named 'comment[something]' in the directory. Otherwise you get unexpected results. Plus csplit will overwrite files, for example, if there is a comment0022 out there already, it may be clobbered by csplit.

Last edited by jim mcnamara; 12-01-2010 at 04:11 PM..
This User Gave Thanks to jim mcnamara For This Post:
# 3  
Old 12-01-2010
syntax error?

It appears that

csplit -f comment -n 4 Me_thread_spell.txt /;;;/

is exactly what I need, but when I run it, I get an error:

-bash: syntax error near unexpected token `;;;'

Any ideas?
# 4  
Old 12-01-2010
my bad - '/;;;/'

you need single quotes around the regex... see above
This User Gave Thanks to jim mcnamara For This Post:
# 5  
Old 12-01-2010
Put quotes around /;;;/

In she shell it's a seperator, you could do echo a ; echo b to run two commands on one line. so it has special meaning to the shell and needs quoting to put it in a string.
This User Gave Thanks to Corona688 For This Post:
# 6  
Old 12-02-2010
When I run

csplit -f comment -n 4 Me_thread_spell.txt '/;;;/'

It only separates the main file into 2 files (which aren't text files), rather than the 2500 times that the actual ;;; occurs in my file. It should create 2500 text files, each with an individual instance that includes all the text between ;;; and the next ;;;.

I'm missing something here.... I just can't figure it out!

---------- Post updated 12-02-10 at 02:46 PM ---------- Previous update was 12-01-10 at 11:09 PM ----------

PLEASE HELP! I really don't want to have to copy and paste 2500 files. :-)
# 7  
Old 12-02-2010
You agreed not to bump posts when you registered.

How about awk? It has built-in split-on-string abillities.
Code:
echo "a;;;b;;;c;;;d;;;" | awk 'BEGIN {RS=";;;" ; N=1 } { f=sprintf("comment%d.txt", N++); print $0 > f; }'

this creates comment1.txt through comment5.txt when I run it. Run it on a file like

Code:
awk 'BEGIN {RS=";;;" ; N=1 } { f=sprintf("comment%d.txt", N++); print $0 > f; }' filename

plain awk might not work if the individual records are huge in size, but should work with reasonable sizes, and GNU awk (gawk) can tolerate almost anything...
This User Gave Thanks to Corona688 For This Post:
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Split and Rename Split Files

Hello, I need to split a file by number of records and rename each split file with actual filename pre-pended with 3 digit split number. What I have tried is the below command with 2 digit numeric value split -l 3 -d abc.txt F (# Will Produce split Files as F00 F01 F02) How to produce... (19 Replies)
Discussion started by: techedipro
19 Replies

2. Shell Programming and Scripting

sed to rename files in bash loop

I am trying to use sed to rename all .txt files in /home/cmccabe/test. However, I am getting an error that I seems to be putting the files in a new directory s, instead of in the original. Thank you :). bash # rename classified cd /home/cmccabe/test pattern2_old="_classify"... (2 Replies)
Discussion started by: cmccabe
2 Replies

3. Shell Programming and Scripting

Usage of #!/bin/sh vs #!/bin/bash shell scripts?

Some question about the usage of shell scripts: 1.) Are the commands of the base shell scripts a subset of bash commands? 2.) Assume I got a long, long script WITHOUT the first line. How can I find out if the script was originally designed für "sh" or "bash"? 3.) How can I check a given... (3 Replies)
Discussion started by: pstein
3 Replies

4. Shell Programming and Scripting

Split and rename files

Hello, Need to split files into n number of files and rename the files Example: Input: transaction.txt.1aa transaction.txt.1ab ...... Output: transaction.txt.1 transaction.txt.2 transaction.txt.3 (3 Replies)
Discussion started by: krux_rap
3 Replies

5. UNIX for Dummies Questions & Answers

Split and Rename multiple files

Hi, I have a data file like below messageid|email|timestamp 750452173|123@googlemail.com|2013-05-24 16:14:32 750464921|000@gmail.com|2013-06-13 19:38:01 750385426|001@googlemail.com|2013-01-06 12:06:36 750373470|000@wz.eu|2012-11-30 22:32:07 . . I want to split the files based on the... (4 Replies)
Discussion started by: armsaran
4 Replies

6. UNIX for Dummies Questions & Answers

Bash script to rename files in a directory

Dear friends, I have created a script to rename all files in a directory by appending the file name with username (who created the file), the date it was created. For example, "apple.doc" should be renamed to "johnFeb23apple.doc" where "john" is the owner and "Feb23" is file created date. It... (4 Replies)
Discussion started by: djsnifer
4 Replies

7. UNIX for Dummies Questions & Answers

Bash script to rename all files within a folder...

Hi. I don't have any experience with making scripts in bash. I need a simple script to rename all files in a folder to the format file1.avi, file2.avi, file3.avi, and so on..... Please note that the original files have different filenames and different extensions. But they all need to be... (2 Replies)
Discussion started by: dranzer
2 Replies

8. Shell Programming and Scripting

awk split and rename files

I have a file test1.html like below: <dctm_topnav_en_US> <html> ..... </html> <dctm_topnav_en_CA> <html> ..... </html> <dctm_topnav_en_FR> <html> ..... </html> I need to use awk to split this into three file names like en_US.html , en_CA.html, en_FR.html each having content between... (4 Replies)
Discussion started by: vijay52
4 Replies

9. Shell Programming and Scripting

#!/bin/bash and #1bin/sh command not found error on mac osx terminal/shell script

i am having a weird error on mac os x running some shell scripts. i am a complete newbie at this and this question concerns 2 scripts. one of which a friend of mine wrote (videochecker.sh) a couple weeks ago and it's been running fine on another machine. then last week i wrote capture.sh and it... (2 Replies)
Discussion started by: danpaluska
2 Replies

10. Shell Programming and Scripting

Hello - new here - bash script - need to rename and zip files.

I'm working on a project that basically unzips three zip files. When these unzip they create about 70+ directories with subdirectories of year/month with about 3 to 9 pdf files in each directory. Basically, I'm needing to figure out a way to zip these pdf files up. for instance the script... (1 Reply)
Discussion started by: Aixia
1 Replies
Login or Register to Ask a Question