Split the file with renaming.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Split the file with renaming.
# 1  
Old 01-18-2013
Split the file with renaming.

Hi Team,.

I am writing a script find the files which is not .gz format, split it into files which are 50000 line entries per file(using split -l 50000) then compress the smaller files using gzip and delete the original/big file.

After splitting it is creating the files in its own naming format where as I want to keep the same name with .seg1, .seg2, .seg3 so on.

I am not getting a hint to get the same naming with .seg*

could you help.

-rw-rw-r-- 1 scmpadm scmpuser 2390849170 Jan 15 10:58 activity.log.20130114
-rw-rw---- 1 scmpadm scmpuser 43977463 Jan 18 06:00 bgaa
-rw-rw---- 1 scmpadm scmpuser 44904640 Jan 18 06:04 bgab
-rw-rw---- 1 scmpadm scmpuser 48075578 Jan 18 06:07 bgac
-rw-rw---- 1 scmpadm scmpuser 50093878 Jan 18 06:09 bgad
-rw-rw---- 1 scmpadm scmpuser 51356468 Jan 18 06:14 bgae
-rw-rw---- 1 scmpadm scmpuser 49490389 Jan 18 06:20 bgaf
-rw-rw---- 1 scmpadm scmpuser 50395455 Jan 18 06:25 bgag
-rw-rw---- 1 scmpadm scmpuser 49959803 Jan 18 06:32 bgah
-rw-rw---- 1 scmpadm scmpuser 48350470 Jan 18 06:37 bgai
-rw-rw---- 1 scmpadm scmpuser 49689464 Jan 18 06:43 bgaj
-rw-rw---- 1 scmpadm scmpuser 38375424 Jan 18 06:45 bgak

activity.log.20130114
is the big file I wanted to get it as
activity.log.20130114.seg1.gz
activity.log.20130114.seg2.gz
activity.log.20130114.seg3.gz
activity.log.20130114.seg4.gz ....
# 2  
Old 01-18-2013
Code:
FIRST=1
FILE=1
SPL=50000
while [ $SPL -le `wc -l activity.log.20130114`]
do
sed -n '$FIRST,$SPLp' activity.log.20130114 > activity.log.20130114.seg{$FILE}
FIRST=`expr $FIRST + 50000`
SPL=`expr $SPL + 50000`
FILE=`expr $FILE + 1`
done

# 3  
Old 01-24-2013
Hi Pikk45. Thank you for the solution.

this code does it for one file. but this folder gets files everyday.

I tried to run this code I got the following error.

./test.sh[4]: test: 0403-021 A ] character is missing.

the script should pick the file from the folder and split and zip.

let me know what should I change.
# 4  
Old 01-24-2013
Try this:
Code:
split -d -l 50000 activity.log.20130114 activity.log.20130114.seg
gzip activity.log.20130114.seg*

# 5  
Old 01-28-2013
I think I did not asked my query properly, and the list of the files caused an confusion.

let me reiterate.

The files list provided above are created in the dir /data/OAS/rcap/

the script should be able to pick the file one by one and split. by having the suffix originalfilename.seg** where ** could be numbers or alphabetics.

later the split files should be gzipped by removing the original file.

---------- Post updated at 04:06 PM ---------- Previous update was at 04:04 PM ----------

Hi Pikk45,.

I think I did not asked my query properly, and the list of the files caused an issue.

let me reiterate.

The files list provided above are created in the dir /data/OAS/rcap/

the script should be able to pick the file one by one and split. by having the suffix originalfilename.seg** where ** could be numbers or alphabetics.

later the split files should be gzipped by removing the original file.

Could you let me know the changes accordingly.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Renaming file and check for the renamed file existence

Hi Am trying to move a file from one name to another When I do "ls" to check for the moved filename I can see the file but when I try the same with a script am unable.. I think am doing some pretty silly error.. please help.. toMove=`ls | grep -E "partition.+"` mv $toMove partition._org... (7 Replies)
Discussion started by: Priya Amaresh
7 Replies

2. UNIX for Dummies Questions & Answers

Renaming File

Hi there, I have 350 files in this directory: /home/adams/29 that was mistakenly renamed in this format: TTFILE_BIT_638478.txt.dat I want to take out the trailing .dat so that it ends in .txt: TTFILE_BIT_638478.txt I need help please. Thank you. (6 Replies)
Discussion started by: Creems
6 Replies

3. Shell Programming and Scripting

renaming the default output name of the split command

Hello, I would like to split a file but I dont what the subfiles to be named the way they are: ex: default name: xaa xab xac xad xae desired name: b01 b02 b03 b04 b05 I know I can rename them afterwards however I have a varialbe split lenghth, in other words I am not sure how... (2 Replies)
Discussion started by: smarones
2 Replies

4. UNIX for Dummies Questions & Answers

Need help with Renaming a file

I have a file named as Pro_PatAct_MMDDYYYY.csv. I need to renmae it to this Pro_PatAct.csv without the date timestamp. Can someone help me to achieve this using a regular expn. (3 Replies)
Discussion started by: imran_affu
3 Replies

5. Shell Programming and Scripting

Renaming file

Hello, I have an outstanding issue..Iam on linux and using a putty to connect to my server and then fire our unix shell script. At location /usr/sam a file called "er 1 32.txt" out boss transfer via application. From my end on terminal when i want to transfer this file to some other location... (2 Replies)
Discussion started by: j_panky
2 Replies

6. Shell Programming and Scripting

renaming files using split with a delimiter

I have a directory of files that I need to rename by splitting the first and second halves of the filenames using the delimiter "-O" and then renaming with the second half first, followed by two underscores and then the first half. For example, natfinal1995annvol1_14.pdf -O filenum-20639 will be... (2 Replies)
Discussion started by: swimulator
2 Replies

7. Shell Programming and Scripting

File renaming from list of names contained in another file

I have to rename a large number of files so that the name of each file corresponds to a code number that is given side by side in a list (textfile). The list contains in column A the filename of the actual files to be renamed and in column B the name (a client code, 9 digits) that has to be... (7 Replies)
Discussion started by: netfreighter
7 Replies

8. Shell Programming and Scripting

Renaming a file use another file as a sequence calling a shl

have this shl that will FTP a file from the a directory in windows to UNIX, It get the name of the file stored in this variable $UpLoadFileName then put in the local directory LocalDir="${MPATH}/xxxxx/dat_files" that part seems to be working, but then I need to take that file and rename, I am using... (3 Replies)
Discussion started by: rechever
3 Replies

9. Shell Programming and Scripting

Renaming a file

I am a complete Unix newbie and I need some help! (Please...) I need to rename a file from the following format: Test_Test_EAR_1234.ear To the following: Test_Test_EAR.ear In other words, I need to remove everything after & including the final underscore up to the "." What is the best... (6 Replies)
Discussion started by: VeloLisa
6 Replies

10. UNIX for Dummies Questions & Answers

Split a file with no pattern -- Split, Csplit, Awk

I have gone through all the threads in the forum and tested out different things. I am trying to split a 3GB file into multiple files. Some files are even larger than this. For example: split -l 3000000 filename.txt This is very slow and it splits the file with 3 million records in each... (10 Replies)
Discussion started by: madhunk
10 Replies
Login or Register to Ask a Question