Visit The New, Modern Unix Linux Community


Split the file with renaming.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Split the file with renaming.
# 1  
Split the file with renaming.

Hi Team,.

I am writing a script find the files which is not .gz format, split it into files which are 50000 line entries per file(using split -l 50000) then compress the smaller files using gzip and delete the original/big file.

After splitting it is creating the files in its own naming format where as I want to keep the same name with .seg1, .seg2, .seg3 so on.

I am not getting a hint to get the same naming with .seg*

could you help.

-rw-rw-r-- 1 scmpadm scmpuser 2390849170 Jan 15 10:58 activity.log.20130114
-rw-rw---- 1 scmpadm scmpuser 43977463 Jan 18 06:00 bgaa
-rw-rw---- 1 scmpadm scmpuser 44904640 Jan 18 06:04 bgab
-rw-rw---- 1 scmpadm scmpuser 48075578 Jan 18 06:07 bgac
-rw-rw---- 1 scmpadm scmpuser 50093878 Jan 18 06:09 bgad
-rw-rw---- 1 scmpadm scmpuser 51356468 Jan 18 06:14 bgae
-rw-rw---- 1 scmpadm scmpuser 49490389 Jan 18 06:20 bgaf
-rw-rw---- 1 scmpadm scmpuser 50395455 Jan 18 06:25 bgag
-rw-rw---- 1 scmpadm scmpuser 49959803 Jan 18 06:32 bgah
-rw-rw---- 1 scmpadm scmpuser 48350470 Jan 18 06:37 bgai
-rw-rw---- 1 scmpadm scmpuser 49689464 Jan 18 06:43 bgaj
-rw-rw---- 1 scmpadm scmpuser 38375424 Jan 18 06:45 bgak

activity.log.20130114
is the big file I wanted to get it as
activity.log.20130114.seg1.gz
activity.log.20130114.seg2.gz
activity.log.20130114.seg3.gz
activity.log.20130114.seg4.gz ....
# 2  
Code:
FIRST=1
FILE=1
SPL=50000
while [ $SPL -le `wc -l activity.log.20130114`]
do
sed -n '$FIRST,$SPLp' activity.log.20130114 > activity.log.20130114.seg{$FILE}
FIRST=`expr $FIRST + 50000`
SPL=`expr $SPL + 50000`
FILE=`expr $FILE + 1`
done

# 3  
Hi Pikk45. Thank you for the solution.

this code does it for one file. but this folder gets files everyday.

I tried to run this code I got the following error.

./test.sh[4]: test: 0403-021 A ] character is missing.

the script should pick the file from the folder and split and zip.

let me know what should I change.
# 4  
Try this:
Code:
split -d -l 50000 activity.log.20130114 activity.log.20130114.seg
gzip activity.log.20130114.seg*

# 5  
I think I did not asked my query properly, and the list of the files caused an confusion.

let me reiterate.

The files list provided above are created in the dir /data/OAS/rcap/

the script should be able to pick the file one by one and split. by having the suffix originalfilename.seg** where ** could be numbers or alphabetics.

later the split files should be gzipped by removing the original file.

---------- Post updated at 04:06 PM ---------- Previous update was at 04:04 PM ----------

Hi Pikk45,.

I think I did not asked my query properly, and the list of the files caused an issue.

let me reiterate.

The files list provided above are created in the dir /data/OAS/rcap/

the script should be able to pick the file one by one and split. by having the suffix originalfilename.seg** where ** could be numbers or alphabetics.

later the split files should be gzipped by removing the original file.

Could you let me know the changes accordingly.

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #156
Difficulty: Easy
The first two-network TCP/IP communications test was performed between Stanford and University College London, in 1975.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Renaming file and check for the renamed file existence

Hi Am trying to move a file from one name to another When I do "ls" to check for the moved filename I can see the file but when I try the same with a script am unable.. I think am doing some pretty silly error.. please help.. toMove=`ls | grep -E "partition.+"` mv $toMove partition._org... (7 Replies)
Discussion started by: Priya Amaresh
7 Replies

2. UNIX for Dummies Questions & Answers

Renaming File

Hi there, I have 350 files in this directory: /home/adams/29 that was mistakenly renamed in this format: TTFILE_BIT_638478.txt.dat I want to take out the trailing .dat so that it ends in .txt: TTFILE_BIT_638478.txt I need help please. Thank you. (6 Replies)
Discussion started by: Creems
6 Replies

3. Shell Programming and Scripting

renaming the default output name of the split command

Hello, I would like to split a file but I dont what the subfiles to be named the way they are: ex: default name: xaa xab xac xad xae desired name: b01 b02 b03 b04 b05 I know I can rename them afterwards however I have a varialbe split lenghth, in other words I am not sure how... (2 Replies)
Discussion started by: smarones
2 Replies

4. UNIX for Dummies Questions & Answers

Need help with Renaming a file

I have a file named as Pro_PatAct_MMDDYYYY.csv. I need to renmae it to this Pro_PatAct.csv without the date timestamp. Can someone help me to achieve this using a regular expn. (3 Replies)
Discussion started by: imran_affu
3 Replies

5. Shell Programming and Scripting

Renaming file

Hello, I have an outstanding issue..Iam on linux and using a putty to connect to my server and then fire our unix shell script. At location /usr/sam a file called "er 1 32.txt" out boss transfer via application. From my end on terminal when i want to transfer this file to some other location... (2 Replies)
Discussion started by: j_panky
2 Replies

6. Shell Programming and Scripting

renaming files using split with a delimiter

I have a directory of files that I need to rename by splitting the first and second halves of the filenames using the delimiter "-O" and then renaming with the second half first, followed by two underscores and then the first half. For example, natfinal1995annvol1_14.pdf -O filenum-20639 will be... (2 Replies)
Discussion started by: swimulator
2 Replies

7. Shell Programming and Scripting

File renaming from list of names contained in another file

I have to rename a large number of files so that the name of each file corresponds to a code number that is given side by side in a list (textfile). The list contains in column A the filename of the actual files to be renamed and in column B the name (a client code, 9 digits) that has to be... (7 Replies)
Discussion started by: netfreighter
7 Replies

8. Shell Programming and Scripting

Renaming a file use another file as a sequence calling a shl

have this shl that will FTP a file from the a directory in windows to UNIX, It get the name of the file stored in this variable $UpLoadFileName then put in the local directory LocalDir="${MPATH}/xxxxx/dat_files" that part seems to be working, but then I need to take that file and rename, I am using... (3 Replies)
Discussion started by: rechever
3 Replies

9. Shell Programming and Scripting

Renaming a file

I am a complete Unix newbie and I need some help! (Please...) I need to rename a file from the following format: Test_Test_EAR_1234.ear To the following: Test_Test_EAR.ear In other words, I need to remove everything after & including the final underscore up to the "." What is the best... (6 Replies)
Discussion started by: VeloLisa
6 Replies

10. UNIX for Dummies Questions & Answers

Split a file with no pattern -- Split, Csplit, Awk

I have gone through all the threads in the forum and tested out different things. I am trying to split a 3GB file into multiple files. Some files are even larger than this. For example: split -l 3000000 filename.txt This is very slow and it splits the file with 3 million records in each... (10 Replies)
Discussion started by: madhunk
10 Replies

Featured Tech Videos