07-23-2012
How to split 10GB file into small Sizes
Hi Team
I have one 10 Gb log file
I want to split it into say 10 of 1-1Gb file
pls share ur experiences how to do this?
Thanks in advance,
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi
I want to split a file that has 'n' number of records into 16 small files.
Can some one suggest me how to do this using Unix script?
Thanks
rrkk (10 Replies)
Discussion started by: rrkks
10 Replies
2. Shell Programming and Scripting
I have one large file, after every 200 line i have to split the file and the add header and footer to each small file?
It is possible to add different header and footer to each file? (7 Replies)
Discussion started by: ashish4422
7 Replies
3. Shell Programming and Scripting
I'm not a unix guy so excuses my ignorance... I'm the database ETL guy.
I'm trying to be proactive and devise a plan B for a ETL process where I expect a file 10X larger than what I process daily for a recast job. The ETL may handle it but I just don't know.
This file may need to be split... (3 Replies)
Discussion started by: john091
3 Replies
4. Shell Programming and Scripting
I have 2 big files in the size of gb. They are same with respect to content, both are “,” delimited. Now both of them are created by two different processes but has the same logic. The problem is they are differing only in few bytes for e.g one file is 202195751 bytes other is 202195773. So... (2 Replies)
Discussion started by: dsravan
2 Replies
5. UNIX for Dummies Questions & Answers
Dear all,
I have a big file:2879(rows)x400,170 (columns) like below. I 'd like to split the file into small pieces:2879(rows)x2000(columns) per file (the last small piece will be 2879x170.
So far, I only know how to create one samll piece at one time. But actually I need to repeat this work... (6 Replies)
Discussion started by: forevertl
6 Replies
6. Shell Programming and Scripting
In a 10-50GB file , at end of file there is Control-z character
tried the below options,
1. perl -p -i -e 's/^Z//g' new.txt
2. perl -0777lwi -032e0 new.txt
and Sed command, dos2unix etc
it takes more time to remove the control-z. need a command or perl program to GO TO LAST LINE OF FILE ... (7 Replies)
Discussion started by: prsam
7 Replies
7. Shell Programming and Scripting
Hi,
I need to split a large array "@sharedArray" into 10 small arrays.
The arrays should be like @sharedArray1,@sharedArray2,@sharedArray3...so on..
Can anyone help me with the logic to do so :(:confused: (6 Replies)
Discussion started by: rkrish
6 Replies
8. Shell Programming and Scripting
Hi folks,
Is it possible to create 10GB file in linux and transferring the file to other remote server?
Regards,
J (3 Replies)
Discussion started by: scriptscript
3 Replies
9. Shell Programming and Scripting
Dear shell experts,
I would like to spilt a txt file into small ones. However, I did not know how to program use shell. If someone could help, it is greatly appreciated!
Specifically, I supposed there is file named A.txt. The content of the file likes this:
Subject run condtion ACC time... (3 Replies)
Discussion started by: psychmyluo
3 Replies
10. UNIX for Beginners Questions & Answers
I Have a large file with 24hrs log in the below format.i need to split the large file in to 24 small files on one hour based.i.e ex:from 09:55 to 10:55,10:55-11:55
can any one help me on this.!
... (20 Replies)
Discussion started by: Raghuram717
20 Replies
LEARN ABOUT NETBSD
icecream
ICECREAM(1) icecream ICECREAM(1)
NAME
icecream - download icecast and shoutcast streams, redirecting all fetched content to stdout and/or to disk at the same time
SYNOPSIS
icecream [OPTIONS] URL [URL..]
DESCRIPTION
icecream is a lightweight, non-interactive, stream download utility. It connects to icecast and shoutcast servers or direct stream URLs,
and redirects all fetched content to stdout and/or to media files on your disk.
Listen to the stream piping the output to a stdin-capable media player. Save the stream to a named file or split it into different tracks.
It is possible to redirect the stream and save it to disk at the same time.
icecream is able to parse pls and m3u playlists, and to download mp3 and ogg direct stream URLs. If the stream is anonymous it will be
saved as 'stream-time.mp3', where time is actual timestamp.
OPTIONS
-h, --help
Print a help message describing all options
-q, --quiet
Turn off output
-v, --verbose
Be verbose
-t, --tracks
Split stream into tracks (if possible)
--name=NAME
Save the stream to file specified by NAME. Format codes starting with "%" will be replaced. See the date command for valid format
codes.
--stop=N[units]
Stop downloading the stream after N kb/mb/min/songs
--user-agent=AGENT
Set user-agent header to AGENT
--stdout
Output stream to stdout (implies -q)
--sync Turn syncing on, required for some mpeg players that read from stdin
--debug Turn on debugging outputs
EXAMPLES
Streaming to mpg123
icecream --stdout http://radio.com/playlist.pls | mpg123 -
Split stream into different tracks
icecream -t http://metal.org/radio.pls
Split stream into tracks and play with vlc at the same time
icecream -t --stdout http://streaming.com/playlist.pls | vlc file:/dev/stdin
Prepare a 74 minute CD
icecream -t --stop 74min http://trace.net/playlist.m3u
Use a filename with today's date as output
icecream -q --name 'radio_%Y_%m_%d' --stop 60min http://radio.com/playlist.pls
BUGS
You are welcome to send bug reports about icecream to our mailing list. Feel free to visit http://icecream.sourceforge.net
AUTHOR
Written by Gil Megidish <gil at megidish.net>
1.3 2008-04-28 ICECREAM(1)