07-23-2012
supose this is my file
abc.log 10GB
i want this to be
abc1.log 1gb
abc.log 1gb
------ and so one
Please share the command
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi
I want to split a file that has 'n' number of records into 16 small files.
Can some one suggest me how to do this using Unix script?
Thanks
rrkk (10 Replies)
Discussion started by: rrkks
10 Replies
2. Shell Programming and Scripting
I have one large file, after every 200 line i have to split the file and the add header and footer to each small file?
It is possible to add different header and footer to each file? (7 Replies)
Discussion started by: ashish4422
7 Replies
3. Shell Programming and Scripting
I'm not a unix guy so excuses my ignorance... I'm the database ETL guy.
I'm trying to be proactive and devise a plan B for a ETL process where I expect a file 10X larger than what I process daily for a recast job. The ETL may handle it but I just don't know.
This file may need to be split... (3 Replies)
Discussion started by: john091
3 Replies
4. Shell Programming and Scripting
I have 2 big files in the size of gb. They are same with respect to content, both are “,” delimited. Now both of them are created by two different processes but has the same logic. The problem is they are differing only in few bytes for e.g one file is 202195751 bytes other is 202195773. So... (2 Replies)
Discussion started by: dsravan
2 Replies
5. UNIX for Dummies Questions & Answers
Dear all,
I have a big file:2879(rows)x400,170 (columns) like below. I 'd like to split the file into small pieces:2879(rows)x2000(columns) per file (the last small piece will be 2879x170.
So far, I only know how to create one samll piece at one time. But actually I need to repeat this work... (6 Replies)
Discussion started by: forevertl
6 Replies
6. Shell Programming and Scripting
In a 10-50GB file , at end of file there is Control-z character
tried the below options,
1. perl -p -i -e 's/^Z//g' new.txt
2. perl -0777lwi -032e0 new.txt
and Sed command, dos2unix etc
it takes more time to remove the control-z. need a command or perl program to GO TO LAST LINE OF FILE ... (7 Replies)
Discussion started by: prsam
7 Replies
7. Shell Programming and Scripting
Hi,
I need to split a large array "@sharedArray" into 10 small arrays.
The arrays should be like @sharedArray1,@sharedArray2,@sharedArray3...so on..
Can anyone help me with the logic to do so :(:confused: (6 Replies)
Discussion started by: rkrish
6 Replies
8. Shell Programming and Scripting
Hi folks,
Is it possible to create 10GB file in linux and transferring the file to other remote server?
Regards,
J (3 Replies)
Discussion started by: scriptscript
3 Replies
9. Shell Programming and Scripting
Dear shell experts,
I would like to spilt a txt file into small ones. However, I did not know how to program use shell. If someone could help, it is greatly appreciated!
Specifically, I supposed there is file named A.txt. The content of the file likes this:
Subject run condtion ACC time... (3 Replies)
Discussion started by: psychmyluo
3 Replies
10. UNIX for Beginners Questions & Answers
I Have a large file with 24hrs log in the below format.i need to split the large file in to 24 small files on one hour based.i.e ex:from 09:55 to 10:55,10:55-11:55
can any one help me on this.!
... (20 Replies)
Discussion started by: Raghuram717
20 Replies
LEARN ABOUT CENTOS
unbuffer
UNBUFFER(1) General Commands Manual UNBUFFER(1)
NAME
unbuffer - unbuffer output
SYNOPSIS
unbuffer program [ args ]
INTRODUCTION
unbuffer disables the output buffering that occurs when program output is redirected from non-interactive programs. For example, suppose
you are watching the output from a fifo by running it through od and then more.
od -c /tmp/fifo | more
You will not see anything until a full page of output has been produced.
You can disable this automatic buffering as follows:
unbuffer od -c /tmp/fifo | more
Normally, unbuffer does not read from stdin. This simplifies use of unbuffer in some situations. To use unbuffer in a pipeline, use the
-p flag. Example:
process1 | unbuffer -p process2 | process3
CAVEATS
unbuffer -p may appear to work incorrectly if a process feeding input to unbuffer exits. Consider:
process1 | unbuffer -p process2 | process3
If process1 exits, process2 may not yet have finished. It is impossible for unbuffer to know long to wait for process2 and process2 may
not ever finish, for example, if it is a filter. For expediency, unbuffer simply exits when it encounters an EOF from either its input or
process2.
In order to have a version of unbuffer that worked in all situations, an oracle would be necessary. If you want an application-specific
solution, workarounds or hand-coded Expect may be more suitable. For example, the following example shows how to allow grep to finish pro-
cessing when the cat before it finishes first. Using cat to feed grep would never require unbuffer in real life. It is merely a place-
holder for some imaginary process that may or may not finish. Similarly, the final cat at the end of the pipeline is also a placeholder
for another process.
$ cat /tmp/abcdef.log | grep abc | cat
abcdef
xxxabc defxxx
$ cat /tmp/abcdef.log | unbuffer grep abc | cat
$ (cat /tmp/abcdef.log ; sleep 1) | unbuffer grep abc | cat
abcdef
xxxabc defxxx
$
BUGS
The man page is longer than the program.
SEE ALSO
"Exploring Expect: A Tcl-Based Toolkit for Automating Interactive Programs" by Don Libes, O'Reilly and Associates, January 1995.
AUTHOR
Don Libes, National Institute of Standards and Technology
1 June 1994 UNBUFFER(1)