csplit to stdout


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers csplit to stdout
# 1  
Old 01-26-2011
csplit to stdout

I want to split big files based on a pattern to stdout.

Although csplit works well for me splitting the output into separate files (e.g. xx00, xx01, xx02, ...), the following is not working as expected:

<code>
# assuming pattern occurs less than 100 times
csplit bigfile '%pattern%' '{100}'
</code>

The manpage says no output file is created, but when I run csplit as above, files are indeed created (xx00, ...).

What am I doing wrong?
# 2  
Old 01-26-2011
What does it mean to split a file, to stdout? They'd all end up in the same place, so hardly splitting...

All the manpage I have says about stdout is that it'll print byte counts to it. Nothing about not creating files.
# 3  
Old 01-26-2011
Quote:
Originally Posted by Corona688
What does it mean to split a file, to stdout? They'd all end up in the same place, so hardly splitting...

All the manpage I have says about stdout is that it'll print byte counts to it. Nothing about not creating files.
Let me rephrase.

"bigfile" is actually not a file, it's a stream of bytes from the network.

What splitting to stdout means is that splitting does not require 2X the file size, i.e. bigfile + all the split segments, if one wants to do some processing on the split segments before saving them as "bigfile" locally (the processing being done using shell utilities that, unlike getchar in C, cannot recognise EOF).

Here's what one manpage says on the arguments to csplit:

<code>
The args operands may be a combination of the following patterns:

/regexp/[[+|-]offset]
Create a file containing the input from the current line to (but not including) the next line matching the given basic regular expression. An optional offset from the line that matched may be specified.

%regexp%[[+|-]offset]
Same as above but a file is not created for the output.

line_no
Create containing the input from the current line to (but not including) the specified line number.

{num}
Repeat the previous pattern the specified number of times. If it follows a line number pattern, a new file will be created for each line_no lines, num times. The first line of the file is line number 1 for historic reasons.
</code>

I think I misinterpreted the existence of arg with the % delimiter to imply the segments could be printed to stdout, without creating files. Reading the manpage again, I think that arg is only for excluding some segments from being saved. And I'm not sure printing the segments to stdout would make any difference, i.e., shell utilities still could not detect the EOF. But, I was curious.

Last edited by uiop44; 01-26-2011 at 07:08 PM..
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Linux csplit command

Input file: CLK00027 TESTDATA 0 S 600000 \r 0001RFC 192321 321321 321321 \r 0002 rfd this is testdata1 CLK00027 TESTDATA 0 S 600000 \r 0001RFC 192321 321321 321321 \r 0002 rfd this is testdata2 I Need to split this file into seperate files file1.txt CLK00027 ... (1 Reply)
Discussion started by: VXANDERB
1 Replies

2. Shell Programming and Scripting

simple csplit problem

Hello am trying to split the following top output at the TTY line but having some issues: System: msisprd Sun Oct 9 09:35:37 2011 Load averages: 0.14, 0.17, 0.16 411 processes: 361 sleeping, 50 running Cpu states: CPU LOAD USER NICE SYS ... (3 Replies)
Discussion started by: delphys
3 Replies

3. Shell Programming and Scripting

CSPLIT help

I have a file with contents <wmqi> sdf sdf sdffghghhjjfh </wmqi> <wmqi> gh dfg hhjhj sdfsdf g </wmqi> <wmqi> dfgdf fg dfgfg </wmqi> <wmqi> (6 Replies)
Discussion started by: Shivdatta
6 Replies

4. Shell Programming and Scripting

csplit suffix-format: how to?

I am using GNU csplit to extract chapters from a big file into smaller files. I want to use the -b option -b, --suffix-format=FORMAT use sprintf FORMAT instead of %d but I have failed so far. 1) All the generated files need to have a suffix .txt at the end 2) They have to look like... (1 Reply)
Discussion started by: MarioColuzzi
1 Replies

5. Shell Programming and Scripting

csplit issue

Hello all The below command works perfeft when executed from the shell prompt. "csplit -f first Allocation.log.1 "%. Oct 02 .%" {1} " and generates first00 file. But it fails if I include the same command in my script. I am trying to automate some process and this is turning... (1 Reply)
Discussion started by: uandme2k2
1 Replies

6. Shell Programming and Scripting

Problem with csplit

Hi All, I have a strange behaviour from csplit command on some text files. I have a comma separated file and data with in double quotes. This file is being generated from Sequential stage of Data stage tool. This file has 67 fields in each records with some null and blank in the data.... (4 Replies)
Discussion started by: shreekrishnagd
4 Replies

7. UNIX for Dummies Questions & Answers

Split files using Csplit

I have an excel file with more than 65K records... Since excel does not take more than 65K records i wan to split the file and send it as two excel files... Could some help me how to use the csplit by specifiying the no of records (7 Replies)
Discussion started by: savitha
7 Replies

8. UNIX for Dummies Questions & Answers

csplit limitations

I am trying to use the csplit file on a file that contains records that have more than 2048 characters on a line. The resultant split file seems to ignore the rest of the line and I lose the data. Is there any way that csplit can handle record lengths greater than 2048? Thanks (0 Replies)
Discussion started by: ravagga
0 Replies

9. UNIX for Advanced & Expert Users

csplit not behaving

I have a large file with the first 2 characters of each line determining the type of record. type 03 being a subheader and then it will have multiple 04 records. eg: 03,xxx,xxxx,xxxx 04,xxxxxxxxxxxxxxxxxxxxxxxxxxxx 04,xxxxxxxxxxxxxxxxxxxxxxxxxxxx 03,xxx,xxx,xxx ... (2 Replies)
Discussion started by: badg3r
2 Replies

10. Shell Programming and Scripting

csplit problem....please help me

Dear Friends, please help me. I am using csplit to split the files, i.e., csplit -f filetype_ -n 3 filename '/regexpn/' {5} which will split file "filename" into 5 subfiles filetype_000 filetype_001 filetype_002 filetype_003 filetype_004 but if I run the csplit commad again it will... (2 Replies)
Discussion started by: kumar1
2 Replies
Login or Register to Ask a Question