04-23-2017
Hi Xterra,
I think you don't understand what is being suggested. If you have a file containing a million records, each of those records has a 1st line that is one of four values, and you want to create four output files where each of those output files contains all records that have the same 1st line; then you do not want to read that input file 4 times. You want to read it once and create all of your 4 output files in one pass. Doing this you read a million records, write a million records, and you're done.
What you are asking to do instead is read a million records, write ~250000 records to one file, and write ~750000 records to another file; then you read ~750000 records, write ~250000 records to one file, and write ~500000 records to another file; then you read ~500000 records, write ~250000 records to one file, and write ~250000 records to another file; and then you read ~250000 records, write ~250000 records to one file and write 0 records to another file. Why would you want to read ~2.5 million records and write ~2.5 million records instead of reading 1 million records and write 1 million records?
The code that you currently have is reading 4 million records and writing 1 million records (i.e., 5 million I/O operations). What you are asking to do would read 2.5 million records and write 2.5 million records (i.e., 5 million I/O operations). Even if we skip the last read and write and just rename one of the last two output files, your plan still has 4.5 million I/O operations instead of the 2 million I/O operations being proposed by RudiC and jim mcnamara.
Is there something else that you haven't told us about your data that would affect what I assume you are trying to do?
This User Gave Thanks to Don Cragun For This Post:
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I want to search a file for a string and then if the string is found I need the line that the string is on - but also the previous two lines from the file (that the pattern will not be found in)
This is on solaris
Can you help? (2 Replies)
Discussion started by: frustrated1
2 Replies
2. Programming
Hi,
Could any one tell me how to reduce the size of an executable file of C under
Unix.
thanks (2 Replies)
Discussion started by: useless79
2 Replies
3. Solaris
My root file size has reached 80% and I am looking where all i can reduce the file size . Here is the output of top directories in / . To me none of this looks useful but not sure . We use an appplication and email. Which all can be deleted . Please advise .
2016989
989445 /var
930059 ... (2 Replies)
Discussion started by: Hitesh Shah
2 Replies
4. Shell Programming and Scripting
guys,
I need to know how to assing pattern matched string as an input command variable. Here it goes'
My script is something like this.
./routing.sh <Server> <enable|disable>
## This Script takes an input <Server> variable from this line of the script ##
echo $1 | egrep... (1 Reply)
Discussion started by: raghunsi
1 Replies
5. Shell Programming and Scripting
I am trying to search a file for a patterns ERR- in a file and return a count for each of the error reported
Input file is a free flowing file without any format
example of output
ERR-00001=5
....
ERR-01010=10
.....
ERR-99999=10 (4 Replies)
Discussion started by: swayam123
4 Replies
6. Shell Programming and Scripting
Hi,
I have two files file1.txt and file2.txt. Please see the attachments.
In file2.txt (which actually is a diff output between two versions of file1.txt.), I extract the pattern corresponding to 1172c1172. Now ,In file1.txt I have to search for this pattern 1172c1172 and if found, I have to... (9 Replies)
Discussion started by: saurabh kumar
9 Replies
7. Shell Programming and Scripting
I have a file full of coordinates of the form:
37.68899917602539 58.07500076293945 57.79100036621094
The numbers don't always have the same number of decimal points. I need to reduce the decimal points of all the numbers (there are 128 rows of 3 numbers) to 2.
I have tried to do this... (2 Replies)
Discussion started by: crunchgargoyle
2 Replies
8. Shell Programming and Scripting
Hi,
Below is my requirement
File1:
svasjsdhvassdvasdhhgvasddhvasdhasdjhvasdjsahvasdjvdasjdvvsadjhv
vdjvsdjasvdasdjbasdjbasdjhasbdasjhdbjheasbdasjdsajhbjasbjasbhddjb
svfsdhgvfdshgvfsdhfvsdadhfvsajhvasjdhvsajhdvsadjvhasjhdvjhsadjahs
File2:
sdh
hgv
I need a command such that... (8 Replies)
Discussion started by: imrandec85
8 Replies
9. Shell Programming and Scripting
One more question:
I want to grep "COS_12_TM_4 pattern from a file look likes :
"COS_12_TM_4" " ];I am taking scan_out as the input from the user.
How to search "COS_12_TM_4" in the file which is corresponds to scan_out (12 Replies)
Discussion started by: Preeti Chandra
12 Replies
10. UNIX for Beginners Questions & Answers
I have this fileA
TEST FILE ABC
this file contains ABC;
TEST FILE DGHT this file contains DGHT;
TEST FILE 123
this file contains ABC,
this file contains DEF,
this file contains XYZ,
this file contains KLM
;
I want to have a fileZ that has only (begin search pattern for will be... (2 Replies)
Discussion started by: vbabz
2 Replies
ungetwc(3C) ungetwc(3C)
NAME
ungetwc() - push a wide character back into an input stream
SYNOPSIS
Obsolescent Interface
Remarks:
This function is compliant with the XPG4 Worldwide Portability Interface wide-character I/O functions. It parallels the 8-bit character
I/O function defined in ungetc(3S).
DESCRIPTION
pushes the character corresponding to the wide-character code wc into the buffer associated with an input stream. That wide-character
code, wc, is returned by the next call to (see getwc(3C)) on that stream. A successful intervening call to a file positioning function
with stream or erases all memory of the pushed-back characters.
affects only the buffer associated with the input stream. It does not affect the contents of the file corresponding to stream.
One character of pushback is guaranteed.
If wc equals does nothing to the buffer and returns
The definition for this function, the type and the value are provided in the header.
Obsolescent Interface
pushes a wide character back into an input stream.
APPLICATION USAGE
After is applied to a stream, the stream becomes wide-oriented (see orientation(5)).
EXTERNAL INFLUENCES
Locale
The category determines how wide character conversions are done.
International Code Set Support
Single- and multi-byte character code sets are supported.
RETURN VALUE
If successful, and return wc and clear the end-of-file indicator for the stream. and return if they cannot insert the wide character.
WARNINGS
is an obsolescent interface supported only for compatibility with existing DCE applications. New multithreaded applications should use
AUTHOR
and were developed by OSF and HP.
SEE ALSO
flockfile(3S), fseek(3S), fgetpos(3S), getwc(3C), setbuf(3S), orientation(5), thread_safety(5).
STANDARDS CONFORMANCE
ungetwc(3C)