Quick way to select many records from a large file
I have a file, named records.txt, containing large number of records, around 0.5 million records in format below:
Another file is a key file, named key.txt, which is the list of some numbers in the first column of file records.txt.
There are about 0.2 million numbers in key.txt. Now I am trying to pick out the records from records.txt based on key.txt. I tried scripts below:
pick_records.s
I ran the scripts by: source pick_records.s > output.txt
The scripts did the job but ran slow. I am wondering if there is more efficient way to achieve this task.
Thanks.
Last edited by vgersh99; 04-27-2015 at 08:08 PM..
Reason: code tags, please!
As part of a bigger task, I had to read thru a file and separate records into various batches based on a field. Specifically, separate records based on the value in the batch field as defined below. The batch field left-justified numbers.
The datafile is here
> cat infile
12345 1 John Smith ... (5 Replies)
what is the correct command for finding the largest file and displaying it without any error information?
I can find it, but how do I display it in the same command? (6 Replies)
Hello,
I have got one file with more than 120+ million records(35 GB in size). I have to extract some relevant data from file based on some parameter and generate other output file.
What will be the besat and fastest way to extract the ne file.
sample file format :--... (2 Replies)
Hi,
I have a huge file say with 2000000 records. The file has 42 fields. I would like to pick randomly 1000 records from this huge file. Can anyone help me how to do this? (1 Reply)
Dear list
its my first post and i would like to greet everyone
What i would like to do is select records 7 and 11 from each files in a folder then run an executable inside the script for the selected parameters.
The file format is something like this
7 100 200
7 100 250
7 100 300 ... (1 Reply)
Hello gurus,
I am new to "awk" and trying to break a large file having 4 million records into several output files each having half million but at the same time I want to keep the similar key records in the same output file, not to exist accross the files.
e.g. my data is like:
Row_Num,... (6 Replies)
Hello:
I am new to shell script programming. Now I would like to select specific records block from a file. For example, current file "xyz.txt" is containing 1million records and want to select the block of records from line number 50000 to 100000 and save into a file. Can anyone suggest me how... (3 Replies)
Hello All,
I have a large file, more than 50,000 lines, and I want to split it in even 5000 records. Which I can do using
sed '1d;$d;' <filename> | awk 'NR%5000==1{x="F"++i;}{print > x}'Now I need to add one more condition that is not to break the file at 5000th record if the 5000th record... (20 Replies)
Hi All
I would like to modify a file like this:
>antax gioq21 tris notes
abcdefghij
klmnopqrs
>betax gion32 ter notes2
tuvzabcdef
ahgskslsooin this:
>tris
abcdefghij
klmnopqrs
>ter
tuvzabcdef
ahgskslsoo
So, I would like to remove the first two fields(and output field 3) in record... (4 Replies)
Discussion started by: giuliangiuseppe
4 Replies
LEARN ABOUT DEBIAN
otfdump
OTFDUMP(1) User Commands OTFDUMP(1)NAME
otfdump - otfdump
DESCRIPTION
otfdump - convert otf traces or parts of it into a human readable, long
version
Options:
-h, --help
show this help message
-V show OTF version
-f <n> set max number of filehandles available (default: 50)
-o <file>
output file if the ouput file is unspecified the stdout will be used
--num <a> <b>
output only records no. [a,b]
--time <a> <b> output only records with time stamp in [a,b]
--nodef
omit definition records
--noevent
omit event records
--nostat
omit statistic records
--nosnap
omit snapshot records
--nomarker
omit marker records
--nokeyvalue
omit key-value pairs
--fullkeyvalue show key-value pairs including the contents
of byte-arrays
--procs <a>
show only processes <a> <a> is a space-seperated list of process-tokens
--records <a>
show only records <a> <a> is a space-seperated list of record-type-numbers record-type-numbers can be found in OTF_Definitions.h
(OTF_*_RECORD)
-s, --silent
do not display anything except the time otfdump needed to read the tracefile
otfdump 1.10.2 May 2012 OTFDUMP(1)