How to Pick Random records from a large file Post: 302341798

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

how to pick distinct records..........

How can i pick distinct records which consists of duplicate data from a ASCII file using UNIX commands

2. Shell Programming and Scripting

Extract data from large file 80+ million records

Hello, I have got one file with more than 120+ million records(35 GB in size). I have to extract some relevant data from file based on some parameter and generate other output file. What will be the besat and fastest way to extract the ne file. sample file format :--...

3. Shell Programming and Scripting

Pick random file from ls command.

Lets say I want to pick a random file when I do an "ls" command. I don't have set number of files in each directory. ls | head -1 This gives me the first one in each directory, is there a way to do the same but pick a random one.

4. Shell Programming and Scripting

awk - splitting 1 large file into multiple based on same key records

Hello gurus, I am new to "awk" and trying to break a large file having 4 million records into several output files each having half million but at the same time I want to keep the similar key records in the same output file, not to exist accross the files. e.g. my data is like: Row_Num,...

5. Shell Programming and Scripting

Parse large file on line count (random lines)

I have a file that needs to be parsed into multiple files every time there line contains a number 1. the problem i face is the lines are random and the file size is random. an example is that on line 4, 65, 187, 202 & 209 are number 1's so there has to be file breaks between all those to create 4...

6. Shell Programming and Scripting

Need to generate a file with random data. /dev/[u]random doesn't exist.

Need to use dd to generate a large file from a sample file of random data. This is because I don't have /dev/urandom. I create a named pipe then: dd if=mynamed.fifo do=myfile.fifo bs=1024 count=1024 but when I cat a file to the fifo that's 1024 random bytes: cat randomfile.txt >...

7. Shell Programming and Scripting

Split a large file in n records and skip a particular record

Hello All, I have a large file, more than 50,000 lines, and I want to split it in even 5000 records. Which I can do using sed '1d;$d;' <filename> | awk 'NR%5000==1{x="F"++i;}{print > x}'Now I need to add one more condition that is not to break the file at 5000th record if the 5000th record...

8. Shell Programming and Scripting

[Solved] Help with random pick 1000 number from range 1 to 150000

Hi, Do anybody knows how to use awk or any command to random print out 1000 number which start from range 1 to 150000? I know that "rand" in awk can do similar random selection. But I have no idea how to write a code that can random pick 1000 number from range 1 to 150000 :confused: ...

9. Shell Programming and Scripting

Quick way to select many records from a large file

I have a file, named records.txt, containing large number of records, around 0.5 million records in format below: 28433005 1 1 3 2 2 2 2 2 2 2 2 2 2 2 28433004 0 2 3 2 2 2 2 2 2 1 2 2 2 2 ... Another file is a key file, named key.txt, which is the list of some numbers in the first column of...

10. Shell Programming and Scripting

Selecting random columns from large dataset in UNIX

Dear folks I have a large data set which contains 400K columns. I decide to select 50K determined columns from the whole 400K columns. Is there any command in unix which could do this process for me? I need to also mention that I store all of the columns id in one file which may help to select...

LEARN ABOUT DEBIAN

ldns-walk

ldns-walk(1)						      General Commands Manual						      ldns-walk(1)

NAME

       ldns-walk - Retrieve the contents of a DNSSEC signed zone

SYNOPSIS

       ldns-walk [ OPTION ] ZONE

DESCRIPTION

       ldns-walk  is  used  to	retrieve  the  contents  of  a DNSSEC signed zone.  It does this through NSEC-walking (following the chain of NSEC
       records) and 'guessing' the next non-existent owner name for each NSEC.

       Note that it might get stuck on some wildcard records when used through a caching forwarder. This problem can be circumvented  by  querying
       the authoritative nameserver directly (with the @ argument).

       Of course the nameserver that is used must be DNSSEC-aware.

OPTIONS

       -f     Do  a  'full'  zone walk; by default, ldns-walk will only show the names, and types present at those names. If this option is given,
	      all resource records will be printed.

       -s name
	      Start the walk with this owner name. Useful when continuing the walk for a large zone.

       @ nameserver
	      Send the queries to this nameserver.

BUGS

       The full zone walk function is not complete yet, it does not correctly print delegation records

AUTHOR

       Written by Jelte Jansen as an example for ldns usage.

REPORTING BUGS

       Report bugs to <ldns-team@nlnetlabs.nl>.

COPYRIGHT

       Copyright (C) 2005 NLnet Labs. This is free software. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A  PARTICULAR  PUR-
       POSE.

								    21 Nov 2005 						      ldns-walk(1)

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

how to pick distinct records..........

Discussion started by: ss4u

2. Shell Programming and Scripting

Extract data from large file 80+ million records

Discussion started by: learner16s

3. Shell Programming and Scripting

Pick random file from ls command.

Discussion started by: elbombillo

4. Shell Programming and Scripting

awk - splitting 1 large file into multiple based on same key records

Discussion started by: kam66