|
|
|
google unix.com
|
|||||||
| Forums | Register | Forum Rules | Links | Albums | FAQ | Members List | Calendar | Search | Today's Posts | Mark Forums Read |
| Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here. |
More UNIX and Linux Forum Topics You Might Find Helpful
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| Extract data from large file 80+ million records | learner16s | Shell Programming and Scripting | 2 | 06-02-2009 01:48 PM |
| Need to pick a .dat file dynamically | ramprius | Shell Programming and Scripting | 10 | 05-26-2009 12:54 AM |
| how can i pick the latest log file as per below | mail2sant | Shell Programming and Scripting | 4 | 04-05-2009 02:14 AM |
| Count No of Records in File without counting Header and Trailer Records | guiguy | Shell Programming and Scripting | 2 | 06-07-2007 01:15 PM |
| how to pick distinct records.......... | ss4u | UNIX for Dummies Questions & Answers | 3 | 01-04-2007 02:39 AM |
 |
Powered by 
|
|
|
LinkBack | Thread Tools | Search this Thread | Rate Thread | Display Modes |
|
|
08-06-2009
|
||||
|
||||
How to Pick Random records from a large file
Hi,
I have a huge file say with 2000000 records. The file has 42 fields. I would like to pick randomly 1000 records from this huge file. Can anyone help me how to do this? |
|
08-06-2009
|
||||
|
||||
|
If you choose a random starting point, then select 1 record at every fixed m interval you for a reasonable number of times (1000 is beyond what is needed have a statistically valid sample ie - random sample of the population. Since you have 200000 records start somewhere between 1 and 200, then step forward by 200 records 1000 times. Code:
awk -v start=$RANDOM 'BEGIN{ start=start %200; start++}
{ if(FNR==start) {print $0; start+=200}
}' inputfile > newfile
|
|
| Bookmarks |
| Tags |
| random records, unix |
| Thread Tools | Search this Thread |
| Display Modes | Rate This Thread |
|
|
Hybrid Mode
