Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Random lines selection form a file.

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Random lines selection form a file.
# 1  
Old 06-10-2008
Random lines selection form a file.
Quote:
I have a data file with 1000 records, I would like to select 100 lines/records randomly from the data file(data.dat) to other file data_extract.dat

The contents of the data.dat file as below:
Code:
>cat data.dat
0001 Robbert
0002 Nick
0003 Mark
.......
1000 Jarek

Quote:
Do you have any suggestion for this with korn shell script?

Regards.
McLan
# 2  
Old 06-10-2008
Prepend a random number to each line, sort the file, take the first few lines, and remove the leading random number.

Code:
 awk 'BEGIN {srand()} {printf "%05.0f %s \n",rand()*99999, $0; }' datafile | sort -n | head -100 | sed 's/^[0-9]* //'

This User Gave Thanks to Perderabo For This Post:
eldeingles
# 3  
Old 06-10-2008
I afraid, if this is going to generate any duplicate random numbers in range from 1 to 100? If yes then how can you avoid that?
If this generates the duplicate numbers then the same record will be picked up more than once.

Cheers,
McLan
# 4  
Old 06-10-2008
Why don't you try it? The range of rand()*99999 is not 1-100, duplicate random numbers are possible, duplicate lines selected from your file are not possible.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need to generate a file with random data. /dev/[u]random doesn't exist.

Need to use dd to generate a large file from a sample file of random data. This is because I don't have /dev/urandom. I create a named pipe then: dd if=mynamed.fifo do=myfile.fifo bs=1024 count=1024 but when I cat a file to the fifo that's 1024 random bytes: cat randomfile.txt >... (7 Replies)
Discussion started by: Devyn
7 Replies

2. Shell Programming and Scripting

Random shuffle of lines of a TXT file

Hello friends, I have a TXT file with 300 lines in it. I need to shuffle all the lines (randomly) so that they get into different order. Can anyone pls provide easy way, if any? I got it done by doing this below but I see it very lengthy/inefficient way. call random function to generate... (2 Replies)
Discussion started by: prvnrk
2 Replies

3. Shell Programming and Scripting

Need to remove a selection of rows separated by blank lines

hello, here is an example: 9.07 9.05 0.00 2.28 0.00 0.08 1.93 3.62 10.97 12.03 12.03 0.00 2.73 0.00 0.07 (3 Replies)
Discussion started by: Baron1
3 Replies

4. UNIX for Dummies Questions & Answers

Random selection of subset of sample from file

Hello Could you please help me to find a code that can randomly select 1224 lines from a file of 12240 and make tn output with 1224 line each. my input is txt file with 12240 lines like : 13474 999003507 0 0 2 -9 13475 999003508 0 0 2 -9 13476 999003509 0 0 1 -9 13477 999003510 0 0 1 -9 ... (7 Replies)
Discussion started by: biopsy
7 Replies

5. Shell Programming and Scripting

Remove x lines form top and y lines form bottom using AWK?

How to remove x lines form top and y lines form bottom. This works, but like awk only cat file | head -n-y | awk 'NR>(x-1)' so remove last 3 lines and 5 firstcat file | head -n-3 | awk 'NR>4' (5 Replies)
Discussion started by: Jotne
5 Replies

6. Shell Programming and Scripting

Parse large file on line count (random lines)

I have a file that needs to be parsed into multiple files every time there line contains a number 1. the problem i face is the lines are random and the file size is random. an example is that on line 4, 65, 187, 202 & 209 are number 1's so there has to be file breaks between all those to create 4... (6 Replies)
Discussion started by: darbs121
6 Replies

7. Shell Programming and Scripting

Random File Selection and Moving

OK, I am stumpped. I have this shell Script that I want to randomly select a file with the extention of .sct. Then using a portion of its file name select the six related .mot files. Then move them all to another folder. I also need a user input form for the number of .SCT files to randomly select... (6 Replies)
Discussion started by: stak1993
6 Replies

8. Shell Programming and Scripting

delete lines form file

Hi i am writing a cron job. in this script i need to delete some line which is match with some pattern. following code i written for deletion sed '1,'$Max_LIneNo' d' myfile.txt >tempfile.tmp mv tempfile.tmp myfile.txt this command is working fine but the problem is that after this... (5 Replies)
Discussion started by: Himanshu_soni
5 Replies

9. Shell Programming and Scripting

PERL: Extract random record which has 4 lines each

Hi, I have a data file with millions of record (N). Each record was saved in 4 lines. So there are total of NX4 lines in the data file. For Example: Host1 a b c d Host2 e f g h Host3 i j k (2 Replies)
Discussion started by: phoeberunner
2 Replies

10. Shell Programming and Scripting

How to extract visually blank lines form the file

Hi, Could some one help me to get rid of visually blank lines from a file using shell or awk or sed (on Solaris machine)? When I use grep grep -v ^$ inputfile >outputfile it removes some blank lines.. but it seems some tab plus space balnk lines remains. thaen I used "grep -v '^]*$' ... (1 Reply)
Discussion started by: hadsuresh
1 Replies
Login or Register to Ask a Question