How to read and write a random row from a file?


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers How to read and write a random row from a file?
# 8  
Old 05-13-2008
Since my sed doesn't know about "-f -"....
Code:
#! /usr/bin/ksh
file=somefilename
N=$(wc -l < $file|sed 's/ //g')
awk -v N=$N 'BEGIN { for (i=1; i<=10; i++)  printf "%.0fp\n", N*rand() ; exit } ' |sort -n |\
sed '$s/p/{p;q;}/' > tempfile
sed -n -f tempfile $file
rm tempfile
exit 0

This is indeed about as fast as it can get via a script.
# 9  
Old 05-13-2008
If the rows are fixed length you could always use ksh93's <# (()) functionality to quickly seek to the desired line as the following example demonstrates.
Code:
#!/bin/ksh93

TMP=file.$$

cat <<EOT >$TMP
aaa
bbb
ccc
ddd
eee
fff
ggg
hhh
EOT

command exec 3<> $TMP

for i in 0 8 12 20
do
   if (( $(3<# ((${i}))) != ${i} ))
   then
      print "ERROR: Not at position $i"
      exit 1
   else
      read -u3 LINE
      print $LINE
   fi
done

rm $TMP
exit 0

# 10  
Old 05-13-2008
Hi.

If it is important to read the file only once, say for extremely long files, the algorithm (due to Waterman) described by Knuth may be useful:
Quote:
D. E. Knuth: The Art of Computer Programming, Volume 2, Third edition. Reading: Addison-Wesley, 1997. ISBN: 0-201-89684-2.
When I first ran across this (many years ago), I was amazed that this could be done.

One can find an implementation of this as Algorithm B in perl module Algorithm::Numerical::Sample, described at Algorithm::Numerical::Sample - Draw samples from a set - search.cpan.org

The heart of the code that one might use for a sample size of $n from the file in the indirect handle $FILE would be similar to:
Code:
    use Algorithm::Numerical::Sample  qw /sample/;

    $sampler
      = Algorithm::Numerical::Sample::Stream->new( sample_size => $n );
    while (<$FILE>) {
      $sampler->data($_);
    }
    @a = $sampler->extract;
    print @a;

Best wishes ... cheers, drl
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Read line from the file and append it to each row

Hi All, We have a file in the following format: 0.010000 $ ITI 11 LV2 $ 40456211 $ 0.135000 $ ITI 11 LV1 $ 40512211 $ 1.215600 $ ITI 11 ITI3 $ 41406211 $ 24/05/2014 14:05:02 0.030000 $ ITI 11 LV2 $ 40456211 $ ... (3 Replies)
Discussion started by: gauravsinghal79
3 Replies

2. Shell Programming and Scripting

Read/write perl file

Hi I am trying to build a web form where it can take the input from the user and write it to a file. And when I will open that form again that for should read the file that was created at the 1st step and all the fields should auto populate from that file. I have 20 text fields in my form. I... (1 Reply)
Discussion started by: sauravrout
1 Replies

3. Shell Programming and Scripting

Read row number from 1 file and print that row of second file

Hi. How can I read row number from one file and print that corresponding record present at that row in another file. eg file1 1 3 5 7 9 file2 11111 22222 33333 44444 55555 66666 77777 88888 99999 (3 Replies)
Discussion started by: Abhiraj Singh
3 Replies

4. Shell Programming and Scripting

Perl write and read on same file

Hi, I am trying to do a write operation followed by a read operation on the same file through Perl, expecting the output produced by read to contain the new lines added, as follows: #! /usr/bin/perl -w open FH, "+< testfile" or die "$@"; print FH "New content added\n"; while (my $line =... (1 Reply)
Discussion started by: royalibrahim
1 Replies

5. Shell Programming and Scripting

Read and write in the file

Hello Guys, How all are doing? I have an issue in Unix and want help from all of you I have a file in UNIX which it read by line by line , If at the end of line '0' is written the it should fetch that line into another file and change '0' to '1' and If at the end of line '1' is written then it... (10 Replies)
Discussion started by: adisky123
10 Replies

6. Shell Programming and Scripting

File Read and Write

I have got a file in following format: AAAAAAA BBBBBBBB CCCCCCC DDDDDDD I am trying to read this file and out put it in following format: AAAAAAA,BBBBBBB,CCCCCCC,DDDDDD Preferred method is shell or Perl. Any help appreciated. (11 Replies)
Discussion started by: Araoki
11 Replies

7. Linux

File read/ write operation

Hi, I am creating a progress bar for file upload for which I have CGI script which copies the data and depending on certain bytes it increments the progress bar. Here, I am writing the incremented value to a file which is read by Ajax at server end. However, here I want to ask that, is it... (18 Replies)
Discussion started by: xs2punit
18 Replies

8. IP Networking

read/write,write/write lock with smbclient fails

Hi, We have smb client running on two of the linux boxes and smb server on another linux system. During a backup operation which uses smb, read of a file was allowed while write to the same file was going on.Also simultaneous writes to the same file were allowed.Following are the settings in the... (1 Reply)
Discussion started by: swatidas11
1 Replies

9. Shell Programming and Scripting

Read random line from a text file

I have a text file with hundreds of lines, i wish to run a script and reads a random line to pass it to another command line such as: for line in `cat file |grep random line`; do echo $line |mail my@example.com ; done thank you (6 Replies)
Discussion started by: Bashar
6 Replies

10. Shell Programming and Scripting

read and write from a file

I have tried to show the file name whose size is greater than 200 byte in current directory. Please help me. ls -l | tr -s " " " " | cut -f 5,9 -d " " >out.txt #set -a x `cat out.txt` i=0 `cat out.txt` | while do read x echo $x #re=200 j=0 if }" < "200" ] then echo $j j=`expr $j... (2 Replies)
Discussion started by: rinku
2 Replies
Login or Register to Ask a Question