Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Randomly selecting sequences and generating specific output files
Top Forums Shell Programming and Scripting Randomly selecting sequences and generating specific output files Post 302786141 by Scrutinizer on Wednesday 27th of March 2013 02:22:17 AM
Old 03-27-2013
Here is a start in awk you could try. The first file is read twice solely to determine the number of records..

Code:
awk -v s=3 -v iter=1 '
  NR==FNR { 
    next 
  }
  FNR==1 {
    if(!set) {
      srand();
      n=NR-1
      for(i=1; i<=s; i++) {
        line=0
        while(!line || line in A) line=int(rand()*n)+1
        A[line]
      }
      set=1
    }
    close(f)
    f=FILENAME ".out" iter
  } 
  FNR in A {
    print > f
  }
' infile1 infile1 infile2

You could embed it in a shell loop that increases the iter variable ( -v iter="$loopvar" )


--

Quote:
Originally Posted by Don Cragun
In you output samples, the lines in each set of output files are all in the order in which they appeared in the input files. Is that a requirement for your output, or is it just a coincidence in the random numbers used for your example?

With truly random numbers, the output could contain more than one copy of some output lines. Is it a requirement that the output lines be unique?
Isn't that a matter of sampling with/without replacement?
Last edited by Scrutinizer; 03-28-2013 at 11:13 AM.. Reason: Added parentheses to rand and stand, to make it work in gawk
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Generating files of specific size

I've been working on getting a script to take size, dir name and file name variables from an input file and creating the same dir structure along with the file of specific size. An example of the input file: size/dirname/filename 2100/JAN_06/12345ABC.TCC 2354/FEB_06/24564XYZ.NOS... (2 Replies)
Discussion started by: nxd25
2 Replies

2. UNIX for Dummies Questions & Answers

randomly renaming files

I have a directory of files that look like filename 001.ext, filename 002.ext, etc. I'd like to rename the files with unique random numbered names, so that the original filenames are stripped and the files are given a new, random number name. I'm not super new to UNIX, but I don't often use it for... (2 Replies)
Discussion started by: platz
2 Replies

3. Shell Programming and Scripting

PERL - Selecting specific files based on 'date stamp' values

Hi, I've list of files in a directory, which have date stamp value in their names. ex: abc_data_20071102.csv, abc_data_20091221.csv, abc_data_20100110.csv, abc_data_20100222.csv, abc_data_20080620.csv,... etc., I need to select and process only files, within the given date... (4 Replies)
Discussion started by: ganapati
4 Replies

4. Shell Programming and Scripting

Trimming sequences based on specific pattern

My files look like this And I need to cut the sequences at the last "A" found in the following 'pattern' -highlighted for easier identification, the pattern is the actual file is not highlighted. The expected result should look like this Thus, all the sequences would end with AGCCCTA... (2 Replies)
Discussion started by: Xterra
2 Replies

5. Shell Programming and Scripting

Removing specific sequences from file

My file looks like this But I need to remove the entry with the identifier >Reference1 along with the entire sequence. Thus, I will end up having the following file Thanks in advance! (2 Replies)
Discussion started by: Xterra
2 Replies

6. UNIX for Dummies Questions & Answers

Selecting specific line using awk

Hi, I would like to get the specific line from the file taking specific coloumn as reference. Thanks and Regards (1 Reply)
Discussion started by: kkarthik_kaja
1 Replies

7. Shell Programming and Scripting

Selecting sequences based on scores

I have two files with thousands of sequences of different lengths. infile1 contains the actual sequences and infile2 the scores for each A, T, G and C in infile1. Something like this: infile1: >HZVJKYI01ECH5R TTGATGTGCCAGCTGCCGTTGGTGTGCCAA >HZVJKYI01AQWJ8 GGATATGATGATGAACTGGTTTGGCACACC... (4 Replies)
Discussion started by: Xterra
4 Replies

8. Shell Programming and Scripting

Help with selecting files from "diff" output

I have two directories Dir_A and Dir_A_Arc. Need help with a shell script. The shell script needs to take the path to these two directories as parameters $1 and $2. The script needs to check if any files exist in these directories and if either of the directories are empty then exit... (5 Replies)
Discussion started by: gaurav99
5 Replies

9. UNIX for Beginners Questions & Answers

Split Command Generating Incomplete Output Files

Hello All, May i please know how do i ensure my split command would NOT generate incomplete output files like below, the last lines in each file is missing some columns or last line is complete. split -b 50GB File File_ File_aa |551|70210203|xxxxxxx|12/22/2010 20:44:58|11/01/2010... (1 Reply)
Discussion started by: Ariean
1 Replies

10. UNIX for Beginners Questions & Answers

How to add specific bases at the beginning and ending of all the fasta sequences?

Hi, I have to add 7 bases of specific nucleotide at the beginning and ending of all the fasta sequences of a file. For example, I have a multi fasta file namely test.fasta as given below test.fasta >TalAA18_Xoo_CIAT_NZ_CP033194.1:_2936369-2939570:+1... (1 Reply)
Discussion started by: dineshkumarsrk
1 Replies

LEARN ABOUT FREEBSD

random

RANDOM(6)							 BSD Games Manual							 RANDOM(6)

NAME

     random -- random lines from a file or random numbers

SYNOPSIS

     random [-elrUuw] [-f filename] [denominator]

DESCRIPTION

     Random has two distinct modes of operations.  The default is to read in lines from the standard input and randomly write them out to the
     standard output with a probability of 1 / denominator.  The default denominator for this mode of operation is 2, giving each line a 50/50
     chance of being displayed.

     The second mode of operation is to read in a file from filename and randomize the contents of the file and send it back out to standard out-
     put.  The contents can be randomized based off of newlines or based off of space characters as determined by isspace(3).  The default
     denominator for this mode of operation is 1, which gives each line a chance to be displayed, but in a random(3) order.

     The options are as follows:

     -e      If the -e option is specified, random does not read or write anything, and simply exits with a random exit value of 0 to denominator
	     - 1, inclusive.

     -f filename
	     The -f option is used to specify the filename to read from.  Standard input is used if filename is set to '-'.

     -l      Randomize the input via newlines (the default).

     -r      The -r option guarantees that the output is unbuffered.

     -U      Tells random(6) that it is okay for it to reuse any given line or word when creating a randomized output.

     -u      Tells random(6) not to select the same line or word from a file more than once (the default).  This does not guarantee uniqueness if
	     there are two of the same tokens from the input, but it does prevent selecting the same token more than once.

     -w      Randomize words separated by isspace(3) instead of newlines.

SEE ALSO

     random(3), fortune(6)

HISTORY

     The functionality to randomizing lines and words was added in 2003 by Sean Chittenden <seanc@FreeBSD.org>.

BUGS

     No index is used when printing out tokens from the list which makes it rather slow for large files (10MB+).  For smaller files, however, it
     should still be quite fast and efficient.

BSD
								 February 8, 2003							       BSD
All times are GMT -4. The time now is 10:02 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy