Sponsored Content
Top Forums Shell Programming and Scripting Randomly selecting sequences and generating specific output files Post 302786675 by hanson44 on Thursday 28th of March 2013 04:26:35 AM
Old 03-28-2013
See if the following bash script works. It works fine here. If $RANDOM function causes error because not bash, there are other ways to generate random numbers under Unix.
Code:
# usage: temp.sh iterations sample_size seed

iterations=${1:-10} sample_size=${2:-3} # RANDOM=$3

seq_cnt=`cat Infile1 | wc -l`
if [ $sample_size -gt $seq_cnt ]; then echo ss too big!!; exit; fi

iteration=1
while [ $iteration -le $iterations ]; do
  rm -f prev_used_nums.txt unsorted-1.txt unsorted-2.txt
  touch prev_used_nums.txt
  seqs_written=0
  while [ $seqs_written -lt $sample_size ]; do
    let "num_to_use = 1 + ($RANDOM % seq_cnt)"
    grep -q $num_to_use prev_used_nums.txt
    if [ $? -eq 0 ]; then continue; fi # already used $num_to_use
    sed -n "$num_to_use { p; q }" Infile1 >> unsorted-1.txt
    sed -n "$num_to_use { p; q }" Infile2 >> unsorted-2.txt
    echo $num_to_use >> prev_used_nums.txt
    let "seqs_written++"
  done
  sort unsorted-1.txt > OutFile1-$iteration
  sort unsorted-2.txt > OutFile2-$iteration
  let "iteration++"
done

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Generating files of specific size

I've been working on getting a script to take size, dir name and file name variables from an input file and creating the same dir structure along with the file of specific size. An example of the input file: size/dirname/filename 2100/JAN_06/12345ABC.TCC 2354/FEB_06/24564XYZ.NOS... (2 Replies)
Discussion started by: nxd25
2 Replies

2. UNIX for Dummies Questions & Answers

randomly renaming files

I have a directory of files that look like filename 001.ext, filename 002.ext, etc. I'd like to rename the files with unique random numbered names, so that the original filenames are stripped and the files are given a new, random number name. I'm not super new to UNIX, but I don't often use it for... (2 Replies)
Discussion started by: platz
2 Replies

3. Shell Programming and Scripting

PERL - Selecting specific files based on 'date stamp' values

Hi, I've list of files in a directory, which have date stamp value in their names. ex: abc_data_20071102.csv, abc_data_20091221.csv, abc_data_20100110.csv, abc_data_20100222.csv, abc_data_20080620.csv,... etc., I need to select and process only files, within the given date... (4 Replies)
Discussion started by: ganapati
4 Replies

4. Shell Programming and Scripting

Trimming sequences based on specific pattern

My files look like this And I need to cut the sequences at the last "A" found in the following 'pattern' -highlighted for easier identification, the pattern is the actual file is not highlighted. The expected result should look like this Thus, all the sequences would end with AGCCCTA... (2 Replies)
Discussion started by: Xterra
2 Replies

5. Shell Programming and Scripting

Removing specific sequences from file

My file looks like this But I need to remove the entry with the identifier >Reference1 along with the entire sequence. Thus, I will end up having the following file Thanks in advance! (2 Replies)
Discussion started by: Xterra
2 Replies

6. UNIX for Dummies Questions & Answers

Selecting specific line using awk

Hi, I would like to get the specific line from the file taking specific coloumn as reference. Thanks and Regards (1 Reply)
Discussion started by: kkarthik_kaja
1 Replies

7. Shell Programming and Scripting

Selecting sequences based on scores

I have two files with thousands of sequences of different lengths. infile1 contains the actual sequences and infile2 the scores for each A, T, G and C in infile1. Something like this: infile1: >HZVJKYI01ECH5R TTGATGTGCCAGCTGCCGTTGGTGTGCCAA >HZVJKYI01AQWJ8 GGATATGATGATGAACTGGTTTGGCACACC... (4 Replies)
Discussion started by: Xterra
4 Replies

8. Shell Programming and Scripting

Help with selecting files from "diff" output

I have two directories Dir_A and Dir_A_Arc. Need help with a shell script. The shell script needs to take the path to these two directories as parameters $1 and $2. The script needs to check if any files exist in these directories and if either of the directories are empty then exit... (5 Replies)
Discussion started by: gaurav99
5 Replies

9. UNIX for Beginners Questions & Answers

Split Command Generating Incomplete Output Files

Hello All, May i please know how do i ensure my split command would NOT generate incomplete output files like below, the last lines in each file is missing some columns or last line is complete. split -b 50GB File File_ File_aa |551|70210203|xxxxxxx|12/22/2010 20:44:58|11/01/2010... (1 Reply)
Discussion started by: Ariean
1 Replies

10. UNIX for Beginners Questions & Answers

How to add specific bases at the beginning and ending of all the fasta sequences?

Hi, I have to add 7 bases of specific nucleotide at the beginning and ending of all the fasta sequences of a file. For example, I have a multi fasta file namely test.fasta as given below test.fasta >TalAA18_Xoo_CIAT_NZ_CP033194.1:_2936369-2939570:+1... (1 Reply)
Discussion started by: dineshkumarsrk
1 Replies
ldns-gen-zone(1)					      General Commands Manual						  ldns-gen-zone(1)

NAME
ldns-gen-zone - read a zonefile and print it while adding DS records and extra RR's SYNOPSIS
ldns-gen-zone ZONEFILE DESCRIPTION
ldns-gen-zone reads a DNS zone file and prints it. It is build for speed, not for a nice formatting. The output has one resource record per line and no pretty-printing makeup. DNSSEC data (NSEC, NSEC3, RRSIG or DNSKEY) is not stripped. You may want to use ldns-read-zone for that. Existing DS records are also not stripped. The idea is to use this tool for quickly generating a representative artificial zonefile from a real zonefile, to use it for testing pur- poses. OPTIONS
-a NUM Adds NUM extra artificial NS RRSets to the output. The RRSets owner names start with 'xn--' in an attempt to ensure uniqueness (nl.-zone does not support IDN's - and this tool was written with that knowledge in mind). An artificial NS RRSet has two NS records; ns1.example.com and ns2.example.com. -p NUM Add NUM% of DS RRSets to the NS RRSets (anywhere between 1-4 DS records per RRSet). -o ORIGIN Sets an $ORIGIN, which can be handy if the one in the zonefile is set to '@' for example. If there is an $ORIGIN in the zonefile, this option will silently be ignored. -s This is the recommended way of processing large zones that are already sorted and canonicalized (ie lowercase). It skips the sorting and canonicalization step that is required for properly grouping RRSets together (before adding any DS records to them. Skipping this step will speed things up. It is not recommended to use this option if you want to add DS records to unsorted, non-canonicalized zones. -h Show usage and exit. -v Show version and exit. EXAMPLES
ldns-gen-zone -a 100000 -p 10 -s ./zonefile.txt Read a zonefile, add 100.000 artificial NS RRSets and 10% of DS records, print it to standard output. Don't sort (will only work well if the input zonefile is already sorted and canonicalized). ldns-gen-zone -p 10 -s -o nl zonefile.txt | named-compilezone -s relative -i none -o zonefile_10.txt nl /dev/stdin This creates a nicely formatted zone file with the help of named-compilezone. It adds 10% DS records to the .nl zone, reformats it and saves it as zonefile_10.txt. AUTHOR
Initially written by Marco Davids, several modifications added by Miek Gieben, both from SIDN. REPORTING BUGS
Report bugs to <ldns-team@nlnetlabs.nl>. BUGS
Only undiscovered ones. CAVEATS
May require a machine with a considerable amount of memory for large zone files. Fake DS records hashes are generated as digest type SHA-256 (RFC4509). Be aware not to change the DIGESTTYPE #define in the source code in anything else but 2 if you want to keep things realistic. Despite a number of efforts, this program is still not the fastest in the world. COPYRIGHT
Copyright (C) 2010 SIDN. This is free software. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. 10 June 2010 ldns-gen-zone(1)
All times are GMT -4. The time now is 12:35 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy