Get 20% of lines in File randomly


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Get 20% of lines in File randomly
# 8  
Old 10-20-2012
@Lem Thank you so much for help
Please i'd like to apply your program on this file: Myfile
Code:
xxxxxxxx-50
xxxxxxxx-51
xxxxxxxx-52
xxxxxxxx-53
./program Myfile 10     #25 percent

My question please how can i eliminate 50 an 51 , i mean i'd like 25% of this list other than this two lines 50 and 51, and i'd like to specify it in the command like that
Code:
./program Myfile 10 50 51

Have you an idea please ?
Thank you so much for help Smilie

Last edited by Scott; 10-20-2012 at 08:21 AM.. Reason: Code tags
# 9  
Old 10-22-2012
Quote:
Originally Posted by chercheur857
My question please how can i eliminate 50 an 51 , i mean i'd like 25% of this list other than this two lines 50 and 51, and i'd like to specify it in the command like that
Code:
./program Myfile 10 50 51

Have you an idea please ?
Thank you so much for help Smilie
Code:
#!/bin/bash

inputfile="/tmp/buddyfile"
cp "$1" $inputfile
(( $# < 3 )) || {
        string="-${3}$"
        for ((a=4;a<=$#;a++)); do
                string+="\|-${!a}$"
        done
        sed -i "/$string/d" $inputfile; }

range=$(wc -l $inputfile | cut -d " " -f1)
(( $range > 32768 )) && exit              ### max number of lines for this script: 32768
percent=$2                                ### set this  as you like, [1-100]
(( 0 < $2 )) && (( $2 < 101 )) || exit
lim=$(( $range * $percent / 100 ))
for ((i=0;i<lim;i++)); do
        num=$(( $RANDOM % $range + 1 ));
        arr[$i]=$num;
        for ((j=0;j<i;j++)); do
                (( ${arr[$j]} == $num )) && {
                        let i--
                        break; }
        done
done

for linenum in "${arr[@]}"; do
        line=$(sed -n "$linenum p" $inputfile)
        ### your stuff here, for example: ###
        echo "$linenum"$'\t'"### $line"
done

exit 0

Changes are in bold. I didn't test it, but it should work.

Of course if 50, 51 and friends are true line numbers counting from 1 (first line) onwards, we can simplify the new part of the script.

Beware: if you have a file of 100 lines, and you exclude 40 lines, and you want 30% of the lines, you'll get 30% of the remaining 60 lines, so you'll get 18 lines. Is this right for you, or do you still want 30 lines among the 60 remaining lines?
--
Bye

Last edited by Lem; 10-22-2012 at 06:43 AM..
This User Gave Thanks to Lem For This Post:
# 10  
Old 10-22-2012
i have tested your code in my example but it display nothing
i run
Code:
./program Myfile 25 50 52

budyfile contain
Code:
xxxxxxxx-51
xxxxxxxx-53

2 lines (but 25% of lines =1)
Have you an idea please ?
Thank you so much for help.

Last edited by Scott; 10-22-2012 at 09:53 AM.. Reason: Code tags
# 11  
Old 10-22-2012
No, 25% is calculated on the remaining lines (after you excluded two lines).
So 25% of 2 is 0.5, which rounds to zero.

Try: ./program Myfile 50 50 52
You'll get 1 line randomly chosen among 2 good lines.
4 total lines - 2 excluded lines = 2 good lines, and 50% of 2 good lines is 1 - randomly chosen - line. Smilie
--
Bye
This User Gave Thanks to Lem For This Post:
# 12  
Old 10-22-2012
Thank you Lem so much for help,
i try to test your script on this file:
Code:
xxxxxxxx-94.yyyyy.zzzzzzzz.aa
xxxxxxxx-95.yyyyy.zzzzzzzz.aa
xxxxxxxx-96.yyyyy.zzzzzzzz.aa
xxxxxxxx-97.yyyyy.zzzzzzzz.aa
xxxxxxxx-98.yyyyy.zzzzzzzz.aa

i run
Code:
./prog.sh  file.txt 50 94 97

it display:
Code:
4	### xxxxxxxx-97.yyyyy.zzzzzzzz.aa
3	### xxxxxxxx-96.yyyyy.zzzzzzzz.aa

have you an idea please ? i'm so sorry if i disturb you
Thank you so much
BestRegards

Last edited by Scott; 10-22-2012 at 09:52 AM.. Reason: For the LAST time.... CODE TAGS
# 13  
Old 10-22-2012
In this file "-94" and alike are not at the end of lines, as they were before. So no lines were excluded. You got 50% of 5 lines, so 2 lines (2.5 rounds to 2).

If you want a good excluding pattern, please show very well how lines can be built.
Is it good a pattern that matches -XY., where X and Y are digits? That means: do you have -XY. once and only once in each line?
--
Bye
This User Gave Thanks to Lem For This Post:
# 14  
Old 10-22-2012
where X and Y are digits? That means: do you have -XY. once and only once in each line?

Yes X and Y are digits, XY no redendant in other line (once and only once in each line)
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to select lines randomly without replacement in UNIX?

Dear Folks I have one column of 15000 lines and want to select randomly 5000 of them in five different times without replacement. I am aware that command 'shuf' and 'sort -R' could select randomly those lines but I am not sure how could I avoid the replacement of selection line. Does anyone have... (10 Replies)
Discussion started by: sajmar
10 Replies

2. Shell Programming and Scripting

Randomly create time in UNIX

Hey, How can i create randomly create time N times. Suppose i want to create data for a particualr date 5 times... Mon Jan 19 11:42:50 Mon Jan 19 19:16:40 Mon Jan 19 12:12:33 Mon Jan 19 14:26:27 Mon Jan 19 12:29:53 Mon Jan 19 13:30:31 I want the script to create N times randome... (2 Replies)
Discussion started by: jaituteja
2 Replies

3. Shell Programming and Scripting

Randomly inserting extra columns into csv file

Hi Tech Guru, I have a test file as below , which needs some more fields to be populated randomly : dks3243;12;20130823;1420;25m;0;syt dks3243;rocy;10 dks3243;kiop;18 sde21p4;77;20151210;8479;7py;9;vfr sde21p4;temp;67 sfq6i01;12;20120123;3412;4rd;7;jui sfq6i01;uymk;90 sfq6i01;kiop;51 ... (8 Replies)
Discussion started by: Lokesha
8 Replies

4. UNIX for Dummies Questions & Answers

How to randomly select lines from a text file

I have a text file with 1000 lines, I want to randomly select 200 lines from it and print them as output. How do I go about doing that? Thanks! (7 Replies)
Discussion started by: evelibertine
7 Replies

5. Programming

Java application dying randomly

Hi, (First post, please be gental!) I have a java app that I am running on unix (centos) But it keeps dying randomly. The times seem random from anything between 3 hours and 3 days. I have a cronjob running to restart it when ever it dies but I would rather this happened less often. ... (2 Replies)
Discussion started by: sm9ai
2 Replies

6. Shell Programming and Scripting

Cron job randomly once a day

I want to create a cron job randomly once a day for my site's registration. The responsible file for registrations is a config file and I need to change the contents twice on day (on and off) I know the way for random cron job for example */n * * * * /usr/local/bin/php... (6 Replies)
Discussion started by: lucker
6 Replies

7. UNIX for Dummies Questions & Answers

randomly renaming files

I have a directory of files that look like filename 001.ext, filename 002.ext, etc. I'd like to rename the files with unique random numbered names, so that the original filenames are stripped and the files are given a new, random number name. I'm not super new to UNIX, but I don't often use it for... (2 Replies)
Discussion started by: platz
2 Replies

8. Shell Programming and Scripting

use awk to read randomly located columns in an excel file

Hi, I have an excel file that have a random count of columns/fields and what im trying to do is to only retrieve all the rows under 2 specific field headers. I can use the usually command for awk which is awk 'print{ $1 $2}' > output.txt, but the location of the 2 specific field headers is... (9 Replies)
Discussion started by: mdap
9 Replies

9. Shell Programming and Scripting

Read line from file randomly

I have data file with customer.dat, and this contains the customer names >cat customer.dat FirstName1 LastName1 FistName2 LastName1 FistName3 MiddleName3 LastName3 This file can contain areoun 100 customer names. Regards, (1 Reply)
Discussion started by: McLan
1 Replies

10. Shell Programming and Scripting

how to select a value randomly

on my desktop i am using the kde rotating desktop image option. this rotates images randomly every half hour. now, i would like to write an html file which will have an inline frame with some text, maybe system messages, or my friends live journal thati read alot, or unix.com! however, i dont want... (1 Reply)
Discussion started by: norsk hedensk
1 Replies
Login or Register to Ask a Question