06-15-2017
Thank you Rudi. I should learn awk, shouldn't I. That is a good way to count the occurrences. Is there a way, having counted the occurrences, to echo an entire line, if and only if the 1st 3rd 5th and 7th field of that line all appear at least 4 times in the file? (For the smaller sample data I posted, it would find an answer if we searched for lines whose entries all appear at least twice, instead of four times.)
You are correct not to worry about repeats within a single line, this is ruled out by construction of the data.
P.s. apologies if I overreacted--I think what was irritating was not that someone would want to make sure my question wasn't homework (I agree that a forum can quickly become useless to experts if it is overrun by homework questions), but instead the order to "please explain the company you work for and the nature of the problem you are working on", not only because it is intrusive, but because it suggests that only people who work for a company with a work-related problem can legitimately ask for scripting assistance here. But: your forum, your rules, ok.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
i have a file a.txt and following is only one portion.
I want to search <branch value="/dev36/AREA/" include="yes"></branch> present in between
<template_file name="Approve External" path="core/approve/bin" and </template_file>
where the no of lines containing "<branch value= " is increasing ... (2 Replies)
Discussion started by: millan
2 Replies
2. Shell Programming and Scripting
My sql file xyz_abc.sql in this file there are multiple sql block in this block I need to find the following block
rem Subset Rows (&&tempName.*)
CREATE VIEW &&tempName.* AS
SELECT *
FROM &&tempName.*
WHERE f is not null
and replace with following code
rem Subset Rows... (9 Replies)
Discussion started by: Zaheer.mic
9 Replies
3. Shell Programming and Scripting
Hi,
I would like to know how to solve one of my problems using expert unix commands.
I have a file with occasional blank lines;
for example;
dertu
frthu
fghtu
frtty
frtgy
frgtui
frgtu
ghrye
frhutp
frjuf
I need to edit the file so that the file looks like this; (10 Replies)
Discussion started by: Lucky Ali
10 Replies
4. UNIX for Dummies Questions & Answers
Hello all, new to unix and have just found the forum.
I think I will be here quite often, and hope that in time i will be able to provide soem help, role on not being a newbie anymore :)
I have a question which iI am hoping someone could help me with.
If i have a file with lines in in thus... (8 Replies)
Discussion started by: grom
8 Replies
5. Shell Programming and Scripting
i have a file containing over 1 million records,and i want to print about 300,000 line containing a some specific words.
file has content.
eg
1,rrt,234
3,fgt,678
4,crf,456
5,cde,drt
6,cfg,123
and i want to print the line with the word fgt,crf
this is just an example,my file is so... (2 Replies)
Discussion started by: tomjones
2 Replies
6. UNIX for Dummies Questions & Answers
Hi Experts,
I have a file which contains some text. i need to print the word next to a given keyword. Please help.
Ex: test.txt
=====================
NEXT HOST
=====================
AEADBAS001
access-list 1 permit xxxxxxxxxxxxxx
ip access-list extended BLA_Outgoing_Filter... (6 Replies)
Discussion started by: mwrg
6 Replies
7. Shell Programming and Scripting
i need to get least lines of file after last word TEST in file, and send that lines to mail
example of file structure:
TEST 10.10.2010
jdfjdnjfndjfndnfkdk
djfjdnfjkdjkfnjkdfk
jdfjdjfnjdjnfjkdnfjk
TEST 11.10.2010
jdjfnjdnfdkdfjdfjdnk
jdnfjdnjkfndnfjdnfjk
fjdnfjkndnfdfnjdnfjk
TEST... (6 Replies)
Discussion started by: waso
6 Replies
8. Shell Programming and Scripting
Hi All,
I want to read all lines after a perticular string {SET UP VALUES}apprears in the file.
SET UP values contains direcory, number of days and file type.
Step1: Read all lines below SET UP VALUES string.
Step2: If set up values are not present in each record then read from default... (4 Replies)
Discussion started by: Nagaraja Akkiva
4 Replies
9. UNIX for Advanced & Expert Users
Hi ,
i have a file with data as below.This is same file. But actual file contains to many rows.
i want to search for a string "Field 039 00" and delete that line and previous 3 lines in that file.. Can some body suggested me how can i do using either sed or awk command ?
Field 004... (7 Replies)
Discussion started by: vadlamudy
7 Replies
10. UNIX for Beginners Questions & Answers
Guys i need an idea for one logic..in shell scripting am struggling with a logic...So the thing is... i need to search for a word in a huge log file and i need to continue to print few more lines from that line and the consecutive line has to end when it finds the line with date..because i know... (1 Reply)
Discussion started by: Prathi
1 Replies
rl(1) User Commands rl(1)
NAME
rl - Randomize Lines.
SYNOPSIS
rl [OPTION]... [FILE]...
DESCRIPTION
rl reads lines from a input file or stdin, randomizes the lines and outputs a specified number of lines. It does this with only a single
pass over the input while trying to use as little memory as possible.
-c, --count=N
Select the number of lines to be returned in the output. If this argument is omitted all the lines in the file will be returned in
random order. If the input contains less lines than specified and the --reselect option below is not specified a warning is printed
and all lines are returned in random order.
-r, --reselect
When using this option a single line may be selected multiple times. The default behaviour is that any input line will only be
selected once. This option makes it possible to specify a --count option with more lines than the file actually holds.
-o, --output=FILE
Send randomized lines to FILE instead of stdout.
-d, --delimiter=DELIM
Use specified character as a "line" delimiter instead of the newline character.
-0, --null
Input lines are terminated by a null character. This option is useful to process the output of the GNU find -print0 option.
-n, --line-number
Output lines are numbered with the line number from the input file.
-q, --quiet, --silent
Be quiet about any errors or warnings.
-h, --help
Show short summary of options.
-v, --version
Show version of program.
EXAMPLES
Some simple demonstrations of how rl can help you do everyday tasks.
Play a random sound after 4 minutes (perfect for toast):
sleep 240 ; play `find /sounds -name '*.au' -print | rl --count=1`
Play the 15 most recent .mp3 files in random order.
ls -c *.mp3 | head -n 15 | rl | xargs --delimiter='
' play
Roll a dice:
seq 6 | rl --count 2
Roll a dice 1000 times and see which number comes up more often:
seq 6 | rl --reselect --count 1000 | sort | uniq -c | sort -n
Shuffle the words of a sentence:
echo -n "The rain in Spain stays mainly in the plain."
| rl --delimiter=' ';echo
Find all movies and play them in random order.
find . -name '*.avi' -print0 | rl -0 | xargs -n 1 -0 mplayer
Because -0 is used filenames with spaces (even newlines and other unusual characters) in them work.
BUGS
The program currently does not have very smart memory management. If you feed it huge files and expect it to fully randomize all lines it
will completely read the file in memory. If you specify the --count option it will only use the memory required for storing the specified
number of lines. Improvements on this area are on the TODO list.
The program uses the rand() system random function. This function returns a number between 0 and RAND_MAX, which may not be very large on
some systems. This will result in non-random results for files containing more lines than RAND_MAX.
Note that if you specify multiple input files they are randomized per file. This is a different result from when you cat all the files and
pipe the result into rl.
COPYRIGHT
Copyright (C) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008 Arthur de Jong.
This is free software; see the license for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Version 0.2.7 Jul 2008 rl(1)