|
|
Sponsored Content
Top Forums
Shell Programming and Scripting
Shell script to find longest phrase
Post 302290634 by stargazerr on Monday 23rd of February 2009 10:24:29 PM
02-23-2009
The text file is nothing but a news article after parsing HTML tags and extracting the content using XML. I use the following to extract and print the most bigrams for now.
tr -sc 'a-zA-z0-9.' '\012' < $1 > bigrams1
tail -n+2 bigrams1 > bigrams2
paste bigrams1 bigrams2
Here $1 is the name of the file containing the actual text. I am passing this as an argument for now.
Thing is, detecting bigrams and trigrams is easy. Is there a way to detect the longest phrase that appears at least twice ? It could be a bigram, a trigram or n-gram.
Thanks.
SG
tr -sc 'a-zA-z0-9.' '\012' < $1 > bigrams1
tail -n+2 bigrams1 > bigrams2
paste bigrams1 bigrams2
Here $1 is the name of the file containing the actual text. I am passing this as an argument for now.
Thing is, detecting bigrams and trigrams is easy. Is there a way to detect the longest phrase that appears at least twice ? It could be a bigram, a trigram or n-gram.
Thanks.
SG
|
|
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Dear All,
To find the length of the longest line from a file i have used wc -L which is giving the proper output...
But the problem is AIX os does not support wc -L command.
so is there any other way 2 to find out the length of the longest line using awk or sed ?
Regards,
Pankaj (1 Reply)
Discussion started by: panknil
1 Replies
2. Shell Programming and Scripting
Okie here is my problem,
1. I have a directory with a ton of files.
2. I want to first get an input on how many days ago the files were created.
3. I will take those files and put it into another file
4. Then I will take the last # from each line and subtract by 1 then diff the line from the... (1 Reply)
Discussion started by: bigboizvince
1 Replies
3. Shell Programming and Scripting
I've got a script which finds *.txt files in directories and subdirectories after providing the path by the user and then searches in the files for phrase given by the user
How to write script in such way that the paths to the found *.txt files and the phrase given by the user were both... (2 Replies)
Discussion started by: patrykxes
2 Replies
4. Shell Programming and Scripting
Return the position of matched string from right, awk match can do from left only.
e.g return pos 7 for search string "service" from "AA-service"
or return the matched string "service", then caculate the string length.
Thanks!. (3 Replies)
Discussion started by: honglus
3 Replies
5. Shell Programming and Scripting
Hello everyone...
I need to find out, how to find longest line or possibly lines in several files which are arguments for script. The thing is, that I tried some possibilities before, but nothing worked correctly.
Example
when i use:
awk ' { if ( length > L ) { L=length ;s=$0 } }END{ print... (23 Replies)
Discussion started by: 1tempus1
23 Replies
6. Shell Programming and Scripting
Hello all,
I need to find the longest string in a select field and print that field.
I have tried a few different methods and I always end up one step from where I need to be.
Methods thus far:
nawk '{if (length($1) > long) long=length($1); if(length($1)==long) print $1}'
The above... (6 Replies)
Discussion started by: SEinT
6 Replies
7. Shell Programming and Scripting
Hello
My question is: How to find out the shell of the shell script which we are running? I am writing a script, say f1.sh, as below:
#!/bin/ksh
echo "Sample script"
From the first line, we can say this script will run in ksh. But, how can we prove it? Can we print anything inside... (6 Replies)
Discussion started by: guruprasadpr
6 Replies
8. Shell Programming and Scripting
I want to make a script which takes the number of argument, add those argument and gives output to the user, but I am not getting through...
Script that i am using is below :
#!/bin/bash
sum=0
for i in $@
do
sum=$sum+$1
echo $sum
shift
done
I am executing the script as... (3 Replies)
Discussion started by: mukulverma2408
3 Replies
9. Shell Programming and Scripting
I want to burst a report by using the page number value in the report header. Each section starts with *PAGE NO:* 1 Each section might have several pages, but the next section always starts back at 1.
So I want to find the "*PAGE NO:* 1" value and pull all lines that follow until "*PAGE NO:* 1"... (4 Replies)
Discussion started by: Scottie1954
4 Replies
10. Shell Programming and Scripting
Hello,
I am looking for a shell script that can
1- take as input a variable, like "server.cpu"
2- do a search for that variable in a directory that contains subdirectories.
The search will start at the last subdirectory working up to the top level if I can not find the file
3-... (7 Replies)
Discussion started by: georg2014
7 Replies
LEARN ABOUT DEBIAN
nwdiag
NWDIAG(1) General Commands Manual NWDIAG(1) NAME
nwdiag - generate network-diagram image file from spec-text file. SYNOPSIS
nwdiag [options] files DESCRIPTION
This manual page documents briefly the nwdiag commands. nwdiag is generate sequence-diagram image file from spec-text file. OPTIONS
These programs follow the usual GNU command line syntax, with long options starting with two dashes (`-'). A summary of options is included below. For a complete description, see the Info files. -h, --help show this help message and exit. --version show program's version number and exit. -a, --antialias Pass diagram image to anti-alias filter. -c FILE, --config=FILE read configurations from FILE. -o FILE write diagram to FILE. -f FONT, --font=FONT use FONT to draw diagram. -T TYPE Output diagram as TYPE format. SEE ALSO
The programs are documented fully by http://blockdiag.com/en/nwdiag/ AUTHOR
nwdiag was written by Takeshi Komiya <i.tkomiya@gmail.com> This manual page was written by Kouhei Maeda <mkouhei@palmtb.net>, for the Debian project (and may be used by others). June 11, 2011 NWDIAG(1)