Sponsored Content
Top Forums Shell Programming and Scripting Extract all the sentences from a text file that matches a pattern list Post 302948036 by RudiC on Thursday 25th of June 2015 03:13:20 AM
Old 06-25-2015
Let me say first that there's incredibly refined and sophisticated algorithms out there, used by e.g. the various search engines to analyse all the internet sites around the globe and to hand you the results in a split second, so anything posted here is a clumsy approach cobbled together without any optimisation. Anyhow, try
Code:
awk '
FNR==NR         {T[$1]
                 next
                }
                {CNT=0
                 n=split (tolower($0), L)
                 for (i=1; i<=n; i++) if (L[i] in T) CNT++
#                print CNT, CNT/n
                 if (CNT/n >= 0.8) print $0
                }
' list text

You may want/need to get rid of punctuation first in a real world sample.
This User Gave Thanks to RudiC For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extract if pattern matches

Hi All, I have an input below. I tried to use the awk below but it seems that it ;s not working. Can anybody help ? My concept here is to find the 2nd field of the last occurrence of such pattern " ** XXX ccc ccc cc cc ccc 2007 " . In this case, the 2nd field is " XXX ". With this "XXX" term... (20 Replies)
Discussion started by: Raynon
20 Replies

2. Programming

How to extract a sentences of word from a text file.

Hi , i have a text file that contain a story How do i extract the out all the sentences that contain the word Mon. in C++ I only want to show those sentences that contain the word mon eg. Monkey on a tree. Rabbit jumping around the tree. I am very rich, I have lots of money. Today... (1 Reply)
Discussion started by: xiaojesus
1 Replies

3. Shell Programming and Scripting

sed: Find start of pattern and extract text to end of line, including the pattern

This is my first post, please be nice. I have tried to google and read different tutorials. The task at hand is: Input file input.txt (example) abc123defhij-E-1234jslo 456ujs-W-abXjklp From this file the task is to grep the -E- and -W- strings that are unique and write a new file... (5 Replies)
Discussion started by: TestTomas
5 Replies

4. Shell Programming and Scripting

extract unique pattern from large text file

Hi All, I am trying to extract data from a large text file , I want to extract lines which contains a five digit number followed by a hyphen , like 12345- , i tried with egrep ,eg : egrep "+" text.txt but which returns all the lines which contains any number of digits followed by hyhen ,... (19 Replies)
Discussion started by: shijujoe
19 Replies

5. Shell Programming and Scripting

Extract list of IP addresses from a text file.

I have an xml file with IP addresses all over the show. I want to print only the IP addresses and cut off any text before or after the IP address. Example: Note: The IP addresses (x.x.x.x) do not consistently appear in the xml file as per the pattern below. Sometimes there are text before... (8 Replies)
Discussion started by: lewk
8 Replies

6. Shell Programming and Scripting

Blocks of text in a file - extract when matches...

I sat down yesterday to write this script and have just realised that my methodology is broken........ In essense I have..... ----------------------------------------------------------------- (This line really is in the file) Service ID: 12345 ... (7 Replies)
Discussion started by: Bashingaway
7 Replies

7. Shell Programming and Scripting

Extract specific line in an html file starting and ending with specific pattern to a text file

Hi This is my first post and I'm just a beginner. So please be nice to me. I have a couple of html files where a pattern beginning with "http://www.site.com" and ending with "/resource.dat" is present on every 241st line. How do I extract this to a new text file? I have tried sed -n 241,241p... (13 Replies)
Discussion started by: dejavo
13 Replies

8. Shell Programming and Scripting

Extract sentence and its details from a text file based on another file of sentences

Hi I have two text files. The first file is TEXTFILEONE.txt as given below: <Text Text_ID="10155645315851111_10155645333076543" From="460350337461111" Created="2011-03-16T17:05:37+0000" use_count="123">This is the first text</Text> <Text Text_ID="10155645315851111_10155645317023456"... (7 Replies)
Discussion started by: my_Perl
7 Replies

9. UNIX for Beginners Questions & Answers

Extract the whole set if a pattern matches

Hi, I have to extract the whole set if a pattern matches.i have a file called input.txt input.txt ------------ CREATE TABLE ABC ( A, B, C ); CREATE TABLE XYZ ( X, Y, Z, P, Q ); (6 Replies)
Discussion started by: raju2016
6 Replies

10. Shell Programming and Scripting

Match text to lines in a file, iterate backwards until text or text substring matches, print to file

hi all, trying this using shell/bash with sed/awk/grep I have two files, one containing one column, the other containing multiple columns (comma delimited). file1.txt abc12345 def12345 ghi54321 ... file2.txt abc1,text1,texta abc,text2,textb def123,text3,textc gh,text4,textd... (6 Replies)
Discussion started by: shogun1970
6 Replies
Text::Greeking(3pm)					User Contributed Perl Documentation				       Text::Greeking(3pm)

NAME
Text::Greeking - a module for generating meaningless text that creates the illusion of the finished document. SYNOPSIS
#!/usr/bin/perl -w use strict; use Text::Greeking; my $g = Text::Greeking->new; $g->paragraphs(1,2) # min of 1 paragraph and a max of 2 $g->sentences(2,5) # min of 2 sentences per paragraph and a max of 5 $g->words(8,16) # min of 8 words per sentence and a max of 16 print $g->generate; # use default Lorem Ipsum source DESCRIPTION
Greeking is the use of random letters or marks to show the overall appearance of a printed page without showing the actual text. Greeking is used to make it easy to judge the overall appearance of a document without being distracted by the meaning of the text. This is a module is for quickly generating varying meaningless text from any source to create this illusion of the content in systems. This module was created to quickly give developers simulated content to fill systems with simulated content. Instead of static Lorem Ipsum text, by using randomly generated text and optionally varying word sources, repetitive and monotonous patterns that do not represent real system usage is avoided. METHODS
Text::Greeking->new Constructor method. Returns a new instance of the class. $g->init Initializes object with defaults. Called by the constructor. Broken out for easy overloading to enable customized defaults and other behaviour. $g->sources([@ARRAY]) Gets/sets the table of source word collections current in memory as an ARRAY reference $g->add_source($text) The class takes a body of text passed as a SCALAR and processes it into a list of word tokens for use in generating random filler text later. $g->generate Returns a body of random text generated from a randomly selected source using the minimum and maximum values set by paragraphs, sentences, and words minimum and maximum values. If generate is called without any sources a standard Lorem Ipsum block is used added to the sources and then used for processing the random text. $g->paragraphs($min,$max) Sets the minimum and maximum number of paragraphs to generate. Default is a minimum of 2 and a maximum of 8. $g->sentences($min,$max) Sets the minimum and maximum number of sentences to generate per paragraph. Default is a minimum of 2 and a maximum of 8. $g->words($min,$max) Sets the minimum and maximum number of words to generate per sentence. Default is a minimum of 5 and a maximum of 15. SEE ALSO
http://en.wikipedia.org/wiki/Greeking TO DO
HTML output mode including random hyperlinked phrases. Configurable punctuation controls. PARTICIPATION
I welcome and accept patches in diff format. If you wish to hack on this code, please fork the git repository found at: <http://github.com/tima/perl-text-greeking/>. If you have something to push back to my repository, just use the "pull request" button on the github site. LICENSE
The software is released under the Artistic License. The terms of the Artistic License are described at <http://www.perl.com/language/misc/Artistic.html>. AUTHOR &; COPYRIGHT Except where otherwise noted, Text::Greeking is Copyright 2005-2009, Timothy Appnel, tima@cpan.org. All rights reserved. perl v5.10.0 2009-08-28 Text::Greeking(3pm)
All times are GMT -4. The time now is 01:31 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy