I'm trying to capture an english sentence in a record.
This sentence could be very long and span multiple lines in a file.
My perfect record separator would be a period, exclamation point, question mark, semicolon or colon.
However, my code sees the word "Mr." it thinks that's the end of the sentence because of the period that is part of Mr. So I want it detect that "Mr." is NOT part of the record separator.
Semantically:
Not (Mr.) but ok with any of these [.!?;:]
But syntactically I don't know how to do this, I'm trying like this:
Code:
RS = (^Mr. | [.!?;:])
But its not working?
Moderator's Comments:
Please use CODE tags when displaying sample input, output, and code segments (as required by forum rules).
Last edited by Don Cragun; 08-08-2017 at 01:15 AM..
Reason: Add CODE tags.
Ive got a file with words and also numbers.
Bla BLA
10 10
11 29
12 89
13 35
And i need to change "10,29,89,25" and also remove anything that contains actually words... (4 Replies)
I can print a line with an expression using this:
awk '/regex/'
I can print the line immediately before an expression using this:
awk '/regex/{print x};{x=$0}'
How do I print the line immediately before and then the line with the expression? (2 Replies)
Hi All,
I've got some strange behaviour going on when trying to manipulate a file that contains spaces.
My input file looks something like this:
xxxxxxxxx,yyyy,sss sss sss,bbbbbbb
If I use awk:
When running from the command line I get:
sss sss sss
But when running from a... (7 Replies)
Hello world,
I was wondering if there is a nicer way to write the following code (in AWK):
awk '
FNR==NR&&$1~/^m$/{tok1=1}
FNR==NR&&$1~/^m10$/{tok1=1}
' my_file
In fact, it looks for m2, m4, m6, m8 and m10 and then return a positive flag. The problem is how to define 10 thanks... (3 Replies)
I'm working on a different stage of a project that someone helped me address elsewhere in these threads.
The .docs I'm cycling through look roughly like this:
1 of 26 DOCUMENTS
Copyright 2010 The Age Company Limited
All Rights Reserved
The Age (Melbourne, Australia)
November 27, 2010... (9 Replies)
Hi,
I have a bunch of records within a directory where each one has this form:
(example file1)
1 2 50 90 80 90 43512 98 0909 79869 -9 7878 33222 8787 9090 89898 7878 8989 7878 6767 89 89 78676 9898 000 7878 5656 5454 5454
and i want for all of these files to be... (3 Replies)
How do I use single quotes as record separator in awk?
I just couldn't figure that out. I know how to use single quotes as field separator, and double quotes as both field and record separator ... (1 Reply)
Hi all,
How am I read a file, find the match regular expression and overwrite to the same files.
open DESTINATION_FILE, "<tmptravl.dat" or die "tmptravl.dat";
open NEW_DESTINATION_FILE, ">new_tmptravl.dat" or die "new_tmptravl.dat";
while (<DESTINATION_FILE>)
{
# print... (1 Reply)
Hello to all,
Please some help on this. I have the file in format as below.
How can I set the record separator as the string below in red
"No. Time Source Destination Protocol Length Info"
I've tried code below but it doesn't seem to... (6 Replies)
Discussion started by: cgkmal
6 Replies
LEARN ABOUT DEBIAN
ucto
ucto(1) General Commands Manual ucto(1)NAME
ucto - Unicode Tokenizer
SYNOPSYS
ucto [[options]] [input-file] [[output-file]]
DESCRIPTION
ucto ucto tokenizes text files: it separates words from punctuation, splits sentences (and optionally paragraphs), and finds paired quotes.
Ucto is preconfigured with tokenisation rules for several languages.
OPTIONS -c configfile
read settings from a file
-d value
set debug mode to 'value'
-e value
set input encoding. (default UTF8)
-f
disable filtering of special characters
-L language
Automatically selects a configuration file by language code. e.g. 'fr' will select the file tokconfig-fr from the installation
directory
-l
Convert to all lowercase
-u
Convert to all uppercase
-n
Assume one sentence per line on input
-m
Emit one sentence per line on output
--passthru
Don't tokenize, but perform input decoding and simple token role detection
-P
Disable Paragraph Detection
-Q
Enable Quote Detection. (this is experimental and may lead to unexpected results)
-S
Disable Sentence Detection
-s <string>
Set End-of-sentence marker. (Default <utt>)
-V
Show version information
-v
set Verbose mode
-x <DocId>
Output FoLiA XML, use the specified Document ID. (this disables usage of most other options: -nulPQvsS)
-F
Read a FoLiA XML document, tokenize it, and output the modified doc. (this disables usage of most other options: -nulPQvsS)
BUGS
likely
AUTHORS
Maarten van Gompel proycon@anaproy.nl
Ko van der Sloot Timbl@uvt.nl
2011 november 28 ucto(1)
08-08-2017
