Sponsored Content
Top Forums Shell Programming and Scripting finding and removing 2 identical consecutive words in a text Post 302518689 by ctsgnb on Sunday 1st of May 2011 09:28:11 AM
Old 05-01-2011
Code:
awk '{for (i=2;i<=NF;i++) while($(i+1)==$i){sub($i FS $(i+1),$i,$0)}}1' infile

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing identical words in column

I have a file that needs to be cleaned up. Here is the file: Project Project John Project Gary Project Sean Project2 Project2 Lisa Project2 Tyler Project2 Sam Project3 Project3 Mike Project3 Bran I need the o/p to be: Project John Gary Sean Project2 (7 Replies)
Discussion started by: leepet01
7 Replies

2. SuSE

finding and removing block of identical strings

i have a problem in finding block of identical strings...i solved the problem in finding consecutive identical words and now i want to expand the code in order to find and remove consecutive identical block of strings... for example the awk code removing consecutive identical word is:... (2 Replies)
Discussion started by: cocostaec
2 Replies

3. Programming

finding and removing block of identical strings

i have a problem in finding block of identical strings...i solved the problem in finding consecutive identical words and now i want to expand the code in order to find and remove consecutive identical block of strings... for example the awk code removing consecutive identical word is:... (2 Replies)
Discussion started by: cocostaec
2 Replies

4. Shell Programming and Scripting

finding and removing block of identical strings

i have a problem in finding block of identical strings...i solved the problem in finding consecutive identical words and now i want to expand the code in order to find and remove consecutive identical block of strings... for example the awk code removing consecutive identical word is:... (2 Replies)
Discussion started by: cocostaec
2 Replies

5. UNIX for Dummies Questions & Answers

deleting words in list with more than 2 identical adjacent characters

Morning Guys & Gals, I am trying to figure out a way to remove lines from a file that have more than 2 identical characters in sequence.. So if for instance the list would look like ; the output would be ; I can't seem to get my head around perl (among many other... (7 Replies)
Discussion started by: TAPE
7 Replies

6. Shell Programming and Scripting

Finding consecutive same words in a file

Hi All, I tried this but I am having trouble formulating this: I have a file that looks like this (this is a sample file words can be different): network router frame network router computer card host computer card One can see that in this file "network" and "router" occur... (3 Replies)
Discussion started by: shoaibjameel123
3 Replies

7. Shell Programming and Scripting

Removing consecutive lines in a file

We have very large transaction logs that have transactions which start with a line that starts with 'Begin :' and ends with a line that starts with 'End :'. For most transactions there is valid data between those two lines. I am trying to get rid of lines that look like this: Begin :... (11 Replies)
Discussion started by: deneuve01
11 Replies

8. Shell Programming and Scripting

Scan a file in realtime and execute certain commands on encountering 5 consecutive identical lines

Mysql log has something like below: I need a bash shell script that will do the following: 1) The script will scan the mysql.log file constantly in real time (something like tail -F mysql.log) 2) If it encounters 5 consecutive identical lines then it would invoke some commands (say... (4 Replies)
Discussion started by: proactiveaditya
4 Replies

9. Shell Programming and Scripting

Get group of consecutive uppercase words using gawk

Hi I'd like to extract, from a text file, the strings starting with "The Thing" and only composed of words with a capital first letter and apostrophes, like for example: "The Thing I Only" from "those are the The Thing I Only go for whatever." or "The Thing That Are Like Men's Eyewear" ... (7 Replies)
Discussion started by: louisJ
7 Replies

10. UNIX for Beginners Questions & Answers

awk - If field value of consecutive records are the identical print portion of lines

I have some data that looks like this: PXD= ZW< 1,6 QR> QRJ== 1,2(5) QR> QRJ== 4,1(2) QR> QRJ== 4,2 QRB= QRB 4,2 QWM QWM 6,2 R<C ZW< 11,2 R<H= R<J= 6,1 R>H XZJ= 1,2(2) R>H XZJ= 2,6(2) R>H XZJ= 4,1(2) R>H XZJ= 6,2 RDP RDP 1,2 What I would like to do is if fields $1 and $2 are... (5 Replies)
Discussion started by: jvoot
5 Replies
PDF2TXT(1)							  PDFMiner Manual							PDF2TXT(1)

NAME
pdf2txt - extracts text contents of PDF files SYNOPSIS
pdf2txt [option...] file... DESCRIPTION
pdf2txt extracts text contents from a PDF file. It extracts all the text that is to be rendered programmatically, i.e. text represented as ASCII or Unicode strings. It cannot recognize text drawn as images that would require optical character recognition. It also extracts the corresponding locations, font names, font sizes, writing direction (horizontal or vertical) for each text portion. You need to provide a password for protected PDF documents when its access is restricted. You cannot extract any text from a PDF document which does not have extraction permission. OPTIONS
-o file Specifies the output file name. The default is to print the extracted contents to standand output in text format. -p pageno[,pageno,...] Specifies the comma-separated list of the page numbers to be extracted. Page numbers start at one. By default, it extracts text from all the pages. -c codec Specifies the output codec. -t type Specifies the output format. The following formats are currently supported: text Text format. This is the default. html HTML format. It is not recommended. xml XML format. It provides the most information. tag "Tagged PDF" format. A tagged PDF has its own contents annotated with HTML-like tags. pdf2txt tries to extract its content streams rather than inferring its text locations. Tags used here are defined in the PDF Reference, Sixth Edition[1] (S10.7 "Tagged PDF"). -D writing-mode Specifies the writing mode of text outputs: lr-tb Left-to-right, top-to-bottom. tb-rl Top-to-bottom, right-to-left. auto Determine writing mode automatically -M char-margin, -L line-margin, -W word-margin These are the parameters used for layout analysis. In an actual PDF file, text portions might be split into several chunks in the middle of its running, depending on the authoring software. Therefore, text extraction needs to splice text chunks. In the figure below, two text chunks whose distance is closer than the char-margin is considered continuous and get grouped into one. Also, two lines whose distance is closer than the line-margin is grouped as a text box, which is a rectangular area that contains a "cluster" of text portions. Furthermore, it may be required to insert blank characters (spaces) as necessary if the distance between two words is greater than the word-margin, as a blank between words might not be represented as a space, but indicated by the positioning of each word. Each value is specified not as an actual length, but as a proportion of the length to the size of each character in question. The default values are char-margin = 1.0, line-margin = 0.3, and W = 0.2, respectively. -n Suppress layout analysis. -A Force layout analysis for all the text strings, including text contained in figures. -V Enable detection of vertical writing. -s scale Specifies the output scale. This option can be used in HTML format only. -m n Specifies the maximum number of pages to extract. By default, all the pages in a document are extracted. -P password Provides the user password to access PDF contents. -d Increase the debug level. EXAMPLES
Extract text as an HTML file whose filename is output.html: $ pdf2txt -o output.html samples/naacl06-shinyama.pdf Extract a Japanese HTML file in vertical writing: $ pdf2txt -c euc-jp -D tb-rl -o output.html samples/jo.pdf Extract text from an encrypted PDF file: $ pdf2txt -P mypassword -o output.txt secret.pdf SEE ALSO
dumppdf(1) AUTHORS
Jakub Wilk <jwilk@debian.org> Wrote this manual page for the Debian system. Yusuke Shinyama <yusuke@cs.nyu.edu> Author of PDFMiner and its original HTML documentation. NOTES
1. PDF Reference, Sixth Edition http://www.adobe.com/devnet/acrobat/pdfs/pdf_reference_1-7.pdf pdf2txt 08/24/2011 PDF2TXT(1)
All times are GMT -4. The time now is 09:51 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy