remove html tags,consecutive duplicate lines Post: 302527058

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

how to remove duplicate lines

I have following file content (3 fields each line): 23 888 10.0.0.1 dfh 787 10.0.0.2 dssf dgfas 10.0.0.3 dsgas dg 10.0.0.4 df dasa 10.0.0.5 df dag 10.0.0.5 dfd dfdas 10.0.0.5 dfd dfd 10.0.0.6 daf nfd 10.0.0.6 ... as can be seen, that the third field is ip address and sorted. but...

2. Shell Programming and Scripting

Remove duplicate lines

Hi, I have a huge file which is about 50GB. There are many lines. The file format likes 21 rs885550 0 9887804 C C T C C C C C C C 21 rs210498 0 9928860 0 0 C C 0 0 0 0 0 0 21 rs303304 0 9941889 A A A A A A A A A A 22 rs303304 0 9941890 0 A A A A A A A A A The question is that there are a few...

3. Shell Programming and Scripting

remove consecutive duplicate rows

I have some data that looks like, 1 3300665.mol 3300665 5177008 102.093 2 3300665.mol 3300665 5177008 102.093 3 3294015.mol 3294015 5131552 102.114 4 3294015.mol 3294015 5131552 102.114 5 3293734.mol 3293734 5129625 104.152 6 3293734.mol ...

4. Shell Programming and Scripting

Remove lines with duplicate first field

Trying to cut down the size of some log files. Now that I write this out it looks more dificult than i thought it would be. Need a bash script or command that goes sequentially through all lines of a file, and does this: if field1 (space separated) is the number 2012 print the entire line. Do...

5. Homework & Coursework Questions

Script: Removing HTML tags and duplicate lines

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: You will write a script that will remove all HTML tags from an HTML document and remove any consecutive...

6. UNIX for Dummies Questions & Answers

Remove Duplicate Lines

Hi I need this output. Thanks. Input: TAZ YET FOO FOO VAK TAZ BAR Output: YET VAK BAR

7. Shell Programming and Scripting

Remove duplicate lines from a file

Hi, I have a csv file which contains some millions of lines in it. The first line(Header) repeats at every 50000th line. I want to remove all the duplicate headers from the second occurance(should not remove the first line). I don't want to use any pattern from the Header as I have some...

8. Shell Programming and Scripting

Check/print missing number in a consecutive range and remove duplicate numbers

Hi, In an ideal scenario, I will have a listing of db transaction log that gets copied to a DR site and if I have them all, they will be numbered consecutively like below. 1_79811_01234567.arc 1_79812_01234567.arc 1_79813_01234567.arc 1_79814_01234567.arc 1_79815_01234567.arc...

9. Shell Programming and Scripting

How to remove duplicate lines?

Hi All, I am storing the result in the variable result_text using the below code. result_text=$(printf "$result_text\t\n$name") The result_text is having the below text. Which is having duplicate lines. file and time for the interval 03:30 - 03:45 file and time for the interval 03:30 - 03:45 ...

10. Shell Programming and Scripting

Remove duplicate consecutive lines with specific string

Hello, I'm trying to remove the duplicate consecutive lines with specific string "WARNING". File.txt abc; WARNING 2345 WARNING 2345 WARNING 2345 WARNING 2345 WARNING 2345 bcd; abc; 123 123 123 WARNING 1234 WARNING 2345 WARNING 2345 efgh;

LEARN ABOUT SUSE

html::clean

HTML::Clean(3)						User Contributed Perl Documentation					    HTML::Clean(3)

NAME

       HTML::Clean - Cleans up HTML code for web browsers, not humans

SYNOPSIS

	 use HTML::Clean;
	 $h = new HTML::Clean($filename); # or..
	 $h = new HTML::Clean($htmlcode);

	 $h->compat();
	 $h->strip();
	 $data = $h->data();
	 print $$data;

DESCRIPTION

       The HTML::Clean module encapsulates a number of common techniques for minimizing the size of HTML files.  You can typically save between
       10% and 50% of the size of a HTML file using these methods.  It provides the following features:

       Remove unneeded whitespace (begining of line, etc)
       Remove unneeded META elements.
       Remove HTML comments (except for styles, javascript and SSI)
       Replace tags with equivilant shorter tags (<strong> --> <b>)
       etc.

       The entire proces is configurable, so you can pick and choose what you want to clean.

THE HTML
::Clean CLASS
   $h = new HTML::Clean($dataorfile, [$level]);
       This creates a new HTML::Clean object.  A Prerequisite for all other functions in this module.

       The $dataorfile parameter supplies the input HTML, either a filename, or a reference to a scalar value holding the HTML, for example:

	 $h = new HTML::Clean("/htdocs/index.html");
	 $html = "<strong>Hello!</strong>";
	 $h = new HTML::Clean($html);

       An optional 'level' parameter controls the level of optimization performed.  Levels range from 1 to 9.  Level 1 includes only simple fast
       optimizations.  Level 9 includes all optimizations.

   $h->initialize($dataorfile)
       This function allows you to reinitialize the HTML data used by the current object.  This is useful if you are processing many files.

       $dataorfile has the same usage as the new method.

       Return 0 for an error, 1 for success.

   $h->level([$level])
       Get/set the optimization level.	$level is a number from 1 to 9.

   $myref = $h->data()
       Returns the current HTML data as a scalar reference.

   strip(\%options);
       Removes excess space from HTML

       You can control the optimizations used by specifying them in the %options hash reference.

       The following options are recognized:

       boolean values (0 or 1 values)
		 whitespace    Remove excess whitespace
		 shortertags   <strong> -> <b>, etc..
		 blink	       No blink tags.
		 contenttype   Remove default contenttype.
		 comments      Remove excess comments.
		 entities      &quot; -> ", etc.
		 dequote       remove quotes from tag parameters where possible.
		 defcolor      recode colors in shorter form. (#ffffff -> white, etc.)
		 javascript    remove excess spaces and newlines in javascript code.
		 htmldefaults  remove default values for some html tags
		 lowercasetags translate all HTML tags to lowercase

       parameterized values
		 meta	     Takes a space separated list of meta tags to remove,
			     default "GENERATOR FORMATTER"

		 emptytags   Takes a space separated list of tags to remove when there is no
			     content between the start and end tag, like this: <b></b>.
			     The default is 'b i font center'

   compat()
       This function improves the cross-platform compatibility of your HTML.  Currently checks for the following problems:

       Insuring all IMG tags have ALT elements.
       Use of Arial, Futura, or Verdana as a font face.
       Positioning the <TITLE> tag immediately after the <head> tag.

   defrontpage();
       This function converts pages created with Microsoft Frontpage to something a Unix server will understand a bit better.  This function
       currently does the following:

       Converts Frontpage 'hit counters' into a unix specific format.
       Removes some frontpage specific html comments

SEE ALSO

   Modules
       FrontPage::Web, FrontPage::File

   Web Sites
       Distribution Site - http://people.itu.int/~lindner/

AUTHORS

       Paul Lindner for the International Telecommunication Union (ITU)

COPYRIGHT

       The HTML::Strip module is Copyright (c) 1998,99 by the ITU, Geneva Switzerland.	All rights reserved.

       You may distribute under the terms of either the GNU General Public License or the Artistic License, as specified in the Perl README file.

POD ERRORS

       Hey! The above document had some coding errors, which are explained below:

       Around line 66:
	   You forgot a '=back' before '=head2'

       Around line 658:
	   =back without =over

perl v5.12.1							    2010-07-05							    HTML::Clean(3)

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

how to remove duplicate lines

Discussion started by: fredao

2. Shell Programming and Scripting

Remove duplicate lines

Discussion started by: zhshqzyc

3. Shell Programming and Scripting

remove consecutive duplicate rows

Discussion started by: LMHmedchem

4. Shell Programming and Scripting

Remove lines with duplicate first field

Discussion started by: ajp7701