Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: How to join several HTML files?
Operating Systems OS X (Apple) How to join several HTML files? Post 302968532 by bakunin on Thursday 10th of March 2016 11:33:19 AM
Old 03-10-2016
HTML-documents follow a certain structure:

Code:
<HTML>
     <HEAD>
          ....all sorts of text, META-tags, etc. ...
     </HEAD>
     <BODY>
          ....all sorts of text, the real content of your document ...
     </BODY>
</HTML>

In a first step I'd start to extract everything within the <body>...</body>-tags (the "content" in a narrower sense) and put that into into one document, like that:

Code:
<HTML>
     <HEAD>
          ...maybe use the content fo the HEAD-tag from the first document here..
     </HEAD>
     <BODY>
          ....everything between the BODY-tags from the first document....
          ....everything between the BODY-tags from the second document....
          ....everything between the BODY-tags from the third document....
          ...
     </BODY>
</HTML>

Here is a script that should do that. Note that it might fail because to fully "understand" HTML as a language it would have to use a recursive parser, which is too much effort to put it into a casual solution here. It should provide a starting point for you, though.

Note that you might want to refine especially the parameter handling, right now it uses the bare minimum. The same goes for error handling (mistyped file names, ...). Call it like:

Code:
$ script file1 file2 file3 .... > outputfile

Code:
#! /bin/ksh

typeset    fIn=""                        # input filename
typeset -i lFirst=1                      #  flag for first file

while [ -n "$1" ] ; do
     fIn="$1"                            # get next file from input
     shift

     if (( lFirst )) ; then              # first file is copied up to "</body>"
          sed '/<\/body>/ {
                    s/<\/body>.*//
                    :loop
                       n
                       /.*/d
                    b loop
                    }' "$fIn"
          lFirst=0
     else
                                         # subsequent files, only the content
                                         # of "<body>...</body>"
          sed -n '/<body>/,/<\/body>/ {
                        /<body>/ s/.*<body>//
                        /<\/body>/ s/<\/body>.*//
                        p
                    }' "$fIn"
     fi
done

print - "</body>"
print - "</html>"

exit 0

I hope this helps.

bakunin
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

join files

Hi , I want to join 2 files based on 2 column join condition. a11 john 2230 5000 a12 XXX 2230 A B 200 345 Expected O/P John 2230 5000 A B 200 I have tried this awk 'NR==FNR{a=$1;next}a&&sub($1,a)' a11 a12 > a13 (3 Replies)
Discussion started by: mohan705
3 Replies

2. UNIX for Dummies Questions & Answers

how to join files

Earlier I was unable to edit a line in a file because it was too large. I ended up spliting the file(using split command), which produced multiple files (newfileaa newfilebb ....). Now that I have made my edit, I would like to rejoin the files to original form. How can I do this ? Thanks in... (5 Replies)
Discussion started by: jxh461
5 Replies

3. UNIX for Dummies Questions & Answers

Join 2 files with multiple columns: awk/grep/join?

Hello, My apologies if this has been posted elsewhere, I have had a look at several threads but I am still confused how to use these functions. I have two files, each with 5 columns: File A: (tab-delimited) PDB CHAIN Start End Fragment 1avq A 171 176 awyfan 1avq A 172 177 wyfany 1c7k A 2 7... (3 Replies)
Discussion started by: InfoSeeker
3 Replies

4. Shell Programming and Scripting

Join two files

i have two files and i want to join the contents like: file a has content my name is i am i work at and file b has John sims 43 years old maximu ltd and i want to join the two files to get a third file with content reading my name is John sims i am 43 years old i work at... (2 Replies)
Discussion started by: tomjones
2 Replies

5. UNIX for Dummies Questions & Answers

how to join two files using "Join" command with one common field in this problem?

file1: Toronto:12439755:1076359:July 1, 1867:6 Quebec City:7560592:1542056:July 1, 1867:5 Halifax:938134:55284:July 1, 1867:4 Fredericton:751400:72908:July 1, 1867:3 Winnipeg:1170300:647797:July 15, 1870:7 Victoria:4168123:944735:July 20, 1871:10 Charlottetown:137900:5660:July 1, 1873:2... (2 Replies)
Discussion started by: mindfreak
2 Replies

6. UNIX for Dummies Questions & Answers

How to use the the join command to join multiple files by a common column

Hi, I have 20 tab delimited text files that have a common column (column 1). The files are named GSM1.txt through GSM20.txt. Each file has 3 columns (2 other columns in addition to the first common column). I want to write a script to join the files by the first common column so that in the... (5 Replies)
Discussion started by: evelibertine
5 Replies

7. Shell Programming and Scripting

Join 2 files

I have file1.txt BGE179W1 BGE179W2 BGE179W3 BGE187W1 BGE187W2 BGE187W3 BGE194W1 BGE194W2 BGE194W3 BGE227W1 BGE227W2 BGE227W3 BGE288W1 BGE288W2 BGE288W3 BGE650W1 ---------- Post updated at 12:41 AM ---------- Previous update was at 12:39 AM ---------- (5 Replies)
Discussion started by: radius
5 Replies

8. Shell Programming and Scripting

Join two files

Hi, I have two files Files, FileA and FileB which are attached.Each row in the files have 8 tab delimited columns. The two files have to be compared and joined based on first two columns. The resulting file FileC should have: 1. if the data in the first two columns is same in both the... (3 Replies)
Discussion started by: mehar
3 Replies

9. Shell Programming and Scripting

Join two files

I have 2 files: fileA AAA1:AAA2:AAA3:AAA_4:AAA5:AAA_6:AAA7:AAA_8 BBB1:BBB2:BBB3:BBB_4:BBB5:BBB-6 CCC1:CCC2:CCC3:CCC_4fileB AAA_4:XXX1:YYY1 BBB_4:XXX2:YYY2 CCC_4:XXX3:YYY3:ZZZ3 AAA_6:XXX4:YYY4 AAA_8:XXX5:YYY5Result: AAA1:AAA2:AAA3:AAA_4:XXX1:YYY1:AAA5:AAA_6:XXX4:YYY4:AAA7:AAA_8:XXX5:YYY5... (8 Replies)
Discussion started by: vikus
8 Replies

10. Shell Programming and Scripting

Join, merge, fill NULL the void columns of multiples files like sql "LEFT JOIN" by using awk

Hello, This post is already here but want to do this with another way Merge multiples files with multiples duplicates keys by filling "NULL" the void columns for anothers joinning files file1.csv: 1|abc 1|def 2|ghi 2|jkl 3|mno 3|pqr file2.csv: 1|123|jojo 1|NULL|bibi... (2 Replies)
Discussion started by: yjacknewton
2 Replies

LEARN ABOUT MOJAVE

html::tagset5.18

Tagset(3)						User Contributed Perl Documentation						 Tagset(3)

NAME

       HTML::Tagset - data tables useful in parsing HTML

VERSION

       Version 3.20

SYNOPSIS

	 use HTML::Tagset;
	 # Then use any of the items in the HTML::Tagset package
	 #  as need arises

DESCRIPTION

       This module contains several data tables useful in various kinds of HTML parsing operations.

       Note that all tag names used are lowercase.

       In the following documentation, a "hashset" is a hash being used as a set -- the hash conveys that its keys are there, and the actual
       values associated with the keys are not significant.  (But what values are there, are always true.)

VARIABLES

       Note that none of these variables are exported.

   hashset %HTML::Tagset::emptyElement
       This hashset has as values the tag-names (GIs) of elements that cannot have content.  (For example, "base", "br", "hr".)  So
       $HTML::Tagset::emptyElement{'hr'} exists and is true.  $HTML::Tagset::emptyElement{'dl'} does not exist, and so is not true.

   hashset %HTML::Tagset::optionalEndTag
       This hashset lists tag-names for elements that can have content, but whose end-tags are generally, "safely", omissible.	Example:
       $HTML::Tagset::emptyElement{'li'} exists and is true.

   hash %HTML::Tagset::linkElements
       Values in this hash are tagnames for elements that might contain links, and the value for each is a reference to an array of the names of
       attributes whose values can be links.

   hash %HTML::Tagset::boolean_attr
       This hash (not hashset) lists what attributes of what elements can be printed without showing the value (for example, the "noshade"
       attribute of "hr" elements).  For elements with only one such attribute, its value is simply that attribute name.  For elements with many
       such attributes, the value is a reference to a hashset containing all such attributes.

   hashset %HTML::Tagset::isPhraseMarkup
       This hashset contains all phrasal-level elements.

   hashset %HTML::Tagset::is_Possible_Strict_P_Content
       This hashset contains all phrasal-level elements that be content of a P element, for a strict model of HTML.

   hashset %HTML::Tagset::isHeadElement
       This hashset contains all elements that elements that should be present only in the 'head' element of an HTML document.

   hashset %HTML::Tagset::isList
       This hashset contains all elements that can contain "li" elements.

   hashset %HTML::Tagset::isTableElement
       This hashset contains all elements that are to be found only in/under a "table" element.

   hashset %HTML::Tagset::isFormElement
       This hashset contains all elements that are to be found only in/under a "form" element.

   hashset %HTML::Tagset::isBodyMarkup
       This hashset contains all elements that are to be found only in/under the "body" element of an HTML document.

   hashset %HTML::Tagset::isHeadOrBodyElement
       This hashset includes all elements that I notice can fall either in the head or in the body.

   hashset %HTML::Tagset::isKnown
       This hashset lists all known HTML elements.

   hashset %HTML::Tagset::canTighten
       This hashset lists elements that might have ignorable whitespace as children or siblings.

   array @HTML::Tagset::p_closure_barriers
       This array has a meaning that I have only seen a need for in "HTML::TreeBuilder", but I include it here on the off chance that someone
       might find it of use:

       When we see a "<p>" token, we go lookup up the lineage for a p element we might have to minimize.  At first sight, we might say that if
       there's a p anywhere in the lineage of this new p, it should be closed.	But that's wrong.  Consider this document:

	 <html>
	   <head>
	     <title>foo</title>
	   </head>
	   <body>
	     <p>foo
	       <table>
		 <tr>
		   <td>
		      foo
		      <p>bar
		   </td>
		 </tr>
	       </table>
	     </p>
	   </body>
	 </html>

       The second p is quite legally inside a much higher p.

       My formalization of the reason why this is legal, but this:

	 <p>foo<p>bar</p></p>

       isn't, is that something about the table constitutes a "barrier" to the application of the rule about what p must minimize.

       So @HTML::Tagset::p_closure_barriers is the list of all such barrier-tags.

   hashset %isCDATA_Parent
       This hashset includes all elements whose content is CDATA.

CAVEATS

       You may find it useful to alter the behavior of modules (like "HTML::Element" or "HTML::TreeBuilder") that use "HTML::Tagset"'s data tables
       by altering the data tables themselves.	You are welcome to try, but be careful; and be aware that different modules may or may react
       differently to the data tables being changed.

       Note that it may be inappropriate to use these tables for producing HTML -- for example, %isHeadOrBodyElement lists the tagnames for all
       elements that can appear either in the head or in the body, such as "script".  That doesn't mean that I am saying your code that produces
       HTML should feel free to put script elements in either place!  If you are producing programs that spit out HTML, you should be intimately
       familiar with the DTDs for HTML or XHTML (available at "http://www.w3.org/"), and you should slavishly obey them, not the data tables in
       this document.

SEE ALSO

       HTML::Element, HTML::TreeBuilder, HTML::LinkExtor

COPYRIGHT &; LICENSE
       Copyright 1995-2000 Gisle Aas.

       Copyright 2000-2005 Sean M. Burke.

       Copyright 2005-2008 Andy Lester.

       This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

ACKNOWLEDGEMENTS

       Most of the code/data in this module was adapted from code written by Gisle Aas for "HTML::Element", "HTML::TreeBuilder", and
       "HTML::LinkExtor".  Then it was maintained by Sean M. Burke.

AUTHOR

       Current maintainer: Andy Lester, "<andy at petdance.com>"

BUGS

       Please report any bugs or feature requests to "bug-html-tagset at rt.cpan.org", or through the web interface at
       <http://rt.cpan.org/NoAuth/ReportBug.html?Queue=HTML-Tagset>.  I will be notified, and then you'll automatically be notified of progress on
       your bug as I make changes.

perl v5.18.2							    2008-02-29								 Tagset(3)
All times are GMT -4. The time now is 05:47 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy