07-22-2008
Simple sed one-liner for fixing unencoded ampersands
Hi,
I recieve some XML-files that constantly has bad encoded content. There are Ampersands that are not encoded correctly causing my XML-parser to halt.
I wrote a sed one-liner to fix any stand alone "&":
sed -e 's/&[^amp;|^apos;|^quot;|^lt;|^gt;]/&/gi' input.xml
testfile for input:
<xml>
<source> " One " </source>
<name>test & test</name>
<last>test2&test2</last>
<address>test3 ' test3</address>
<area> test5 < test5</area>
<post> test6 > </post>
<test> test7 &</test>
</xml>
My problem is that the caracter after the "&" is removed as well, destroying the XML-tag
Output:
<xml>
<source> " One " </source>
<name>test & test</name>
<last>test2&test2</last>
<address>test3 ' test3</address>
<area> test5 < test5</area>
<post> test6 > </post>
<test> test7 &/test>
</xml>
I tried the script on both Unix and in Windows 2000 (with unixutil)
Any Ideas?
/Tobbe
10 More Discussions You Might Find Interesting
1. Linux
I am trying to replace the value of $f3 but its not working . I don't know what I am missing here .
cat dim_copy.20080516.sql | grep -i "create view" | grep -v OPSDM002 | while read f1 f2 f3 f4 f5 f6 f7 f8 f9
do
echo " $f3 "
sed -e... (13 Replies)
Discussion started by: capri_drm
13 Replies
2. Shell Programming and Scripting
Execution of the following segment is giving the error -
Script extract:-
OUT=$DATADIR/sol_rsult_orphn.bcp
TMP1=${OUT}_tmp1
TMP=${OUT}_tmp
( isql -w 400 $dbConnect_OPR <<EOF
select convert(char(10), s.lead_id)
+'|'
+ s.pho_loc_type,
";",
s.sol_rsult_cmnt,
";",
+'|'+ s.del_ind... (3 Replies)
Discussion started by: kzmatam
3 Replies
3. Shell Programming and Scripting
I want to use SED to replace all new line characters of a file, I googled and found this one liner
sed '{:q;N;s/\n//g;t q}' infile
what do :q;N; and t q mean in this script? (6 Replies)
Discussion started by: kevintse
6 Replies
4. Shell Programming and Scripting
Hello,
I'm looking for an awk one-liner that prints the first two data fields, then contains a conditional where if $3>$4, it prints $3-$4. Otherwise, it prints $3. Example:
Data file:
123,456,999,888
333,222,444,555
654,543,345,888
444,777,333,111
Output:
123,456,111
333,222,444... (2 Replies)
Discussion started by: palex
2 Replies
5. Shell Programming and Scripting
hey everyone,
I want to remove some characters from a string that i have with sed. For example if my string is:
a0=bus a1=car a2=truck
I want my output to look like this:
bus car truck
So i want to delete the two characters before the = and including the =. This is what i came up with... (3 Replies)
Discussion started by: GmGeubt
3 Replies
6. UNIX for Advanced & Expert Users
Can anyone explain the below sed oneliner?
sed -e ':a' -e '$q;N;11,$D;ba'
It works same as tail command.
I just want to know how it works.
Thanks
---------- Post updated at 11:42 PM ---------- Previous update was at 11:37 PM ----------
Moderators,
Can you please delete this thread?... (0 Replies)
Discussion started by: pandeesh
0 Replies
7. UNIX for Advanced & Expert Users
Can anyone explain the below sed oneliner?
sed -e ':a' -e '$q;N;11,$D;ba'
It works same as tail command.
I just want to know how it works.
Thanks (1 Reply)
Discussion started by: pandeesh
1 Replies
8. UNIX for Dummies Questions & Answers
I have a data base of part numbers:
AAA Thing1
BBB Thing2
CCC Thing3
File one is a list of part numbers:
AAA234
BBB678
CCC2345
Is there a sed one-line that would compare a data base with and replace the part numbers so that the output looks like this?
AAA234 Thing1
BBB678 Thing2... (5 Replies)
Discussion started by: jimmyf
5 Replies
9. UNIX for Dummies Questions & Answers
I have a data base of part numbers:
AAA Thing1
BBB Thing2
CCC Thing3
File one is a list of part numbers:
XXXX AAA234
XXXX BBB678
XXXX CCC2345
Is there a sed one-line that would compare a data base with and replace the part numbers so that the output looks like this?
XXXX AAA234... (7 Replies)
Discussion started by: jimmyf
7 Replies
10. Shell Programming and Scripting
Hi,
I have the following command.(Delete all trailing blank lines at the end of a file.)
sed -e :a -e '/^\n*$/{$d;N;ba' -e '}'
I don't understand the logic of this command and also I don't understand why -e is used.
Can you please let me know the logic of this command and why three -e... (5 Replies)
Discussion started by: TomG
5 Replies
LEARN ABOUT DEBIAN
cgi::formbuilder::source::yaml
CGI::FormBuilder::Source::YAML(3pm) User Contributed Perl Documentation CGI::FormBuilder::Source::YAML(3pm)
NAME
CGI::FormBuilder::Source::YAML - Initialize FormBuilder from YAML file
SYNOPSIS
use CGI::FormBuilder;
my $form = CGI::FormBuilder->new(
source => {
source => 'form.fb',
type => 'YAML',
},
);
my $lname = $form->field('lname'); # like normal
DESCRIPTION
This reads a YAML (YAML::Syck) file that contains FormBuilder config options and returns a hash to be fed to CGI::FormBuilder->new().
Instead of the syntax read by CGI::FormBuilder::Source::File, it uses YAML syntax as read by YAML::Syck. That means you fully specify the
entire data structure.
LoadCode is enabled, so you can use YAML syntax for defining subroutines. This is convenient if you have a function that generates valida-
tion subrefs, for example, I have one that can check profanity using Regexp::Common.
validate:
myfield:
javascript: /^[sS]{2,50}$/
perl: !!perl/code: >-
{ My::Funk::fb_perl_validate({
min => 2,
max => 50,
profanity => 'check'
})->(shift);
}
POST PROCESSING
There are two exceptions to "pure YAML syntax" where this module does some post-processing of the result.
REFERENCES (ala CGI::FormBuilder::Source::File)
You can specify references as string values that start with &, $, @, or \% in the same way you can with CGI::FormBuilder::Source::File.
If you have a full direct package reference, it will look there, otherwise it will traverse up the caller stack and take the first it
finds.
For example, say your code serves multiple sites, and a menu gets different options depending on the server name requested:
# in My::Funk:
our $food_options = {
www.meats.com => [qw( beef chicken horta fish )],
www.veggies.com => [qw( carrot apple quorn radish )],
};
# in source file:
options: @{ $My::Funk::food_options->{ $ENV{SERVER_NAME} } }
EVAL STRINGS
You can specify an eval statement. You could achieve the same example a different way:
options: eval { $My::Funk::food_options->{ $ENV{SERVER_NAME} }; }
The cost either way is about the same -- the string is eval'd.
EXAMPLE
method: GET
header: 0
title: test
name: test
action: /test
submit: test it
linebreaks: 1
required:
- test1
- test2
fields:
- test1
- test2
- test3
- test4
fieldopts:
test1:
type: text
size: 10
maxlength: 32
test2:
type: text
size: 10
maxlength: 32
test3:
type: radio
options:
-
- 1
- Yes
-
- 0
- No
test4:
options: @test4opts
sort: &Someother::Package::sortopts
validate:
test1: /^w{3,10}$/
test2:
javascript: EMAIL
perl: eq 'test@test.foo'
test3:
- 0
- 1
test4: @test4opts
You get the idea. A bit more whitespace, but it works in a standardized way.
METHODS
new()
Normally not used directly; it is called from CGI::FormBuilder. Creates the "CGI::FormBuilder::Source::YAML" object. Arguments from the
'source' hash passed to CGI::FormBuilder->new() will become defaults, unless specified in the file.
parse($source)
Normally not used directly; it is called from CGI::FormBuilder. Parses the specified source file. No fancy params -- just a single file-
name is accepted. If the file isn't acceptable to YAML::Syck, I suppose it will die.
SEE ALSO
CGI::FormBuilder, CGI::FormBuilder::Source
AUTHOR
Copyright (c) 2006 Mark Hedges <hedges@ucsd.edu>. All rights reserved.
LICENSE
This module is free software; you may copy it under terms of the Perl license (GNU General Public License or Artistic License.)
http://www.opensource.org/licenses/index.html
perl v5.8.8 2007-12-09 CGI::FormBuilder::Source::YAML(3pm)