Sponsored Content
Top Forums Shell Programming and Scripting How to find out the weird blank characters? Post 303003866 by RudiC on Thursday 21st of September 2017 03:48:03 PM
Old 09-21-2017
I don't think it's wise to get rid of them, because they separate (and thus define) the words. Leave them in, count them, and then eliminate the "blank" count.
Those non-ASCII opening and closing double quotes are multibyte unicode characters. It might be easier to convert them to ASCII- quotes beforehand. Same holds true mayhap for the "blank" chars above...

Once converted, this
Code:
awk -F\" '
        {while (!(NF%2))        {getline X
                                 $0 = $0 " " X
                                }
         for (i=2; i<=NF; i+=2) print gsub (/[A-Za-z0-9]+/, "&", $i)
        }
' file

might give you a feeling for the "speech length".

Last edited by RudiC; 09-21-2017 at 05:00 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Blank characters between Datas

Hello, I read a file whose in lines are datas and between thses datas there is blank characters (10, 12 or 5 or 1 .......) So when i use the command while read line in the script(see under) there is also only one character between the datas and the others blank characters are not here. ... (3 Replies)
Discussion started by: steiner
3 Replies

2. Shell Programming and Scripting

Deleting the blank line in a file and counting the characters....

Hi, I am trying to do two things in my script. I will really appreciate any help in this regards. Is there a way to delete a last line from a pipe delimited flat file if the last line is blank. If the line is not blank then do nothing..... Is there a way to count a word that are starting... (4 Replies)
Discussion started by: rkumar28
4 Replies

3. UNIX for Dummies Questions & Answers

How to get rid of all the weird characters and color on bash shell

Does anyone of you know how to turn off color and weird characters on bash shell when using the command "script"? Everytime users on my server used that command to record their script, they either couldn't print it because lp kept giving the "unknown format character" messages or the print paper... (1 Reply)
Discussion started by: Micz
1 Replies

4. Shell Programming and Scripting

Weird Ascii characters in file names

Hi. I have files in my OS that has weird file names with not-conventional ascii characters. I would like to run them but I can't refer them. I know the ascii # of the problematic characters. I can't change their name since it belongs to a 3rd party program... but I want to run it. is there... (2 Replies)
Discussion started by: yamsin789
2 Replies

5. UNIX for Advanced & Expert Users

cat / sed process weird characters

Hi everyone, I'm trying to write a shell script that process a log file. The log format is generally: (8 digit hex of unix time),(system ID),(state)\n My shell script gets the file from the web, saves it in a local text directory. I then want to change the hex to decimal, convert from unix time... (7 Replies)
Discussion started by: bencpeters
7 Replies

6. Shell Programming and Scripting

share a shell script which can replace weird characters in directory or file name

I just finish the shell script . This shell can replace weird characters (such as #$%^@!'"...) in file or directory name by "_" I spent long time on replacing apostrophe in file/directory name added: 2012-03-14 the 124th line (/usr/bin/perl -i -e "s#\'#\\'#g" /tmp/rpdir_level$i.tmp) is... (5 Replies)
Discussion started by: begonia
5 Replies

7. Shell Programming and Scripting

Removing one or more blank characters from beginning of a line

Hi, I was trying to remove the blank from beginning of a line. when I try: sed 's/^ +//' filename it does not work but when I try sed 's/^ *//' filename it works But I think the first command should have also replaced any line with one or more blanks. Kindly help me in understanding... (5 Replies)
Discussion started by: babom
5 Replies

8. Shell Programming and Scripting

Weird ^M characters is disturbing the paste command

Dear all, I have the files: xaa xab xac and I try to paste them using $paste -d, xaa xab xac I see: output 3e-130 ,6e-78 ,5e-74 6e-124 ,0,007 ,0,026 2e-119 When I type: $ paste -d, xaa xab xac |less I see: output 3e-130^M,6e-78^M,5e-74 6e-124^M,0,007^M,0,026 (2 Replies)
Discussion started by: valente
2 Replies

9. Shell Programming and Scripting

Control characters -weird problem

I am using Korn shell on Linux 2.6x platform , and I am suing the following code to capture the lines which contain CONTROL CHARACTERS in my file : awk '/]/ {print NR}' EROLLMENT_INPUT.txt The problem is that this code shows the file has control characters when the file is in folder A ,... (2 Replies)
Discussion started by: kumarjt
2 Replies

10. Shell Programming and Scripting

To check Blank Lines, Blank Records and Junk Characters in a File

Hi All Need Help I have a file with the below format (ABC.TXT) : ®¿¿ABCDHEJJSJJ|XCBJSKK01|M|7348974982790 HDFLJDKJSKJ|KJALKSD02|M|7378439274898 KJHSAJKHHJJ|LJDSAJKK03|F|9898982039999 (cont......) I need to write a script where it will check for : blank lines (between rows,before... (6 Replies)
Discussion started by: chatwithsaurav
6 Replies
Text::Typography(3pm)					User Contributed Perl Documentation				     Text::Typography(3pm)

NAME
Text::Typography - Markup ASCII text with correct typography for HTML SYNOPSIS
use Text::Typography qw(typography); print typography($text); DESCRIPTION
This module is a thin wrapper for John Gruber's SmartyPants plugin for various CMSs. SmartyPants is a web publishing utility that translates plain ASCII punctuation characters into "smart" typographic punctuation HTML enti- ties. SmartyPants can perform the following transformations: o Straight quotes ( " and ' ) into "curly" quote HTML entities o Backticks-style quotes (``like this'') into "curly" quote HTML entities o Dashes ("--" and "---") into en- and em-dash entities o Three consecutive dots ("...") into an ellipsis entity SmartyPants does not modify characters within "<pre>", "<code>", "<kbd>", "<script>", or "<math>" tag blocks. Typically, these tags are used to display text where smart quotes and other "smart punctuation" would not be appropriate, such as source code or example markup. typography($text[, $attributes]) Returns a string marked up with the proper HTML entities for proper typography. For fine grain control over what gets converted, use the $attributes option. The default value is 3. The following numeric values set a group of options: 0 : do nothing 1 : set all 2 : set all, using old school en- and em- dash shortcuts (-- and ---) 3 : set all, using inverted old school en- and em- dash shortcuts (--- and --) For even finer control, specify a string of one or more of the following characters: q : quotes b : backtick quotes (``double'' only) B : backtick quotes (``double'' and `single') d : dashes D : old school dashes i : inverted old school dashes e : ellipses w : convert &quot; entities to " for Dreamweaver users Backslash Escapes If you need to use literal straight quotes (or plain hyphens and periods), SmartyPants accepts the following backslash escape sequences to force non-smart punctuation. It does so by transforming the escape sequence into a decimal-encoded HTML entity: Escape Value Character ------ ----- --------- \ &#92; " &#34; " ' &#39; ' . &#46; . - &#45; - ` &#96; ` This is useful, for example, when you want to use straight quotes as foot and inch marks: 6'2" tall; a 17" iMac. Algorithmic Shortcomings One situation in which quotes will get curled the wrong way is when apostrophes are used at the start of leading contractions. For example: 'Twas the night before Christmas. In the case above, SmartyPants will turn the apostrophe into an opening single-quote, when in fact it should be a closing one. I don't think this problem can be solved in the general case -- every word processor I've tried gets this wrong as well. In such cases, it's best to use the proper HTML entity for closing single-quotes ("&#8217;") by hand. AUTHOR
Thomas Sibley created this module using the code from the SmartyPants CMS plugin by John Gruber (<http://daringfireball.net/projects/smar- typants/>). COPYRIGHT AND LICENSE
Copyright (c) 2003 John Gruber (http://daringfireball.net/) All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name "SmartyPants" nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. This software is provided by the copyright holders and contributors "as is" and any express or implied warranties, including, but not lim- ited to, the implied warranties of merchantability and fitness for a particular purpose are disclaimed. In no event shall the copyright owner or contributors be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not lim- ited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this software, even if advised of the possibility of such damage. perl v5.8.8 2008-03-02 Text::Typography(3pm)
All times are GMT -4. The time now is 12:32 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy