Find invalid character


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Find invalid character
# 8  
Old 10-29-2014
HI,

I have one file in which client names are there, and i need to find client name which has invalid character.

So i make this script.
Code:
while read line
do 
              n=0  while (( $n <= ${#f} ));  do  c="${f:$n:1}"  echo '$c'  if [[ "$c" = *[^[:space:]*] ]];  then   grep -sq $c valid.txt  if  [ $? -eq  1 ];  then  echo "$f" >> f.txt  break  fi  fi  n=$((n+1))  done
done < client.txt

and my valid character are
Code:
a b c d e f g h i j k l m n o p q r s t u v w x y z A B C D E  F G H I J K L M N O P Q R S T U V W X Y Z ( ) { }  \t 1 2 3 4 5 6 7 8 9 0 - / :  ? + \n . , '

But my script is not finding \ character and by using fgrep also it is giving error . I dont know what are invalid character but i have list of valid character which are mentioned above.

I have one client name in which \ slash are there and some how my script is not picking that one line.

Hope this clarify you and please help me how i can correct it.
# 9  
Old 10-29-2014
Quote:
also it is giving error
Giving what error?
# 10  
Old 10-29-2014
If you have bash shell can I suggest the following which doesn't use any external programs and should run a lot quicker:

Code:
#!/bin/bash
valid=( $'\t' $'\n' \\ \( \) \{ \} \' - / : ? + . , {a..z} {A..Z} {0..9} )
v="${valid[@]}"

while read line
do
    for((n=0;n<${#line};n++))
    do
       c=${line:$n:1}
       [[ "$c" = '\' || "$c" = '?' ]] && c=\\"$c"

       if [[ "${v/$c/}" = "${valid[@]}" ]]
       then
          echo "Invalid character: ${line:$n:1} found in $line"
       else
          echo "$c" >> f.txt
       fi
    done
done < client.txt


Last edited by Chubler_XL; 10-29-2014 at 07:24 PM.. Reason: Support question character properly
# 11  
Old 10-29-2014
HI Chubler,

i have ran ur script by modifying little bit, but it is not giving any output. i have created dummy file which has
Code:
#!/bin/bash
valid=( $'\t' $'\n' \\ \( \) \{ \} \' - / : ? + . , {a..z} {A..Z} {0..9} )
v="${valid[@]}"

while read line
do
    for((n=0;n<${#line};n++))
    do
       c=${line:$n:1}
       [[ "$c" = '\' || "$c" = '?' ]] && c=\\"$c"

       if [[ "${v/$c/}" = "${valid[@]}" ]]
       then
          echo "Invalid character: ${line:$n:1} found in $line" >> f.txt
       fi
    done
done < client.txt

client.txt
Code:
cat client.txt
pallvi mahajan
lonodn #enterprise
kite \ pallvi
sunny sapen20\45\65

after script ran, in f.txt it should come all records except 1st one, but it only search London record

Code:
Invalid character: # found in lonodn #enterprise

it is not searches again \ record
# 12  
Old 10-29-2014
This is because the \ character is in the valid list if you want \ as invalid change

this:
Code:
valid=( $'\t' $'\n' \\ \( \) \{ \} \' - / : ? + . , {a..z} {A..Z} {0..9} )

to this:
Code:
valid=( $'\t' $'\n' \( \) \{ \} \' - / : ? + . , {a..z} {A..Z} {0..9} )

Edit: Also to have read process backslash characters properly use -r option:

Code:
while read -r line


Last edited by Chubler_XL; 10-29-2014 at 08:46 PM..
# 13  
Old 10-29-2014
HI chubler,

i remove that character also, still same output.

---------- Post updated at 06:48 PM ---------- Previous update was at 06:47 PM ----------

and also please let us know what this command is doing, do i need to change any thing here also
Code:
[[ "$c" = '\' || "$c" = '?' ]] && c=\\"$c" 
        if [[ "${v/$c/}" = "${valid[@]}" ]]

# 14  
Old 10-29-2014
See updated post: you will need the -r option of read to stop it interpreting the backslash as an escape character on input.

To explain the two lines:
Code:
[[ "$c" = '\' || "$c" = '?' ]] && c=\\"$c"

This code excapes a couple of characters that are specially treated by the Pattern substitution feature of bash. \ escapes the next character. ? is a wildcard representing any single character.

Code:
if [[ "${v/$c/}" = "${valid[@]}" ]]

This code tests if replacing your character with nothing changes the valid characters list. If these strings are the same it means the input character is not in the valid characters list.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Searching invalid character in list of client name

Hi Friend, I have a client name list and client name has some invalid character due to which some issue raised and list of client are15k. I want to make script who find invalid character name. can you please help me how i can make script, i means i need logic. Valid character are :- ... (5 Replies)
Discussion started by: pallvi_mahajan
5 Replies

2. AIX

Bison -pap_expr_yy invalid character:% unexpected "identifier" while running make for Apache2.4.3 64

The Follwing packages are installed on my AIX 6.1 box gcc-4.7.2-1 gcc-c++-4.7.2-1 gcc-cpp-4.7.2-1 gcc-gfortran-4.7.2-1 libgcc-4.7.2-1 libgomp-4.7.2-1 libstdc++-4.7.2-1 libstdc++-devel-4.7.2-1 gmp-5.0.5-1 libmpc-1.0.1-2 libmpc-devel-1.0.1-2 libmpcdec-1.2.6-1 libmpcdec-devel-1.2.6-1... (0 Replies)
Discussion started by: Ashish Gupta
0 Replies

3. Shell Programming and Scripting

Find character and Replace character for given position

Hi, i want find the character '-' in a file from position 284-298, if it occurs i need to replace it with 'O ' for the position in the file. How to do that using SED command. thanks in advance, Sara (9 Replies)
Discussion started by: Sara183
9 Replies

4. Shell Programming and Scripting

How to find invalid URL in a text file using shell script?

How to find and remove invalid URLs in a text file using shell script? (1 Reply)
Discussion started by: vel4ever
1 Replies

5. Shell Programming and Scripting

Find and replace a character

Hi Team, i have 1st cloumn of data containing, LAMSBA01-BA-COFF-YTD LAMSBA01-BA-COFF-ITD LAMSBA01-BA-AGGR-IND . LAMSBA01-BA-CURR-COFF-BAL i need to replace the "-" to "_" (underscore) using AWK . please help me on this. Thanks, Baski (4 Replies)
Discussion started by: baskivs
4 Replies

6. Solaris

An invalid XML character (Unicode: 0x1a)

While uploading an exl file to my application in Solaris 10 the upload failed with error Error! Parsing Error: /SPLM/TC83/tcdata83/model/model_dbextract.xml Line:65576 Column:73 An invalid XML character (Unicode: 0x1a) was found in the value of attribute "unitOfMeasureSymbol" and element is ... (12 Replies)
Discussion started by: karghum
12 Replies

7. UNIX for Dummies Questions & Answers

to find alphanumeric character

Hi All, I am having a data file which consists of lakhs of records in the below foramt: "1223323","4341341","discription aadfad" "3123123","5463456","discription aadfad" "2343324","6565767","discription asdfads" "A3423423","7957456","discription aadfad" "343453B","7957456","discription... (1 Reply)
Discussion started by: abhi_123
1 Replies

8. Linux

Invalid Character

Hi, I am using a Perl script to generate a report file in Linux server. When my input data contains an invalid character which looks like hyphen after that my program is printing junk values in the report. Why that symbol is causing issue and is there a way to tell the server that this is a valid... (1 Reply)
Discussion started by: lawrance_ps
1 Replies

9. Shell Programming and Scripting

Find ^M Character

Hi Experts, Is there a way to find whether a file contains ^M character in it?. It's quite obvious to vi, but can this be accomplished using an automated way to find recursively in a directory. probably i guess the best way would be to 1) Read the file and check if the end of line is ^M... (8 Replies)
Discussion started by: ganga.dharan
8 Replies

10. Shell Programming and Scripting

writing shell script to find line of invalid characters

Hi, I have to write s script to check an input file for invalid characters. In this script I have to find the exact line of the invalid character. If the input file contain 2 invalid character sat line 10 and 17, the script will show the value 10 and 17. Any help is appreciated. (3 Replies)
Discussion started by: beginner82
3 Replies
Login or Register to Ask a Question