string validation on a column


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting string validation on a column
# 1  
Old 11-18-2004
string validation on a column

Hi there I have a file that I recieve that looks like the folowing, As these details are being manually entered by the customer, we sometimes get a few typos on 4th column (the 14 digit number starting with 42....) As you can see the first one has a space as the first character and then a 13 number string (this has been typed wrong)

I have been asked somehow remove any line that does not have th full 14 character numeric string beginning with 42 and place it in an exceptions file (thus insuring it doesnt get processed)


19-5-204057,161104,0090.09, 4209152509001,AUTH CODE:200395,0,101
19-5-204074,161104,0010.00,42067036239101,AUTH CODE:7935 ,0,101
19-5-204075,161104,0010.00,42067036239100,AUTH CODE:3145 ,0,101
23-5-185395,161104,0012.02,42071862819101,AUTH CODE:005932,0,101
23-5-185405,161104,0155.76,42079561219101,AUTH CODE:289603,0,101
23-5-185407,161104,0023.71,42055829039101,AUTH CODE:694281,0,101


I really have no idea where to start on this so any pointers would be great, has awk got the capability to do this ??
# 2  
Old 11-18-2004
You could probably do it with awk, but I like my shell scripts!
Code:
#!/bin/sh

while read line
do
  echo "$line" | cut -d',' -f4 | grep "42[0-9]\{12\}" >/dev/null 2>&1
  if [ "$?" -eq "0" ]; then
    echo "$line" >> good_data
  else
    echo "$line" >> bad_data
  fi
done < testdata

Cheers
ZB
# 3  
Old 11-19-2004
Maybe try egrep (or grep -E)

egrep -v '^[^,]*,[^,]*,[^,]*,[0-9]{14},' infile > badfile
egrep '^[^,]*,[^,]*,[^,]*,[0-9]{14},' infile >goodfile
# 4  
Old 11-19-2004
Using awk ....

----------------------------

awk -F"," '{ if ($4 ~ / /) { } else
{ print $4 } }' testData

-----------------------------------
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Column 2 string count in Column 3

My I/p is Col1|Col2|Col3 2116209997932|POSIX INC|POSIX 2116209997933|POSIX INC|POSIX 2116210089479|POSIX INC|POSIX 2116210180502|POSIX INC|POSIX 2116210512279|POSIX INC|Aero 2116210516838|POSIX INC|POSIX 2116210534342|POSIX INC|postal 2116210534345|POSIX INC|postal ... (6 Replies)
Discussion started by: nikhil jain
6 Replies

2. UNIX for Dummies Questions & Answers

Date column validation in files

HI, i have to do some file validations on which i am receving daily.could any one please let me know how we can able to do validations on below scenarios 1) On Date column Ex If i am receving date in format of YYYYMMDD---- 20141118 --- here i have to check the date value 20141142 if the... (7 Replies)
Discussion started by: bhaskar v
7 Replies

3. Shell Programming and Scripting

String Replacement in a column

I have a text file with the following contents. I am trying to check the column "COL5". If the value is not "0" then I need to replace the value to NA only in COL5. The problem is that I don't have a clear delimiter here, I have only the column names. cat input.txt -08/27/14:08:30:01-- ... (5 Replies)
Discussion started by: ctrld
5 Replies

4. Shell Programming and Scripting

Use a string in one column to get the largest or the smallest of another column

I have data that looks like this: chr1 mm9_knownGene exon 155747075 155747189 0.000000 + . gene_id "Glul"; transcript_id "uc007daq.1"; chr1 mm9_knownGene exon 155750064 155750076 0.000000 + . gene_id "Glul";... (3 Replies)
Discussion started by: pbluescript
3 Replies

5. Shell Programming and Scripting

String validation in shell script

Hi All, I am a newbie...I would like to have a function which ll check if a file contains valid strings before "=" operator. Just to give you my requirement: assume my file has content: hello= gsdgsd sfdsg sgdsg sgdgdg world= gggg hhhh iiiii xxxx= pppp ppppp pppp my... (1 Reply)
Discussion started by: rtagarra
1 Replies

6. Shell Programming and Scripting

Copy column string and put in different column

Hello Here is my input: SU3902 SU3902A NS29C (10) (00) Q1J1 0 SU3902 SU3902B VLR05 (20) (02) Q2H1 4 SU3902 SU3902C NBR22 (30) (06) Q3R5 8 SU3904 SU39047 NSV19 (11) (09) Q4k6 2 SU3904 SU39048 LB231 (12) (05) Q5k1 6 SU3904 SU39049 11VLT (13) (08) Q10C1 10 SU3904 SU3904A 25R05 (15) (06)... (3 Replies)
Discussion started by: pareshkp
3 Replies

7. Shell Programming and Scripting

Get first column from a huge string ..!!

Guys Look at the following string....!! /global/site/vendor/Vignette7/Content/7_5/java5/jre/bin/java -classpath /global/site/vendor/Vignette7/Content/7_5/lib/vgnconfiglauncher.jar -Dcom.vignette.jvmid=V7CDS1CA1 -DVgnStartupClass=com.vignette.config.agent.Agent... (15 Replies)
Discussion started by: ak835
15 Replies

8. UNIX for Advanced & Expert Users

Replace string in column

Hi, I want to replace string in column,Example i have file caleed a1.txt ,want to replace string "A12" with "A23" only in column2 ,not from file itself.Using sed command replace string in file itself. Thanks, Mohan (3 Replies)
Discussion started by: mohan705
3 Replies

9. Shell Programming and Scripting

String Validation program

Hi I want to validate the sting which one having only A-Z, a-z, *, . ,_ and 0-9 digits. Can anyone send me the program? I tried with following program but its taking all special characters like @ , # % and ^. echo " Enter Text :" read text while ') ] do echo "Character is wrong"... (4 Replies)
Discussion started by: mpk2006
4 Replies

10. Shell Programming and Scripting

String validation

All I want to validate a String. If it is "Deepak Xavier" then it will a valid string. But if the value "Deepak #&xavier" then it should be invalid. Please give me some commands. Iam using KORN shell. Thanx in advance. Regards Deepak Xavier (3 Replies)
Discussion started by: DeepakXavier
3 Replies
Login or Register to Ask a Question