Need HELP with AWK split. Need to check for "special characters" in string before splitting the file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need HELP with AWK split. Need to check for "special characters" in string before splitting the file
# 1  
Old 07-30-2012
Need HELP with AWK split. Need to check for "special characters" in string before splitting the file

Hi Experts.

I'm stuck with the below AWK code where i'm trying to move the records containing any special characters in the last field to a bad file.

Code:
awk -F, '{if ($NF ~ /^[0-9]|^[A-Za-z]/) print >"goodfile";else print >"badfile"}' filename

sample data
Code:
1,abc,def,1234,A *
2,bed,dec,342,* A         
3,dec,345,23,*&^          
4,sdf,fgh,234,  
5,ert,345,ghj,C**
6,ert,345,sdf,123          ---- only valid record

The output required must contain the first 5 records in badfile and the last record in good file.
But my above awk logic cosiders only the below records as badfile records:
Code:
2,bed,dec,342,* A 
3,dec,345,23,*&^ 
4,sdf,fgh,234,

The other two invalid records ("A *" and "C**") are being written into goodfile which is wrong.
Please help me fix this.

Note: the $NF values can contain [spaces:] between any alphanumeric chars. However, all spaces or null is considered a bad record.

Thanks Gurus!
# 2  
Old 07-30-2012
Tune your regexp to:
Code:
awk ' /[0-9]$|[A-Za-z]$/{print >"goodfile";next}{print >"badfile"}' infile

# 3  
Old 07-30-2012
I tuned the regex as suggested. But it does not give the required output.
code used:
Code:
awk -F, '{if ($NF ~ /[0-9]$|[A-Za-z]$/) print >"goodfile"; else print >"badfile"}' samp.txt

samp.txt
Code:
1,abc,def,1234,A *
2,bed,dec,342,* A
3,dec,345,23,*&^
4,sdf,fgh,234,
5,ert,345,ghj,C*2
6,ert,345,sdf,123

Output
Code:
$ cat goodfile
2,bed,dec,342,* A
5,ert,345,ghj,C*2
6,ert,345,sdf,123
$ cat badfile
1,abc,def,1234,A *
3,dec,345,23,*&^
4,sdf,fgh,234,

# 4  
Old 07-30-2012
Check your syntax:
Code:
awk -F, '{if ($NF ~ /[0-9]$|[A-Za-z]$/) {print >"goodfile"} else {print >"badfile" }}'  infile

# 5  
Old 07-30-2012
Sorry, Klashxx.
It gives the same output.

Code:
$ cat samp.txt
1,abc,def,1234,A *
2,bed,dec,342,* A
3,dec,345,23,*&^
4,sdf,fgh,234,
5,ert,345,ghj,C*2
6,ert,345,sdf,123

$ awk -F, '{if ($NF ~ /[0-9]$|[A-Za-z]$/) {print >"goodfile"} else {print >"badfile" }}' samp.txt

$ cat goodfile
2,bed,dec,342,* A
5,ert,345,ghj,C*2
6,ert,345,sdf,123

$ cat badfile
1,abc,def,1234,A *
3,dec,345,23,*&^
4,sdf,fgh,234,

---------- Post updated at 05:29 AM ---------- Previous update was at 05:28 AM ----------

I'm using korn shell. Maybe that makes a difference .??
# 6  
Old 07-30-2012
Quote:
Originally Posted by Klashxx
Check your syntax:
Code:
awk -F, '{if ($NF ~ /[0-9]$|[A-Za-z]$/) {print >"goodfile"} else {print >"badfile" }}'  infile

This only looks at $NF's last char. This
Code:
awk -F, '{if ($NF~/[0-9A-Za-z][0-9A-Za-z][0-9A-Za-z]/) print >"goodfile"; else print>"badfile}' samp.txt

will work on the example, but it does not take into account the possible variable length of $NF. The repetition term /.../{length($NF)} does not seem to work, nor does the regex [[:alnum:]] contruct.
This User Gave Thanks to RudiC For This Post:
# 7  
Old 07-30-2012
Try this:
Code:
awk -F, '{if($NF~/^[[:alnum:][:blank:]]+$/) print > "goodfile"; else print > "badfile"}' infile

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Unable to identify the special characters beyond the range of "[\x80-\xFF]"

I want to filter out the special character whose ascii value doesn't fall within the range "" . Example:� or Ć. So in that case is there any defined range which will filter out this characters. I can filter those which falls withing "" . Need to filter those special chracter which doesn't... (14 Replies)
Discussion started by: Abhijit Sen
14 Replies

2. UNIX for Dummies Questions & Answers

PuTTY displaying "special" characters

I'm not really sure which forum this question should go into, so I'm posting it here. I work with AIX and RHEL systems using PuTTY (Release 0.60_q1.129) from a Windows 7 workstation. Some of the files we get from z/OS use "special" characters as delimiters. These characters include Hex 18... (7 Replies)
Discussion started by: derndingle
7 Replies

3. Shell Programming and Scripting

finding the strings beween 2 characters "/" & "/" in .txt file

Hi all. I have a .txt file that I need to sort it My file is like: 1- 88 chain0 MASTER (FF-TE) FFFF 1962510 /TCK T FD2TQHVTT1 /jtagc/jtag_instreg/updateinstr_reg_1 dff1 (TI,SO) 2- ... (10 Replies)
Discussion started by: Behrouzx77
10 Replies

4. Shell Programming and Scripting

how to use "cut" or "awk" or "sed" to remove a string

logs: "/home/abc/public_html/index.php" "/home/abc/public_html/index.php" "/home/xyz/public_html/index.php" "/home/xyz/public_html/index.php" "/home/xyz/public_html/index.php" how to use "cut" or "awk" or "sed" to get the following result: abc abc xyz xyz xyz (8 Replies)
Discussion started by: timmywong
8 Replies

5. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone... (7 Replies)
Discussion started by: shis100
7 Replies

6. Shell Programming and Scripting

Splitting text string with "|" pipe delimiters

Hi, I need to split the fields in some text strings into separate variables. The fields are separated by pipes. The strings look roughly like this: CUSTNAME|hostname|userid|gecos field|staff|disabled||SUDO Currently I can do it using something like cut -d"|" -f1, but when I put that into... (4 Replies)
Discussion started by: alanp36
4 Replies

7. Shell Programming and Scripting

"sed" to check file size & echo " " to destination file

Hi, I've modified the syslogd source to include a thread that will keep track of a timer(or a timer thread). My intention is to check the file size of /var/log/messages in every one minute & if the size is more than 128KB, do a echo " " > /var/log/messages, so that the file size will be set... (7 Replies)
Discussion started by: jockey007
7 Replies

8. Shell Programming and Scripting

how to split special characters "|" using awk

Hi friends I need to splict special character "|" here. Here is my script which giving error LINE=INVTRAN|cd /home/msgGoogle TraxFolderType=`awk -F"|" '{print $1}' $LINE` filePath=`awk -F"|" '{print $2}' $LINE` echo "TraxFolderType: "$TraxFolderType echo "filePath :"$filePath ... (3 Replies)
Discussion started by: krishna9
3 Replies

9. UNIX for Dummies Questions & Answers

perl split funciton - special character "/"

HI, I have a directory structure. /abc/def/ghi/ I want to store it into array. So that if I do a pop function on that array I can easily go to previous directory. But how can i split and store it. @Directory = split(/\//,$DirectoryVarialbe) That doest works. Any other escape sequence... (5 Replies)
Discussion started by: deepakwins
5 Replies

10. Shell Programming and Scripting

split the string "Setview: arv-temp-view"

Hi experts, I would like to split the value of v and assign "arv-temp-view" to another variable d. I want to do it within shell script. My shell is "tcsh" v="Setview: arv-temp-view" split v and store d=arv-temp-view Please help Thanks Amit (2 Replies)
Discussion started by: amitrajvarma
2 Replies
Login or Register to Ask a Question