Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: filtering records based on numeric field value in 8th position
Top Forums Shell Programming and Scripting filtering records based on numeric field value in 8th position Post 302332755 by indusri on Friday 10th of July 2009 03:32:48 AM
Old 07-10-2009
filtering records based on numeric field value in 8th position
I have a ";" delimited file.Whcih conatins a number fileds of length 4 charcters in 8th position
But there is a alphanumeric charcters like :
space, ";" , "," , "/" , "23-1" ,
"23 1" , "aqjhdj" , "jun-23" , "APR-04" , "4:00AM" , "-234" , "56784 ", "." , "+"
"_" , "&" , "*" , "^" , "%" , "!" , "45/3" , "78*a" etc
which all gives the pl/sql numeric error.

But the actual value of filed should be in between 1-9999 only

How can I separate those records which are in the wrong format?
I need to separtae thme from orginal as proper format
and improper format.

I am able to filter some record with awk as given below.


awk -F";" '$8 !~ /[-A-Z ]/ { print $0 >>"proper.txt";next } { print $0 >>"improper.txt"} ' File_name.txt

Please see my input file.

C;4498;qwa;cghy;;;;40;;222122
C;4498;sample;city;;;;34 2;;222123
C;4498;qwe;xcbv;;;;34-2;;222124
C;4498;jj;sffz;;;;41;;222120
C;4498;jj;sffz;;;;41/a;;222120
C;4498;jj;;;;;30;;277789
C;4498;eert;qwq;;;;34 A;;222125
C;4498;;;;;;23;;22
C;4498;jj;szxzzd;;;;34*a;;222127
C;4498;jj;szxzzd;;;;a;;222127
C;4498;jj;szxzzd;;;;57864;;222127
C;4498;jj;szxzzd;;;;34;;222127
C;4498;jj;;;;;30+;;277789
C;4498;jj;;;;;4:00AM;;277789
C;4498;jj;;;;;JUN-04;;277789
C;4498;jj;szxzzd;;;;45;;222
C;4498;jj;szxzzd;;;;34.;;222127
C;4498;jj;sffz;;;;heruru;;222120

I need files as

proper file:

C;4498;qwa;cghy;;;;40;;222122
C;4498;jj;sffz;;;;41;;222120
C;4498;jj;;;;;30;;277789
C;4498;;;;;;23;;22
C;4498;jj;szxzzd;;;;34;;222127
C;4498;jj;szxzzd;;;;45;;222


Improper file:


C;4498;sample;city;;;;34 2;;222123
C;4498;qwe;xcbv;;;;34-2;;222124
C;4498;jj;sffz;;;;41/a;;222120
C;4498;jj;;;;;30;;277789
C;4498;eert;qwq;;;;34 A;;222125
C;4498;jj;szxzzd;;;;34*a;;222127
C;4498;jj;szxzzd;;;;a;;222127
C;4498;jj;szxzzd;;;;57864;;222127
C;4498;jj;;;;;30+;;277789
C;4498;jj;;;;;4:00AM;;277789
C;4498;jj;;;;;JUN-04;;277789
C;4498;jj;szxzzd;;;;34.;;222127
C;4498;jj;sffz;;;;heruru;;222120


Thanks alot for your help.
Please provide the solution ASAP.
Last edited by indusri; 07-10-2009 at 04:41 AM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Filtering records of a file based on a value of a column

Hi all, I would like to extract records of a file based on a condition. The file contains 47 fields, and I would like to extract only those records that match a certain value in one of the columns, e.g. COL1 COL2 COL3 ............... COL47 1 XX 45 ... (4 Replies)
Discussion started by: risk_sly
4 Replies

2. Shell Programming and Scripting

Based on num of records in file1 need to check records in file2 to set some condns

Hi All, I have two files say file1 and file2. I want to check the number of records in file1 and if its atleast 2 (i.e., 2 or greater than 2 ) then I have to check records in file2 .If records in file2 is atleast 1 (i.e. if its not empty ) i have to set some conditions . Could you pls... (3 Replies)
Discussion started by: mavesum
3 Replies

3. Shell Programming and Scripting

filter out all the records which are having space in the 8th filed of my file

I have a file which is having fileds separtaed by delimiter. Ex: C;4498;qwa;cghy;;;;40;;222122 C;4498;sample;city;;;;34 2;;222123 C;4498;qwe;xcbv;;;;34-2;;222124 C;4498;jj;sffz;;;;41;;222120 C;4498;eert;qwq;;;;34 A;;222125 C;4498;jj;szxzzd;;;;34;;222127 out of these records I... (3 Replies)
Discussion started by: indusri
3 Replies

4. Shell Programming and Scripting

Extract file records based on some field conditions

Hello Friends, I have a file(InputFile.csv) with the following columns(the columns are pipe-delimited): ColA|ColB|ColC|ColD|ColE|ColF Now for this file, I have to get those records which fulfil the following condition: If "ColB" is NOT NULL and "ColD" has values one of the following... (9 Replies)
Discussion started by: mehimadri
9 Replies

5. Shell Programming and Scripting

To find sum & average of 8th field

Hi Friends, I have many files like below. total,0.7%,0.0%,0.2%,0.0%,0.2%,0.7%,98.0% total,1.9%,0.0%,0.4%,0.0%,0.0%,6.8%,90.6% total,0.9%,0.0%,0.4%,0.0%,0.0%,0.0%,98.5% total,1.4%,0.0%,0.7%,0.0%,0.2%,2.9%,94.5% total,0.7%,0.0%,0.4%,0.0%,0.0%,0.9%,97.7%... (13 Replies)
Discussion started by: SunilB2011
13 Replies

6. UNIX for Dummies Questions & Answers

Filtering records from 1 file based on some manipulation doen on second file

Hi, I am looking for an awk script which should help me to meet the following requirement: File1 has records in following format INF: FAILEd RECORD AB1234 INF: FAILEd RECORD PQ1145 INF: FAILEd RECORD AB3215 INF: FAILEd RECORD AB6114 ............................ (2 Replies)
Discussion started by: mintu41
2 Replies

7. Shell Programming and Scripting

filtering a numeric value which has '%' using awk

Hello Gurus, I have a requirement where I have to filter a value from some field which has 99% or greater than '99%'.. For ex: The Date (file -- sample.csv) will be like below Field1,Field2,Field3,Field4 860440512,844284992,16155520,99% 860440512,844284992,16155520,94%... (4 Replies)
Discussion started by: raghu.iv85
4 Replies

8. Shell Programming and Scripting

Search for a string at a particular position and replace with blank based on position

Hi, I have a file with multiple lines(fixed width dat file). I want to search for '02' in the positions 45-46 and if available, in that lines, I need to replace value in position 359 with blank. As I am new to unix, I am not able to figure out how to do this. Can you please help me to achieve... (9 Replies)
Discussion started by: Pradhikshan
9 Replies

9. Shell Programming and Scripting

Need help with filtering records in a file

Hi, I have following records in a file more file1.txt setting applicaction ABC for user setting applicaction CDE for user setting applicaction XXX for user logging applicaction XXX for user I need to filter out records which have strings " setting... (5 Replies)
Discussion started by: manid
5 Replies

10. UNIX for Beginners Questions & Answers

Filtering records of a csv file based on a value of a column

Hi, I tried filtering the records in a csv file using "awk" command listed below. awk -F"~" '$4 ~ /Active/{print }' inputfile > outputfile The output always has all the entries. The same command worked for different users from one of the forum links. content of file I was... (3 Replies)
Discussion started by: sunilmudikonda
3 Replies

LEARN ABOUT REDHAT

dos2unix

dos2unix(1)						      General Commands Manual						       dos2unix(1)

NAME

       dos2unix - DOS/MAC to UNIX text file format converter

SYNOPSYS

       dos2unix [options] [-c convmode] [-o file ...] [-n infile outfile ...]

       Options:

       [-hkqV] [--help] [--keepdate] [--quiet] [--version]

DESCRIPTION

       This manual page documents dos2unix, the program that converts plain text files in DOS/MAC format to UNIX format.

OPTIONS

       The following options are available:

       -h --help
	      Print online help.

       -k --keepdate
	      Keep the date stamp of output file same as input file.

       -q --quiet
	      Quiet mode. Suppress all warning and messages.

       -V --version
	      Prints version information.

       -c --convmode convmode
	      Sets conversion mode. Simulates dos2unix under SunOS.

       -o --oldfile file ...
	      Old file mode. Convert the file and write output to it. The program default to run in this mode. Wildcard names may be used.

       -n --newfile infile outfile ...
	      New  file  mode.	Convert the infile and write output to outfile. File names must be given in pairs and wildcard names should NOT be
	      used or you WILL lost your files.

EXAMPLES

       Get input from stdin and write output to stdout.

	      dos2unix

       Convert and replace a.txt. Convert and replace b.txt.

	      dos2unix a.txt b.txt

	      dos2unix -o a.txt b.txt

       Convert and replace a.txt in ASCII conversion mode.  Convert and replace b.txt in ISO conversion mode.  Convert	c.txt  from  Mac  to  Unix
       ascii format.

	      dos2unix a.txt -c iso b.txt

	      dos2unix -c ascii a.txt -c iso b.txt

	      dos2unix -c mac a.txt b.txt

       Convert and replace a.txt while keeping original date stamp.

	      dos2unix -k a.txt

	      dos2unix -k -o a.txt

       Convert a.txt and write to e.txt.

	      dos2unix -n a.txt e.txt

       Convert a.txt and write to e.txt, keep date stamp of e.txt same as a.txt.

	      dos2unix -k -n a.txt e.txt

       Convert and replace a.txt. Convert b.txt and write to e.txt.

	      dos2unix a.txt -n b.txt e.txt

	      dos2unix -o a.txt -n b.txt e.txt

       Convert c.txt and write to e.txt. Convert and replace a.txt.  Convert and replace b.txt. Convert d.txt and write to f.txt.

	      dos2unix -n c.txt e.txt -o a.txt b.txt -n d.txt f.txt

DIAGNOSTICS

BUGS

       The program does not work properly under MSDOS in stdio processing mode.  If you know why is that so, please tell me.

AUTHORS

       Benjamin Lin - <blin@socs.uts.edu.au>

       Bernd Johannes Wuebben (mac2unix mode) <wuebben@kde.org>

MISCELLANY

       Tested environment:

	      Linux 1.2.0 with GNU C 2.5.8

	      SunOS 4.1.3 with GNU C 2.6.3

	      MS-DOS 6.20 with Borland C++ 4.02

       Suggestions and bug reports are welcome.

SEE ALSO

       unix2dos(1) mac2unix(1)

1995.03.31							   dos2unix v3.0						       dos2unix(1)
All times are GMT -4. The time now is 07:34 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy