Find records with specific characters in 2 nd field
Hi ,
I have a requirement to read a file ( 5 fields , ~ delimited) and find the records which contain anything other than Alphabets, Numbers , comma ,space and dot . ie a-z and A-Z and 0-9 and . and " " and , in 2nd field. Once I do that i would want the result to have field1|<flag>
flag can be Y or N .
N - If 2nd field doesnt have anything other above mentioned characters.
Else Y .
I am able to achieve this using below code by reading line by line . Please note second field is "address".
The issue is it is very ineffective and takes almost 30 mins for 100K records. Can I improve it by using better logic. May be by avoiding reading line by line.
Moderator's Comments:
Please use CODE tags as required by forum rules!
Last edited by RudiC; 08-30-2018 at 06:40 PM..
Reason: Changed HTML to CODE tags.
Hello,
Below is my input file's content ( in HP-UX platform ):
ABCD120672-B21 1
ABCD142257-002 1
ABCD142257-003 1
ABCD142257-006 1
From the above, I just want to get the field of 13 characters that comes after 'ABCD' i.e '120672-B21'... . Could... (2 Replies)
I have a text file which represents a http flow like this:
HTTP/1.1 200 OK
Date: Fri, 23 Jan 2009 17:16:24 GMT
Server: Apache
Last-Modified: Fri, 23 Jan 2009 17:08:03 GMT
Accept-Ranges: bytes
Cache-Control: max-age=540
Expires: Fri, 23 Jan 2009 17:21:31 GMT
Vary: Accept-Encoding ... (1 Reply)
Hi Eveyone,
I am working on one shell script to find the specific records from data file and add the totals into variables and print them. you can find the sample data file below for more clarification.
Sample Data File:
PXSTYL00__20090803USA
CHCART00__20090803IND... (7 Replies)
Hi everyone.
I am a newbie to Linux stuff. I have this kind of problem which couldn't solve alone. I have a text file with records separated by empty lines like this:
ID: 20
Name: X
Age: 19
ID: 21
Name: Z
ID: 22
Email: xxx@yahoo.com
Name: Y
Age: 19
I want to grep records that... (4 Replies)
Hello,
I have a file like following:
ALB_13554 1 1 1
ALB_13554 1 2 1
ALB_18544 2 0 2
ALB_18544 1 0 1
This is a sample of my file, my real file has 441845 number of fields. What I want to do is to calculate the number of 1 and 2 in each column using AWK, so, the output file looks like... (5 Replies)
I am trying to find a specific set of characters in a long file. I only want to find the characters in column 265 for 4 bytes.
Is there a search for that? I tried cut but couldn't get it to work.
Ex. I want to find '9999' in column 265 for 4 bytes. If it is in there, I want it to print... (12 Replies)
Need command for position based replace:
I need a command to replace with 0 for characters in the positions 11 to 20 to all the lines starts with 6 in a file.
For example the file ABC.txt has:
abcdefghijklmnopqrstuvwxyz
6abcdefghijklmnopqrstuvwxyz
abcdefghijklmnopqrstuvwxyz... (4 Replies)
I would like to remove all characters starting with "%" and ending with ")" in the 4th field - please help!!
1412007819.864 /device/services/heartbeatxx 204 0.547%!i(int=0) 0.434 0.112
1412007819.866 /device/services/heartbeatxx 204 0.547%!i(int=1) 0.423 0.123... (10 Replies)
Hi All,
Seeking for your assistance to get the difference of field1 and field2 and output all the records if there's a difference. please see below scenario.
file1.txt
250|UPTREND FASHION DESIGN,CORP.|2016-04-04 09:36:13.991257
74|MAINSTREAM BUSINESS INC.|2016-04-04 09:36:13.991257... (1 Reply)
I'm trying to do something like this:
find . -name blablabla -exec ln -s ./"{:53:14} blablabla" \;
The idea is find blablabla and create a symbolic link to it using part of it's path and then it's name, "blablabla."
I just don't know if I can call characters out of a find variable. ... (16 Replies)
Discussion started by: scribling
16 Replies
LEARN ABOUT POSIX
regex
regex(1F) FMLI Commands regex(1F)NAME
regex - match patterns against a string
SYNOPSIS
regex [-e] [ -v "string"] [ pattern template] ... pattern [template]
DESCRIPTION
The regex command takes a string from the standard input, and a list of pattern / template pairs, and runs regex() to compare the string
against each pattern until there is a match. When a match occurs, regex writes the corresponding template to the standard output and
returns TRUE. The last (or only) pattern does not need a template. If that is the pattern that matches the string, the function simply
returns TRUE. If no match is found, regex returns FALSE.
The argument pattern is a regular expression of the form described in regex(). In most cases, pattern should be enclosed in single quotes
to turn off special meanings of characters. Note that only the final pattern in the list may lack a template.
The argument template may contain the strings $m0 through $m9, which will be expanded to the part of pattern enclosed in ( ... )$0 through
( ... )$9 constructs (see examples below). Note that if you use this feature, you must be sure to enclose template in single quotes so
that FMLI does not expand $m0 through $m9 at parse time. This feature gives regex much of the power of cut(1), paste(1), and grep(1), and
some of the capabilities of sed(1). If there is no template, the default is $m0$m1$m2$m3$m4$m5$m6$m7$m8$m9.
OPTIONS
The following options are supported:
-e Evaluates the corresponding template and writes the result to the standard output.
-v "string" Uses string instead of the standard input to match against patterns.
EXAMPLES
Example 1: Cutting letters out of a string
To cut the 4th through 8th letters out of a string (this example will output strin and return TRUE):
`regex -v "my string is nice" '^.{3}(.{5})$0' '$m0'`
Example 2: Validating input in a form
In a form, to validate input to field 5 as an integer:
valid=`regex -v "$F5" '^[0-9]+$'`
Example 3: Translating an environment variable in a form
In a form, to translate an environment variable which contains one of the numbers 1, 2, 3, 4, 5 to the letters a, b, c, d, e:
value=`regex -v "$VAR1" 1 a 2 b 3 c 4 d 5 e '.*' 'Error'`
Note the use of the pattern '.*' to mean "anything else".
Example 4: Using backquoted expressions
In the example below, all three lines constitute a single backquoted expression. This expression, by itself, could be put in a menu defini-
tion file. Since backquoted expressions are expanded as they are parsed, and output from a backquoted expression (the cat command, in this
example) becomes part of the definition file being parsed, this expression would read /etc/passwd and make a dynamic menu of all the login
ids on the system.
`cat /etc/passwd | regex '^([^:]*)$0.*$' '
name=$m0
action=`message "$m0 is a user"`'`
DIAGNOSTICS
If none of the patterns match, regex returns FALSE, otherwise TRUE.
NOTES
Patterns and templates must often be enclosed in single quotes to turn off the special meanings of characters. Especially if you use the
$m0 through $m9 variables in the template, since FMLI will expand the variables (usually to "") before regex even sees them.
Single characters in character classes (inside []) must be listed before character ranges, otherwise they will not be recognized. For exam-
ple, [a-zA-Z_/] will not find underscores (_) or slashes (/), but [_/a-zA-Z] will.
The regular expressions accepted by regcmp differ slightly from other utilities (that is, sed, grep, awk, ed, and so forth).
regex with the -e option forces subsequent commands to be ignored. In other words, if a backquoted statement appears as follows:
`regex -e ...; command1; command2`
command1 and command2 would never be executed. However, dividing the expression into two:
`regex -e ...``command1; command2`
would yield the desired result.
ATTRIBUTES
See attributes(5) for descriptions of the following attributes:
+-----------------------------+-----------------------------+
| ATTRIBUTE TYPE | ATTRIBUTE VALUE |
+-----------------------------+-----------------------------+
|Availability |SUNWcsu |
+-----------------------------+-----------------------------+
SEE ALSO awk(1), cut(1), grep(1), paste(1), sed(1), regcmp(3C), attributes(5)SunOS 5.10 12 Jul 1999 regex(1F)