Sponsored Content
Top Forums Shell Programming and Scripting How to handle grepping variable data containing wildcards? Post 302983348 by abercrom on Monday 10th of October 2016 01:16:56 PM
Old 10-10-2016
How to handle grepping variable data containing wildcards?

I have a lot of files with keywords and unique names. I'm using a shell script to refer to a simple pattern file with comma separated values in order to match on certain keywords. The problem is that I don't understand how to handle the wildcard values when I want to skip over the unique names.

Here is an example of the data patterns in the files:

Code:
abcd efgh ijkl
abcd efgh ijkl
abcd mnop unique-name01 efgh ijkl
abcd mnop unique-name02 efgh ijkl
abcd efgh ijkl


An example of the pattern file:

Code:
abcd efgh ijkl,LABEL001 LABEL002
mnop .* efgh ijkl,LABEL001 LABEL003


The shell script operates like this:

Code:
while IFS= read -r fi
  fn=`awk -F, '{print $NF}'`
  fp=`awk -F, '{print $1}'`

  echo "Pattern: \"$fp\""
  egrep "$fp" $source_file

done < pattern_file


The result looks like this:

Code:
Pattern: "abcd efgh ijkl"

[matches]
Code:
Pattern: "abcd mnop . .. efgh ijkl"

[no matches]

I get correct matches on the patterns that do not use a wildcards, so I know the script is working in that respect.
The wildcard patterns do not work, and it looks like the wildcards themselves are getting mangled somehow. In this example, the pattern has changed from "abcd mnop .* efgh ijkl" to "abcd mnop . .. efgh ijkl"

Thanks

Last edited by Scrutinizer; 10-10-2016 at 02:20 PM.. Reason: additional code tags for data samples
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Grepping using multiple wildcards

Is there anyway you can grep using multiple wildcards? When I run the below line the results return fine; grep 12345 /usr/local/production/soccermatchplus/distributor/clients/*/out/fixtures.xml | awk -F/ '{print $8}' However ideally, I need it to grep for; grep 12345... (3 Replies)
Discussion started by: JayC89
3 Replies

2. UNIX for Dummies Questions & Answers

grepping a variable

I need to pass a parameter that will then be grepped. I need it to grep /paramater and then have a space so if 123 was passed my grep would be grep '/123 ' sample.log this works fine from the command line but when i try and set it searchThis="/$2 " and then run grep $searchThis... (6 Replies)
Discussion started by: magnia
6 Replies

3. Shell Programming and Scripting

Grepping Date Variable

Hello, I would like for the user to input the date for a particular log file, then have the input sent to a variable, which is then used via grep to extra the logs for the specific date the user request. I did some searching, but I still don't understand why I'm not seeing any result. ... (4 Replies)
Discussion started by: ravzter
4 Replies

4. Shell Programming and Scripting

problem in using cgi_script to handle form data

Hi, I have an HTML form through which I get some text as input. i need to run a shell script say script.sh inside a perl-cgi script named main_cgi.sh on the form input. I want to write the contents of the form in a file and then perform some command line operations like grep, cat on the text... (0 Replies)
Discussion started by: piyush_priya
0 Replies

5. Shell Programming and Scripting

Help with grepping data from a text file

Hello, I have a text file which contains a list of strings which I want to grep from another file where these strings occur and print out only these lines. I had earlier used the grep command where File1 was the file containing the strings to be grepped (Source File) and File2 the Target File... (4 Replies)
Discussion started by: gimley
4 Replies

6. Shell Programming and Scripting

Grepping for variable in brackets

I have a script which reads a number out of a log file. The pertinent line is this: cat /tmp/listofnumbers When I run cat /tmp/listofnumbers what I am seeing is on each line. I am trying to make the script read from that file and grep for a variable like the following line should do: ... (4 Replies)
Discussion started by: newbie2010
4 Replies

7. Shell Programming and Scripting

Grepping data using awk

Hello, I have a data in file 1 2000000024776 2000000026979 2000000033355 2000000036309 2000000041291 2000000042679 2000000067221 and in file 2 its like this 2000000024776 16 2000000026979 16 2000000033355 16 (2 Replies)
Discussion started by: mirwasim
2 Replies

8. Shell Programming and Scripting

Grepping the data between 2 date ranges

Hi There, Good Day !! I have txt file containing data in the below format. There are many lines, here i have mentioned for example. cat remo.txt 2/3/2017 file1 3/4/2016 file2 6/6/2015 file5 1/1/2018 file3 4/3/2014 file4 - - - I need to grep the file names for given date rage... (11 Replies)
Discussion started by: kumar85shiv
11 Replies

9. UNIX for Beginners Questions & Answers

Expand Variables and Wildcards into another variable.

Dear Forum members, I am having trouble getting the complete filename (and directory path) in a variable. Output directory mentioned in the code have three files: DISPLAY_CITY_DETAILS_15-05-2019-08-29-26_MIGRATE_london.out DISPLAY_CITY_DETAILS_15-05-2019-08-29-26_MIGRATE_paris.out... (4 Replies)
Discussion started by: chetanojha
4 Replies

10. UNIX for Beginners Questions & Answers

Grepping for one variable while using awk to parse an associated variable

Im trying to search for a single variable in the first field and from that output use awk to extract out the lines that contain a value less than a value stored in another variable. Both the variables are associated with each other. Any guidance is appreciated. File that contains the... (6 Replies)
Discussion started by: ncwxpanther
6 Replies
fmtmsg(1)							   User Commands							 fmtmsg(1)

NAME
fmtmsg - display a message on stderr or system console SYNOPSIS
fmtmsg [-c class] [-u subclass] [-l label] [-s severity] [-t tag] [-a action] text DESCRIPTION
Based on a message's classification component, the fmtmsg utility either writes a formatted message to stderr or writes a formatted message to the console. A formatted message consists of up to five standard components (see environment variable MSGVERB in the ENVIRONMENT VARIABLES section of this page). The classification and subclass components are not displayed as part of the standard message, but rather define the source of the message and direct the display of the formatted message. OPTIONS
The following options are supported: -c class Describes the source of the message. Valid keywords are: hard The source of the condition is hardware. soft The source of the condition is software. firm The source of the condition is firmware. -u subclass A list of keywords (separated by commas) that further defines the message and directs the display of the message. Valid keywords are: appl The condition originated in an application. This keyword should not be used in combination with either util or opsys. util The condition originated in a utility. This keyword should not be used in combination with either appl or opsys. opsys The message originated in the kernel. This keyword should not be used in combination with either appl or util. recov The application will recover from the condition. This keyword should not be used in combination with nrecov. nrecov The application will not recover from the condition. This keyword should not be used in combination with recov. print Print the message to the standard error stream stderr. console Write the message to the system console. print, console, or both may be used. -l label Identifies the source of the message. -s severity Indicates the seriousness of the error. The keywords and definitions of the standard levels of severity are: halt The application has encountered a severe fault and is halting. error The application has detected a fault. warn The application has detected a condition that is out of the ordinary and might be a problem. info The application is providing information about a condition that is not in error. -t tag The string containing an identifier for the message. -a action A text string describing the first step in the error recovery process. This string must be written so that the entire action argument is interpreted as a single argument. fmtmsg precedes each action string with the TO FIX: prefix. text A text string describing the condition. Must be written so that the entire text argument is interpreted as a single argu- ment. EXAMPLES
Example 1 Standard message format The following example of fmtmsg produces a complete message in the standard message format and displays it to the standard error stream. example% fmtmsg -c soft -u recov,print,appl -l UX:cat -s error -t UX:cat:001 -a "refer to manual" "invalid syntax" produces: UX:cat: ERROR: invalid syntax TO FIX: refer to manual UX:cat:138 Example 2 Using MSGVERB When the environment variable MSGVERB is set as follows: MSGVERB=severity:text:action and Example 1 is used, fmtmsg produces: ERROR: invalid syntax TO FIX: refer to manual Example 3 Using SEV_LEVEL When the environment variable SEV_LEVEL is set as follows: SEV_LEVEL=note,5,NOTE the following fmtmsg command: example% fmtmsg -c soft -u print -l UX:cat -s note -a "refer to manual" "invalid syntax" produces: NOTE: invalid syntax TO FIX: refer to manual and displays the message on stderr. ENVIRONMENT VARIABLES
The environment variables MSGVERB and SEV_LEVEL control the behavior of fmtmsg. MSGVERB is set by the administrator in the /etc/profile for the system. Users can override the value of MSGVERB set by the system by resetting MSGVERB in their own .profile files or by changing the value in their current shell session. SEV_LEVEL can be used in shell scripts. MSGVERB tells fmtmsg which message components to select when writing messages to stderr. The value of MSGVERB is a colon-separated list of optional keywords. MSGVERB can be set as follows: MSGVERB=[keyword[:keyword[:...]]] export MSGVERB Valid keywords are: label, severity, text, action, and tag. If MSGVERB contains a keyword for a component and the component's value is not the component's null value, fmtmsg includes that component in the message when writing the message to stderr. If MSGVERB does not include a keyword for a message component, that component is not included in the display of the message. The keywords may appear in any order. If MSGVERB is not defined, if its value is the null string, if its value is not of the correct format, or if it contains keywords other than the valid ones listed above, fmtmsg selects all components. MSGVERB affects only which message components are selected for display. All message components are included in console messages. SEV_LEVEL defines severity levels and associates print strings with them for use by fmtmsg. The standard severity levels shown below cannot be modified. Additional severity levels can be defined, redefined, and removed. 0 (no severity is used) 1 HALT 2 ERROR 3 WARNING 4 INFO SEV_LEVEL is set as follows: description is a comma-separated list containing three fields: SEV_LEVEL= [description[:description[:...]]] export SEV_LEVEL description=severity_keyword, level, printstring severity_keyword is a character string used as the keyword with the -s severity option to fmtmsg. level is a character string that evaluates to a positive integer (other than 0, 1, 2, 3, or 4, which are reserved for the standard severity levels). If the keyword severity_keyword is used, level is the severity value passed on to fmtmsg(3C). printstring is the character string used by fmtmsg in the standard message format whenever the severity value level is used. If SEV_LEVEL is not defined, or if its value is null, no severity levels other than the defaults are available. If a description in the colon separated list is not a comma separated list containing three fields, or if the second field of a comma separated list does not eval- uate to a positive integer, that description in the colon separated list is ignored. EXIT STATUS
The following exit values are returned: 0 All the requested functions were executed successfully. 1 The command contains a syntax error, an invalid option, or an invalid argument to an option. 2 The function executed with partial success, however the message was not displayed on stderr. 4 The function executed with partial success; however, the message was not displayed on the system console. 32 No requested functions were executed successfully. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ SEE ALSO
addseverity(3C), fmtmsg(3C), attributes(5) SunOS 5.11 20 Jul 1994 fmtmsg(1)
All times are GMT -4. The time now is 10:15 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy