Script for extraction of pattern


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Script for extraction of pattern
# 15  
Old 02-20-2017
Is that always 629100 or can it be another string? Is it always in that exact position within the field, or any place?
# 16  
Old 02-20-2017
For each block, it's always in the same position and in the same format and the epsProfileId also in the same position

Code:
dn: EpsStaInfId=EpsStaInf,serv=EPS,mscId=aaaaaa001aaaaaaaa629100100138702,ou=multiSCs,dc=mtncg
structuralObjectClass: EpsStaticInf
objectClass: EpsStaticInf
entryDS: 1
nodeId: 1
createTimestamp: 20170119123400Z
modifyTimestamp: 20170119123400Z
EpsStaInfId: EpsStaInf
EpsProfileId: 29
EpsOdb: 0
EpsRoamAllow: TRUE
CDC: 1
EpsIndSubChargChars: 123

# 17  
Old 02-20-2017
Would that work?
Code:
perl -00 -ne '($m,$e)=/(mscId=[^,]*).*(EpsProfileId:\s+\d+)/s; $m=~/629100/ and print "$m,$e\n"' gillesi.input

or
Code:
perl -00 -ne '($m,$e)=/(mscId=[^,]*).*(EpsProfileId:\s+\d+)/s;$m=~/\d{15}$/ and print "$m,$e\n"' gillesi.input

Code:
mscId=aaaaaa001aaaaaaaa629100100138702,EpsProfileId: 29
mscId=aaaaaa001aaaaaaaa629100100165619,EpsProfileId: 29


Last edited by Aia; 02-20-2017 at 07:10 PM..
# 18  
Old 02-20-2017
If you don't like perl, or if you want a commented script to help you understand what is going on, you could also consider the following bash and /usr/xpg4/bin/awk scripts that should work fine on a Solaris 10 system:
Code:
#!/bin/bash
# Extract script name from final component of path used to invoke this script.
IAm=${0##*/}

# Verify that we have been called with two operands...
if [ $# -ne 2 ]
then	# We were not called with two operands, print usage diagnostic and exit.
	printf 'Usage: %s input_filename output_filename\n' "$IAm" >&2
	exit 1
fi

# The 1st operand (input_filename) is assumed to contain multi-line records
# with each record containing a line of the form (without the leading "# "):
# dn: EpsStaInfId=EpsStaInf,serv=EPS,mscId=string1,ou=multiSCs,dc=mtncg
# followed by a later line in the record of the form (again without the
# leadin "# ":
# EpsProfileId: string2
# where "string1" in the 1st line is a 32 character fixed-length alphanumeric
# string.  If the 6 characters starting at position 18 in this string are
# "629100" as in "aaaaaa001aaaaaaaa629100100138702", we need to extract and
# print the 3rd comma separated field from the 1st line and the entire 2nd line
# and print them as comma separated fields in a single output line in the file
# named by the 2nd operand (output_filename).  If those 6 characters are any
# other string, nothgin from that record from input_filename will be included
# in output_filename.

# Use awk with input fields separated by colons and commas and the output field
# separator set to comma to perform the above specified actions.
/usr/xpg4/bin/awk -F '[:,]' -v OFS=, '
# If the 1st field on the line is the string "dn", determine whether or not
# this is an unencrypted record.
$1 == "dn" {
	# Uncomment the next line to see the fields we are examining.
	#printf("*$1=\"%s\" $4=\"%s\" substr($4,24,6)=\"%s\"\n", $1, $4, substr($4, 24, 6))

	# If the string "629100" appears starting at position 24 (note that we
	# have "mscId=string1" here, not just "string1"):
	if(substr($4, 24, 6) == "629100") {
		# it is, so we have an unencrypted record.
		unencrypted = 1
		# save the mscId field from this line to be printed later.
		mscId = $4
	} else {# t is not, so we have an encrypted record.
		unencrypted = 0
	}

	# We are done with this record, so move on to the next input line.
	next
}
# If we are processing an unencrypted record, look for the "EpsProfileId" data.
unencrypted && ($1 == "EpsProfileId") {
	# We found it.  Print the saved mscId from the previous "dn:" line and
	# the "EpsProfileId" data from this line.
	print mscId, $0
	# and clear the unencrypted flag to speed up processing until we find
	# the next "dn:" line.
	unencrypted = 0
}' "$1" > "$2"	# End the awk script naming the file to be proccessed and
		# redirecting output to the specified output file.

or an alternative awk approach (without the comments):
Code:
#!/bin/bash
# Extract script name from final component of path used to invoke this script.
IAm=${0##*/}

# Verify that we have been called with two operands...
if [ $# -ne 2 ]
then	# We were not called with two operands, print usage diagnostic and exit.
	printf 'Usage: %s input_filename output_filename\n' "$IAm" >&2
	exit 1
fi
/usr/xpg4/bin/awk -F '[:,]' -v OFS=, '
$1 == "dn" { m = (substr($4, 24, 6) == "629100")? $4 : 0; next }
m && ($1 == "EpsProfileId") { print m, $0; m = 0 }' "$1" > "$2"

both of which produce the output you said you wanted in post #12.

In both of these nawk should work as well as /usr/xpg4/bin/awk.

If you uncomment the debugging printf command in the 1st script, you'll get the output:
Code:
*$1="dn" $4="mscId=aaa8c4c6b30a4b1abcfce3990027d6a4" substr($4,24,6)="cfce39"
*$1="dn" $4="mscId=aaa9526a822d46979331a964be201cba" substr($4,24,6)="331a96"
*$1="dn" $4="mscId=aaaaaa001aaaaaaaa629100100138702" substr($4,24,6)="629100"
mscId=aaaaaa001aaaaaaaa629100100138702,EpsProfileId: 29
*$1="dn" $4="mscId=aaaaaa001aaaaaaaa629100100165619" substr($4,24,6)="629100"
mscId=aaaaaa001aaaaaaaa629100100165619,EpsProfileId: 29

showing how awk split the fields in the dn: lines.
These 2 Users Gave Thanks to Don Cragun For This Post:
# 19  
Old 02-22-2017
The first one is working fine. Gave me the output just how i wanted. ThANK YOU. Now i need a tutorial on how to use perl and awk please. A very simple straight forward tutorial.

---------- Post updated at 02:18 AM ---------- Previous update was at 02:10 AM ----------

Don Cragun thank you very much. This was a tutorial to me. The awk is working fine.
# 20  
Old 02-22-2017
I was wondering if you were using disks over nfs or something. That would explain the poor performance.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Extraction of data between parenthesis using multiple pattern

i want to extract all data with in parenthesis from a file by passing a pattern from another file.i have some sql statements in my file and i want to extract those ddl by refering to a pattern which is in another file and before writting into file i need some transformation to do.Basically i want... (1 Reply)
Discussion started by: raj121
1 Replies

2. Shell Programming and Scripting

Pattern Search and Extraction Problem

I have something like this: bash-3.2$ svn info Path: . URL: svn+ssh://nlaedev01@10.209.194.15/files0/nlae_dev_svn/repos/newlook-endeca/trunk Repository Root: svn+ssh://nlaedev01@10.209.194.15/files0/nlae_dev_svn/repos/newlook-endeca Repository UUID: 4e8fbe85-c2e2-42fe-a5fa-f9f9100d2393... (3 Replies)
Discussion started by: ankur328
3 Replies

3. Shell Programming and Scripting

Script to compare pattern and print a different pattern in each line

Hi, I am writing a shell script to parse some files, and gather data. The data in the files is displayed as below. .......xyz: abz: ......qrt: .... .......xyz: abz: ......qrt: ... I have tried using awk and cut, but the position of these values keep changing, so I wasn't able to get... (2 Replies)
Discussion started by: Serena
2 Replies

4. Shell Programming and Scripting

Pattern extraction and usage by shell script

Suppose im in a directory A. which has sub-directories x/y/z m/n/p etc. Iam only considered with those which have a file netl.oa at the lowermost level. So i used the find command which gives me a list in the form ./abc/def/ghi/jkl/netl.oa and so on Now i want the names abc def jkl and ghi. My... (3 Replies)
Discussion started by: sid.verycool
3 Replies

5. UNIX for Dummies Questions & Answers

Extraction of strings from a file, after pattern matching

I need to extract strings from a file. The file contains data like: Plan ABCD IN-+-172BB---118C2C---GGN_342-+-MM77_23--+-LAS24_3|GGK_774 | | \-LAS24_2|GGN_774 | +-AA_800_1-+-BAS_000|GGK_362 | | \-BAS_001|GGK_360 | \-DD_000T1---DAM_001|STEEL_0 Plan SHELL_1... (3 Replies)
Discussion started by: abkush
3 Replies

6. Shell Programming and Scripting

Date and time range extraction via Awk or analysis script?

Hello does anyone know of an awk that will extract log file entries between a specific date and time range, eg: awk '/15\/Dec\/2010:16:10:00/, /15\/Dec\/2010:16:15:00/' access_log but one that works? Or a free command line log file analysis tool/script? I'd like to be able to view... (2 Replies)
Discussion started by: competitions
2 Replies

7. Infrastructure Monitoring

Shell Script Extraction

Hello Users, I am new to unix. I have a requirement to extract the string in the folder with files names XXXX.sev.xxxxx.lookup (There are some more files which I am not interested in like xxxxx.include.xxx.lookup). 1) I am looking for the file with the name "sev" ending with "lookup" ... (11 Replies)
Discussion started by: reachravi70
11 Replies

8. UNIX for Advanced & Expert Users

extraction of data from a text file which follows certain pattern

hi everybody, i have a file, in it I need to extract some data that follows a particular pattern.. For example: my file contains like now running Speak225 sep 22 mon 16:34:05 2008 -------------------------------- ... (4 Replies)
Discussion started by: mohkris
4 Replies

9. Shell Programming and Scripting

Shell script for text extraction from a file

Hi All, I am new to Shell Scripting. I have a file consisting of XML messages.Each message is associated with a timestamp value(it is not a xml field).I need to extract\copy all messages in a particular time interval and put in another new file using Shell Scripting. My XML looks like... (3 Replies)
Discussion started by: vignesh53
3 Replies

10. Shell Programming and Scripting

help with data extraction script

Hello all, Iam newbie here and to unix programming. I have the following text file. A:Woshington,B:London,C:Paris,D:Manchester,C:Lisbon,E:Cape town. Now I would like extract this and store in database. here is the script I have tried but it did work. CITY1:`echo "$text" | grep "A:"... (11 Replies)
Discussion started by: mam
11 Replies
Login or Register to Ask a Question