Login or Register to Ask a Question and Join Our Community


UNIX for Beginners Questions & Answers

Script for extraction of pattern

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Script for extraction of pattern
# 8  
Old 02-19-2017
@ RudiC PLEASE i don't really get your point. When i said that i need the output file i was referring to the post #4. Which's been adressed in post #7.
As for my own attempts, i don't have any. If i had any attempt i wouldn't hesitate to put it here. Please some of these scripts are asked ASAP and i'm learner in scripting so it's somehow not easy for me. Please hope you get it.

---------- Post updated at 09:35 AM ---------- Previous update was at 09:23 AM ----------

Please this is another sample, actually we have blocks with mscId =aaa8c4c6b30a4.....(emcrypted) in the same file, but i don't need those.
@RudiC your code gave me the output but it gave me the encrypted instead of the one i want.

Code:
dn: EpsStaInfId=EpsStaInf,serv=EPS,mscId=aaa8c4c6b30a4b1abcfce3990027d6a4,ou=multiSCs,dc=mtncg
structuralObjectClass: EpsStaticInf
objectClass: EpsStaticInf
entryDS: 1
nodeId: 1
createTimestamp: 20170202223552Z
modifyTimestamp: 20170202223552Z
EpsStaInfId: EpsStaInf
EpsProfileId: 29
EpsOdb: 0
EpsRoamAllow: TRUE
CDC: 1
EpsIndSubChargChars: 123

dn: EpsStaInfId=EpsStaInf,serv=EPS,mscId=aaa9526a822d46979331a964be201cba,ou=multiSCs,dc=mtncg
structuralObjectClass: EpsStaticInf
objectClass: EpsStaticInf
entryDS: 1
nodeId: 1
createTimestamp: 20170119122933Z
modifyTimestamp: 20170119122933Z
EpsStaInfId: EpsStaInf
EpsProfileId: 29
EpsOdb: 0
EpsRoamAllow: TRUE
CDC: 1
EpsIndSubChargChars: 123

dn: EpsStaInfId=EpsStaInf,serv=EPS,mscId=aaaaaa001aaaaaaaa629100100138702,ou=multiSCs,dc=mtncg
structuralObjectClass: EpsStaticInf
objectClass: EpsStaticInf
entryDS: 1
nodeId: 1
createTimestamp: 20170119123400Z
modifyTimestamp: 20170119123400Z
EpsStaInfId: EpsStaInf
EpsProfileId: 29
EpsOdb: 0
EpsRoamAllow: TRUE
CDC: 1
EpsIndSubChargChars: 123

dn: EpsStaInfId=EpsStaInf,serv=EPS,mscId=aaaaaa001aaaaaaaa629100100165619,ou=multiSCs,dc=mtncg
structuralObjectClass: EpsStaticInf
objectClass: EpsStaticInf
entryDS: 1
nodeId: 1
createTimestamp: 20170211115449Z
modifyTimestamp: 20170211115449Z
EpsStaInfId: EpsStaInf
EpsProfileId: 29
EpsOdb: 0
EpsRoamAllow: TRUE
CDC: 1
EpsIndSubChargChars: 123

And about my own attempt, please i'm new in scripting and i'm learning, so for now i don't have any attempt, when i'm good i'll definitely post my attempt.
# 9  
Old 02-19-2017
Hi gillesi,
Note that the way to learn is to do. If you "don't have any attempt", you don't have any way to learn. If you expect this site to act as your unpaid programming staff instead of as a tool to help you learn how to improve your attempts to solve your own problems, I'm afraid we will all be disappointed.
# 10  
Old 02-19-2017
Waouh...ok. I'm really beginner here. But saying that you act as "unpaid programming staff" that's very frustrating to me. I feel like i'm begging for your help or you asking me out of this site. For my 1st thread below was my attempt and i said it was very slow, i spent days waiting for the output.
Code:
count=`grep -wc "MSISDN" DS1_HLR_export170217.ldif` >> OUTPUT

for k in `seq $count`
do

cat DS1_HLR_export170217.ldif| awk -F":" -v var="$k" '$1=="MSISDN" {m++}m==var{print; exit}' >> OUTPUT
cat DS1_HLR_export170217.ldif| awk -F":" -v var="$k" '$1=="IMSI" {m++}m==var{print; exit}' >> OUTPUT
cat DS1_HLR_export170217.ldif| awk -F":" -v var="$k" '$1=="NAM"  {m++}m==var{print; exit}' >> OUTPUT
cat DS1_HLR_export170217.ldif| awk -F":" -v var="$k" '$1=="TS11" {m++}m==var{print; exit}' >> OUTPUT
cat DS1_HLR_export170217.ldif| awk -F":" -v var="$k" '$1=="TS21" {m++}m==var{print; exit}' >> OUTPUT
cat DS1_HLR_export170217.ldif| awk -F":" -v var="$k" '$1=="TS22" {m++}m==var{print; exit}' >> OUTPUT
cat DS1_HLR_export170217.ldif| awk -F":" -v var="$k" '$1=="TS62" {m++}m==var{print; exit}' >> OUTPUT
cat DS1_HLR_export170217.ldif| awk -F":" -v var="$k" '$1=="BAIC" {m++}m==var{print; exit}' >> OUTPUT
cat DS1_HLR_export170217.ldif| awk -F":" -v var="$k" '$1=="BAOC" {m++}m==var{print; exit}' >> OUTPUT
cat DS1_HLR_export170217.ldif| awk -F":" -v var="$k" '$1=="APNID1" {m++}m==var{print; exit}' >> OUTPUT
cat DS1_HLR_export170217.ldif| awk -F":" -v var="$k" '$1=="APNID2" {m++}m==var{print; exit}' >> OUTPUT
echo " " >> OUTPUT
done

# 11  
Old 02-20-2017
Quote:
Originally Posted by gillesi
Waouh...ok. I'm really beginner here. But saying that you act as "unpaid programming staff" that's very frustrating to me. I feel like i'm begging for your help or you asking me out of this site. For my 1st thread below was my attempt and i said it was very slow, i spent days waiting for the output.
Code:
count=`grep -wc "MSISDN" DS1_HLR_export170217.ldif` >> OUTPUT

for k in `seq $count`
do

cat DS1_HLR_export170217.ldif| awk -F":" -v var="$k" '$1=="MSISDN" {m++}m==var{print; exit}' >> OUTPUT
cat DS1_HLR_export170217.ldif| awk -F":" -v var="$k" '$1=="IMSI" {m++}m==var{print; exit}' >> OUTPUT
cat DS1_HLR_export170217.ldif| awk -F":" -v var="$k" '$1=="NAM"  {m++}m==var{print; exit}' >> OUTPUT
cat DS1_HLR_export170217.ldif| awk -F":" -v var="$k" '$1=="TS11" {m++}m==var{print; exit}' >> OUTPUT
cat DS1_HLR_export170217.ldif| awk -F":" -v var="$k" '$1=="TS21" {m++}m==var{print; exit}' >> OUTPUT
cat DS1_HLR_export170217.ldif| awk -F":" -v var="$k" '$1=="TS22" {m++}m==var{print; exit}' >> OUTPUT
cat DS1_HLR_export170217.ldif| awk -F":" -v var="$k" '$1=="TS62" {m++}m==var{print; exit}' >> OUTPUT
cat DS1_HLR_export170217.ldif| awk -F":" -v var="$k" '$1=="BAIC" {m++}m==var{print; exit}' >> OUTPUT
cat DS1_HLR_export170217.ldif| awk -F":" -v var="$k" '$1=="BAOC" {m++}m==var{print; exit}' >> OUTPUT
cat DS1_HLR_export170217.ldif| awk -F":" -v var="$k" '$1=="APNID1" {m++}m==var{print; exit}' >> OUTPUT
cat DS1_HLR_export170217.ldif| awk -F":" -v var="$k" '$1=="APNID2" {m++}m==var{print; exit}' >> OUTPUT
echo " " >> OUTPUT
done

We are not asking you out of this site; we are asking you to pay attention to the suggestions that have been made, we are asking you to clearly describe the format of the input you are processing, we are asking you to clearly describe the output you are trying to produce, we are asking you to answer the questions that we ask, and we are asking you to show us the output you get when you try to run the code we suggest. We are asking you to actively participate in the production of the code needed to produce the output you want! (Note that saying: "It doesn't work." without showing us the EXACT diagnostic messages produced DOES NOT HELP US HELP YOU.)

Above you have shown us a script that you say produces the output you want, but runs too slow. If your input records are in the same order as the output you want, that script could be replaced by a single grep command. But, the multi-line output this code produces is nothing at all like the single output line per input record CSV format output that you said you wanted in your other thread. And, in this thread, you have said that you want only two output fields (but it isn't clear whether the first field is the entire line started with dn: , whether you just want the mscId=value, or whether you want anything at all from that line other than to note that it is the 1st line of each input file record.

For us to to be able to help you in either of these threads, you need to understand what output you are trying to produce, explain clearly to us what that output format is, and show us exactly what output you want produced from a representative sample input file.

You have said that there are some records with emcrypted (sic) values that are to be ignored, but you haven't given us any indication of how to determine which records are to be included and which records are to be ignored. Please clarify your requirements in this regard.
# 12  
Old 02-20-2017
---------- Post updated at 09:35 AM ---------- Previous update was at 09:23 AM ----------

INPUT FILE:
The mscId in the first two block(highlighted in red, encrypted) i don't need them.
In my input file there are thousands of blocks like these followed by the last two block in which i need the mscId highlighted in green. So if there are 500 blocks with the encypted mscId there are also 500 blocks with mscId highlighted in green. Therefore i need the mscId and EpsProfile highlighted in green.

Code:
dn: EpsStaInfId=EpsStaInf,serv=EPS,mscId=aaa8c4c6b30a4b1abcfce3990027d6a4,ou=multiSCs,dc=mtncg
structuralObjectClass: EpsStaticInf
objectClass: EpsStaticInf
entryDS: 1
nodeId: 1
createTimestamp: 20170202223552Z
modifyTimestamp: 20170202223552Z
EpsStaInfId: EpsStaInf
EpsProfileId: 29
EpsOdb: 0
EpsRoamAllow: TRUE
CDC: 1
EpsIndSubChargChars: 123

dn: EpsStaInfId=EpsStaInf,serv=EPS,mscId=aaa9526a822d46979331a964be201cba,ou=multiSCs,dc=mtncg
structuralObjectClass: EpsStaticInf
objectClass: EpsStaticInf
entryDS: 1
nodeId: 1
createTimestamp: 20170119122933Z
modifyTimestamp: 20170119122933Z
EpsStaInfId: EpsStaInf
EpsProfileId: 29
EpsOdb: 0
EpsRoamAllow: TRUE
CDC: 1
EpsIndSubChargChars: 123

dn: EpsStaInfId=EpsStaInf,serv=EPS,mscId=aaaaaa001aaaaaaaa629100100138702,ou=multiSCs,dc=mtncg
structuralObjectClass: EpsStaticInf
objectClass: EpsStaticInf
entryDS: 1
nodeId: 1
createTimestamp: 20170119123400Z
modifyTimestamp: 20170119123400Z
EpsStaInfId: EpsStaInf
EpsProfileId: 29
EpsOdb: 0
EpsRoamAllow: TRUE
CDC: 1
EpsIndSubChargChars: 123

dn: EpsStaInfId=EpsStaInf,serv=EPS,mscId=aaaaaa001aaaaaaaa629100100165619,ou=multiSCs,dc=mtncg
structuralObjectClass: EpsStaticInf
objectClass: EpsStaticInf
entryDS: 1
nodeId: 1
createTimestamp: 20170211115449Z
modifyTimestamp: 20170211115449Z
EpsStaInfId: EpsStaInf
EpsProfileId: 29
EpsOdb: 0
EpsRoamAllow: TRUE
CDC: 1
EpsIndSubChargChars: 123

wanted output:
Code:
mscId=aaaaaa001aaaaaaaa629100100138702,EpsProfileId: 29
mscId=aaaaaa001aaaaaaaa629100100165619,EpsProfileId: 29

Moderator's Comments:
Mod Comment Please use CODE tags when displaying sample input, sample output, and code segments; not just sample input.
Last edited by Don Cragun; 02-20-2017 at 04:18 AM.. Reason: Add missing CODE tags.
# 13  
Old 02-20-2017
It is very nice to know that you don't need the encrypted records and the you need the unencrypted records, but I repeat:
Quote:
You have said that there are some records with emcrypted (sic) values that are to be ignored, but you haven't given us any indication of how to determine which records are to be included and which records are to be ignored. Please clarify your requirements in this regard.
Given that the string values for the mscId fields that you have highlighted in red as being encrypted are:
Code:
aaa8c4c6b30a4b1abcfce3990027d6a4
aaa9526a822d46979331a964be201cba

and the string values for the mscId fields that you have highlighted in green as being unencrypted are:
Code:
aaaaaa001aaaaaaaa629100100138702
aaaaaa001aaaaaaaa629100100165619

both sets of which are 32-character lower-case alphanumeric strings and the remainder of those lines are identical in the encrypted and unencrypted cases, how did you determine which of those strings are encrypted and which are unencrypted?
# 14  
Old 02-20-2017
Quote:
Originally Posted by Don Cragun
It is very nice to know that you don't need the encrypted records and the you need the unencrypted records, but I repeat:

Given that the string values for the mscId fields that you have highlighted in red as being encrypted are:
Code:
aaa8c4c6b30a4b1abcfce3990027d6a4
aaa9526a822d46979331a964be201cba

and the string values for the mscId fields that you have highlighted in green as being unencrypted are:
Code:
aaaaaa001aaaaaaaa629100100138702
aaaaaa001aaaaaaaa629100100165619

both sets of which are 32-character lower-case alphanumeric strings and the remainder of those lines are identical in the encrypted and unencrypted cases, how did you determine which of those strings are encrypted and which are unencrypted?
In the unencrypted format the values 629100 should distinguish the unencrypted mscId i want to extract aaaaaa001aaaaaaaa629100100165619
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Extraction of data between parenthesis using multiple pattern

i want to extract all data with in parenthesis from a file by passing a pattern from another file.i have some sql statements in my file and i want to extract those ddl by refering to a pattern which is in another file and before writting into file i need some transformation to do.Basically i want... (1 Reply)
Discussion started by: raj121
1 Replies

2. Shell Programming and Scripting

Pattern Search and Extraction Problem

I have something like this: bash-3.2$ svn info Path: . URL: svn+ssh://nlaedev01@10.209.194.15/files0/nlae_dev_svn/repos/newlook-endeca/trunk Repository Root: svn+ssh://nlaedev01@10.209.194.15/files0/nlae_dev_svn/repos/newlook-endeca Repository UUID: 4e8fbe85-c2e2-42fe-a5fa-f9f9100d2393... (3 Replies)
Discussion started by: ankur328
3 Replies

3. Shell Programming and Scripting

Script to compare pattern and print a different pattern in each line

Hi, I am writing a shell script to parse some files, and gather data. The data in the files is displayed as below. .......xyz: abz: ......qrt: .... .......xyz: abz: ......qrt: ... I have tried using awk and cut, but the position of these values keep changing, so I wasn't able to get... (2 Replies)
Discussion started by: Serena
2 Replies

4. Shell Programming and Scripting

Pattern extraction and usage by shell script

Suppose im in a directory A. which has sub-directories x/y/z m/n/p etc. Iam only considered with those which have a file netl.oa at the lowermost level. So i used the find command which gives me a list in the form ./abc/def/ghi/jkl/netl.oa and so on Now i want the names abc def jkl and ghi. My... (3 Replies)
Discussion started by: sid.verycool
3 Replies

5. UNIX for Dummies Questions & Answers

Extraction of strings from a file, after pattern matching

I need to extract strings from a file. The file contains data like: Plan ABCD IN-+-172BB---118C2C---GGN_342-+-MM77_23--+-LAS24_3|GGK_774 | | \-LAS24_2|GGN_774 | +-AA_800_1-+-BAS_000|GGK_362 | | \-BAS_001|GGK_360 | \-DD_000T1---DAM_001|STEEL_0 Plan SHELL_1... (3 Replies)
Discussion started by: abkush
3 Replies

6. Shell Programming and Scripting

Date and time range extraction via Awk or analysis script?

Hello does anyone know of an awk that will extract log file entries between a specific date and time range, eg: awk '/15\/Dec\/2010:16:10:00/, /15\/Dec\/2010:16:15:00/' access_log but one that works? Or a free command line log file analysis tool/script? I'd like to be able to view... (2 Replies)
Discussion started by: competitions
2 Replies

7. Infrastructure Monitoring

Shell Script Extraction

Hello Users, I am new to unix. I have a requirement to extract the string in the folder with files names XXXX.sev.xxxxx.lookup (There are some more files which I am not interested in like xxxxx.include.xxx.lookup). 1) I am looking for the file with the name "sev" ending with "lookup" ... (11 Replies)
Discussion started by: reachravi70
11 Replies

8. UNIX for Advanced & Expert Users

extraction of data from a text file which follows certain pattern

hi everybody, i have a file, in it I need to extract some data that follows a particular pattern.. For example: my file contains like now running Speak225 sep 22 mon 16:34:05 2008 -------------------------------- ... (4 Replies)
Discussion started by: mohkris
4 Replies

9. Shell Programming and Scripting

Shell script for text extraction from a file

Hi All, I am new to Shell Scripting. I have a file consisting of XML messages.Each message is associated with a timestamp value(it is not a xml field).I need to extract\copy all messages in a particular time interval and put in another new file using Shell Scripting. My XML looks like... (3 Replies)
Discussion started by: vignesh53
3 Replies

10. Shell Programming and Scripting

help with data extraction script

Hello all, Iam newbie here and to unix programming. I have the following text file. A:Woshington,B:London,C:Paris,D:Manchester,C:Lisbon,E:Cape town. Now I would like extract this and store in database. here is the script I have tried but it did work. CITY1:`echo "$text" | grep "A:"... (11 Replies)
Discussion started by: mam
11 Replies
Login or Register to Ask a Question