Grok filter to extract substring from path and add to host field in logstash

Tags

shell scripts

Top Forums Shell Programming and Scripting Grok filter to extract substring from path and add to host field in logstash

04-24-2015

Moderator

1,837, 668

Join Date: Nov 2012

Last Activity: 30 June 2020, 12:07 PM EDT

Posts: 1,837

Thanks Given: 180

Thanked 668 Times in 590 Posts

Quote:

Originally Posted by RavinderSingh13

Thanks Akshay for pointing it out, I was trying first someting else with match before and later used gsub while posting I forgot to remove match from it. It can be as follows too.

Code:

echo "cdc2a_test3_heep.log" | awk '{gsub(/.*\//,X,$0);gsub(/_.*/,Y,$0);print $0}'
OR
echo "/data/app1a_test2_heep.log" | awk '{gsub(/.*\//,X,$0);gsub(/_.*/,Y,$0);print $0}'

Thanks,
R. Singh

Okay Ravinder, keep in mind before posting answers, sometimes it confuses readers, ultimately you can simplify like this

Code:

$ echo "cdc2a_test3_heep.log" | awk 'gsub(/.*\/|_.*/,"")'
cdc2a

$ echo "/data/app1a_test2_heep.log" | awk 'gsub(/.*\/|_.*/,"")'
app1a

Akshay Hegde

View Public Profile for Akshay Hegde

Find all posts by Akshay Hegde

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to update field using matching value in file1 and substring in field in file2

In the awk below I am trying to set/update the value of $14 in file2 in bold, using the matching NM_ in $12 or $9 in file2 with the NM_ in $2 of file1. The lengths of $9 and $12 can be variable but what is consistent is the start pattern will always be NM_ and the end pattern is always ;...

2. Shell Programming and Scripting

Extract a substring from a file

Hello, A question please. A have a file that contains a string. Ex: AAAABBCCCCCDDEEEEEEEEEEFF I'd want to recover 2 substrings, 'BB' and 'FF' and then leave them in a new file. From position 5, 2 caracters (ex:"BB") and from position 25, 2 caracters (ex:"FF") in a file. Could anoyone help me...

3. Shell Programming and Scripting

Filter uniq field values (non-substring)

Hello, I want to filter column based on string value. All substring matches are filtered out and only unique master strings are picked up. infile: 1 abcd 2 abc 3 abcd 4 cdef 5 efgh 6 efgh 7 efx 8 fgh Outfile: 1 abcd 4 cdef 5 efgh 7 efxI have tried awk '!a++; match(a, $2)>0'...

4. Shell Programming and Scripting

Filter specified path

We have 10 jobs entry in crontab like this 0 7 * * 0 && (source /x/y/z .bashrc ; /x/y/z /test.sh Table1 /ABC/TEST >x/y/z/log (every job have different o/p Path) can any one help me to filter only the output PATH /ABC/TEST from the file dup_cron.

5. Shell Programming and Scripting

How to extract a substring from a string

Hi, I have an input string say for example: ABC,DEF,IJK,LMN,...,XYZ The above string is comma delimited. Now I have to extract the last part after the comma i.e. XYZ. :b:

6. Shell Programming and Scripting

one liner to extract path from PATH variable

Hi, Could anyone help me in writing a single line code by either using (sed, awk, perl or whatever) to extract a specific path from the PATH environment variable? for eg: suppose the PATH is being set as follows PATH=/usr/bin/:/usr/local/bin:/bin:/usr/sbin:/usr/bin/java:/usr/bin/perl3.4 ...

7. Solaris

Extract substring from a string

i have srtring i.e. "NAME,CLASS,AGE" (length of string is not constant) and from this string i've extract each word delimited by "," (comma). INPUT: "NAME,CLASS,AGE" OUTPUT: NAME CLASS AGE how can i do that? i have tried some string manipulation function like...

8. Shell Programming and Scripting

Extract a substring.

I have a shell script that uses wget to grab a bunch of html from a url. URL_DATA=`wget -qO - "$URL1"` I now have a string $URL_DATA that I need to pull a substring out of..say I had the following in my string <p><a href="/scooby/929011567.html">Dog pictures check them out! -</a><font...

9. Shell Programming and Scripting

Need Help... to extract the substring

> tnsping $TWO_TASK | grep HOST Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = 10.12.10.212)(PORT = 1540)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = OMTST15))) I want to extract like this HOST = 10.12.10.212 PORT = 1540 SERVICE_NAME = OMTST15 I...

10. Shell Programming and Scripting

Sed extract substring on (OS X)

On OS 10.4.11 I have filenames like: 670711 SA T2 v1-1_DS_EF.doc CT_670520 AM T1 v1-2_DS_EF.doc CT_670716 - 2 SA T4 v1-2_DS_EF.doc CT_670713 SA T3 v1-1_DS_EF.doc 670421 PA DYP1 v1-1_DS_EF.doc CT_670425 PA DYP2 v1-1_DS_EF.doc CT_670107 RA T3 v1-2_DS_EF.doc CT_670521 AM T2 v1-2_DS_EF.doc...

Shell Programming and Scripting