Extract a portion of string from each line in Linux

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Extract a portion of string from each line in Linux
# 1  
Old 05-18-2018
Extract a portion of string from each line in Linux

Hi I have to extract the destination path information from each record the file is of variable length so I will not be able to use the print command.The search should start on variable "destinationPath" and it should end at immediate "," also the first field has to be printed


Input File:
Code:
1|"delete":true,"destinationPath":"/cd/rest/stone/data","dryRun":false,"write":true
20|"bandwidth":null,"delete":true,"destinationPath":"/ab/test/rock/archive","CopyRun":false

Expected Output:

Code:
1|"destinationPath":"/cd/rest/stone/data"
20|"destinationPath":"/ab/test/rock/archive"

Thanks in advance..!
Moderator's Comments:
Mod Comment Please use CODE tags when displaying sample input, output, and code segments.

Last edited by Don Cragun; 05-18-2018 at 06:58 PM.. Reason: Add CODE tags.
# 2  
Old 05-18-2018
Is this a homework assignment? Homework and coursework questions can only be posted in the Homework & Coursework forum under special homework rules.

Please review the rules, which you agreed to when you registered, if you have not already done so.

If you did not post homework, please explain the the source of the data you are processing and the nature of the problem you are working on. Please also tell us what operating system and shell you're using and show us what you have tried to solve this problem on your own.

If you did post homework in the main forums, please review the guidelines for posting homework and repost.
# 3  
Old 05-19-2018
This is not the homework/Assignment. This is the backend data from the cloudera manager scheduler(BDR). The first field is the BDR id and the record consists of several other setting but we are concentrating on the BDR id and the destination path. We also changed the paths in compliance with security.
# 4  
Old 05-19-2018
Welcome to the forum.


How far would this get you?
Code:
awk -F"[|,]" '{for (i=2; i<=NF; i++) if ($i ~ /destinationPath/)  print $1, $i}' OFS="|" file
1|"destinationPath":"/cd/rest/stone/data"
20|"destinationPath":"/ab/test/rock/archive"

# 5  
Old 05-19-2018
Thanks Rudic this really serves my purpose. I have just one more question. If I want to grep for multiple keywords in each line . Lets say in this case we picked the field "destination" Along with this If we want to collect the field "delete".

Expected output for same input:

Code:
1|"destinationPath":"/cd/rest/stone/data","delete":true
 20|"destinationPath":"/ab/test/rock/archive","delete":true



Moderator's Comments:
Mod Comment Seriously: Please use CODE tags as required by forum rules!

Last edited by RudiC; 05-19-2018 at 04:34 PM.. Reason: Added CODE tags.
# 6  
Old 05-19-2018
Try
Code:
awk -F"[|,]" '
NR == 1         {for (n=split (TGTSTR, T); n; n--) TGT[T[n]]
                }
                {for (i=2; i<=NF; i++) for (t in TGT) if ($i ~ t) TMP = TMP (TMP?",":"") $i
                 print $1, substr (TMP, 2)
                 TMP = ""
                }
' OFS="|" TGTSTR="destinationPath,delete" file
1|delete":true,"destinationPath":"/cd/rest/stone/data"
20|delete":true,"destinationPath":"/ab/test/rock/archive"

This User Gave Thanks to RudiC For This Post:
# 7  
Old 05-19-2018
To get back the leading double-quote character on the field that starts with "delete": in the output, I think you want to change the line:
Code:
                 print $1, substr (TMP, 2)

in RudiC's suggested code to:
Code:
                 print $1, TMP

or change the line:
Code:
                {for (i=2; i<=NF; i++) for (t in TGT) if ($i ~ t) TMP = TMP (TMP?",":"") $i

to:
Code:
                {for (i=2; i<=NF; i++) for (t in TGT) if ($i ~ t) TMP = TMP "," $i

This User Gave Thanks to Don Cragun For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extract portion of data

Hi Gurus, I need some help in extracting some of these information and massage it into the desired output as shown below. I need to extract the last row with the header in below sample which is usually the most recent date, for example: 2012-06-01 142356 mb 519 -219406 mb 1 ... (9 Replies)
Discussion started by: superHonda123
9 Replies

2. Shell Programming and Scripting

How to extract portion of a string?

Hi Gurus, Would like to seek some help on how to extract a portion of string from log's output as shown below. Sample of raw data: piece handle=/test123/disk_dump/test123/df0_cntrl_PCPFCI20120404_68498 tag=TAG20120404T180035 comment=NONE piece... (13 Replies)
Discussion started by: superHonda123
13 Replies

3. Shell Programming and Scripting

Extract a string from a line

Hiee all Can anyone tell me how to extract a string from a given line. STAPISDK_RELEASE_32_BL012_2011_JAN_25.1597 I want to extract BL012 from above. as 102 keeps on changing i want smthing like that it extract BL and 102 extrct by its own. Thankx guyzz (9 Replies)
Discussion started by: abhijtr
9 Replies

4. Shell Programming and Scripting

How to extract a text portion from a file

Can some one help me with shell script to extract a text block between two known strings. The given input file is as below: Name: abs Some tesxt.... Some tesxt.... Some tesxt.... end of text Name: xyz Some tesxt.... Some tesxt.... Some tesxt.... end of text Name: efg Some... (5 Replies)
Discussion started by: ejazs0
5 Replies

5. Shell Programming and Scripting

extract string portion from filename using sed

Hi All, I posted something similar before but I now have a another problem. I have filenames as below TOP_TABIN240_20090323.200903231830 TOP_TABIN235_1_20090323.200903231830 i need to extract the dates as in bold. Using bash v 3.xx Im trying to using the print sed command but... (11 Replies)
Discussion started by: santam
11 Replies

6. Shell Programming and Scripting

extract string portion using sed

Hi All I have 3 files as listed below and highlighted in bold the portions of the filenames I need to extract: TOS_TABIN218_20090323.200903231830 TOS_TABIN219_1_20090323.200903231830 TOS_TABIN219_2_20090323.200903231830 I tried source_tabin_name=`echo $fname | sed 's/_.*//'` but I... (6 Replies)
Discussion started by: santam
6 Replies

7. Shell Programming and Scripting

how to extract a string value from the given line

Hi I have an input as follows: param1:value1|param2:value2|param3:value3|param4:value4|param5:value5 where, "|" and ":" are delimiters Now suppose, I want to extract the value corresponding to "param4", i.e. "value4" in this case. In case we use awk, I want to use the value in shell... (1 Reply)
Discussion started by: gaurav_1711
1 Replies

8. Shell Programming and Scripting

extract date portion from file

Hi, I have a file where there is a date field (single line variable length file) how to extract just the date portion from it the position of date field may vary anywhere in the line but will always have the format mm-dd-yyyy for eg . xxxxxxxxxxxxxxx09-10-2006xxxxxxxxxxxxxxxxxxxx (5 Replies)
Discussion started by: misenkiser
5 Replies

9. Shell Programming and Scripting

extract a portion of log file

hello, I want to grep the log file according to time and get the portion of log from one particular time to other. I can grep for individual lines by time but how should I print lines continuously from given start time till end till given end time. Appreciate your ideas, Thanks chandra (8 Replies)
Discussion started by: chandra004
8 Replies

10. UNIX for Dummies Questions & Answers

How to extract a portion of a string from the whole string

How to extract a portion of a string from a full string using unix. For example: Say source string is = "req92374923.log" I want only the numeric portion of the string say "92374923" how to do that in Unix. (2 Replies)
Discussion started by: ds_sastry
2 Replies
Login or Register to Ask a Question