Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

How to remove everything after a word containing string?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to remove everything after a word containing string?
# 8  
Old 01-25-2019
@nezabudka: The shell cycle is used to create the respective output files. If you do it all in awk, redirect output immediately within it. Like
Code:
awk '
FNR == NR       {T[$1]
                 next
                }
FNR == 1        {if (FN) close (FN)
                 FN = "report_" FILENAME
                }
                {for (t in T) sub (t".*", t)
                 print > FN
                }
'   readfile source*

These 2 Users Gave Thanks to RudiC For This Post:
000vikas nezabudka
# 9  
Old 01-26-2019
Quote:
Originally Posted by nezabudka
Boris
Why do you delete the file extension and immediately add it?
Code:
> report_$(basename "${file/.txt}").txt

And why do you run it in a cycle, only you lose time. Try it first
Code:
awk 'NR==FNR {T[$0]; next} {for(i=1;i<=NF;i++){for(t in T) if ($i ~ t) $0=$i}} 1' readfile source*.txt

To localize the error, try adding a filter, for example
Code:
awk 'NR==FNR {T[$0]; next} NF > 1 && NF < 4 {for(i=1;i<=NF;i++){for(t in T) if ($i ~ t) $0=$i}} 1' readfile source*.txt

PS: readfile and source are the same as I posted under this thread.

Hello,
I am sorry for the headache.
I posted in next script that I have more files to be processed.
So, I edited the main post.
What I typed in first post gives expected result.
I need time to check what was wrong at my end.

Thank you
Boris

--- Post updated at 05:09 PM ---

Hello Again,
Here is the output:

Code:
root@house:~/test# awk 'NR==FNR {T[]; next} {for (t in T) sub (t.*, t)} 1' readfile source
awk: cmd. line:1: NR==FNR {T[]; next} {for (t in T) sub (t.*, t)} 1
awk: cmd. line:1:            ^ syntax error
awk: cmd. line:1: error: invalid subscript expression
awk: cmd. line:1: NR==FNR {T[]; next} {for (t in T) sub (t.*, t)} 1
awk: cmd. line:1:                                         ^ syntax error
awk: cmd. line:1: NR==FNR {T[]; next} {for (t in T) sub (t.*, t)} 1
awk: cmd. line:1:                                              ^ 1 is invalid as number of arguments for sub

Thank you
Boris

--- Post updated at 11:20 PM ---

Hello,
As I faced problems with awk, sorted out in below algorithm shortly:

-First line removed in sourcefile
-Grep all lines containing COL1 in sourcefile > output1
-Grep all lines not-containing COL1 in sourcefile > output2
-Paste -d '\n' both output files

I am sorry for the suffer I caused.

Thank you
Boris
Last edited by baris35; 01-26-2019 at 12:21 AM.. Reason: extra info
# 10  
Old 01-26-2019
Quote:
Originally Posted by baris35
...

Hello Again,
Here is the output:

Code:
root@house:~/test# awk 'NR==FNR {T[]; next} {for (t in T) sub (t.*, t)} 1' readfile source
awk: cmd. line:1: NR==FNR {T[]; next} {for (t in T) sub (t.*, t)} 1
awk: cmd. line:1:            ^ syntax error
awk: cmd. line:1: error: invalid subscript expression
awk: cmd. line:1: NR==FNR {T[]; next} {for (t in T) sub (t.*, t)} 1
awk: cmd. line:1:                                         ^ syntax error
awk: cmd. line:1: NR==FNR {T[]; next} {for (t in T) sub (t.*, t)} 1
awk: cmd. line:1:                                              ^ 1 is invalid as number of arguments for sub

Thank you
Boris
...
Wouldn't it make serious sense to read and try to understand the error message(s)? And, compare your code to the proposal given in post #6 ... see the difference?
# 11  
Old 01-26-2019
Quote:
Originally Posted by RudiC
Wouldn't it make serious sense to read and try to understand the error message(s)? And, compare your code to the proposal given in post #6 ... see the difference?
No, I do not see difference when I run both seperately.
As I do not understand awk, I put codes given in #6 between echo to see what is printing:
s2.sh
Code:
while read COL1 COL2
do
echo " awk 'NR==FNR {T[$1]; next} {for (t in T) if ($1 ~ t) $0 = $1} 1' readfile source "
done<readfile

output
Code:
awk 'NR==FNR {T[]; next} {for (t in T) if ( ~ t) ./s2.sh = } 1' readfile source
awk 'NR==FNR {T[]; next} {for (t in T) if ( ~ t) ./s2.sh = } 1' readfile source
awk 'NR==FNR {T[]; next} {for (t in T) if ( ~ t) ./s2.sh = } 1' readfile source

When I run without echo:

Code:
#!bin/bash
#test1
http://www.aa.bb.cc
#test2
http://www.11.rr.cd
#test3
http://www.22.qq.fc
#!bin/bash
#test1
http://www.aa.bb.cc
#test2
http://www.11.rr.cd
#test3
http://www.22.qq.fc
#!bin/bash
#test1
http://www.aa.bb.cc
#test2
http://www.11.rr.cd
#test3
http://www.22.qq.fc

When I put the second code in #6,

s2.sh
Code:
while read COL1 COL2
do
echo " awk 'NR==FNR {T[$1]; next} {for (t in T) sub (t".*", t)} 1' readfile source "
done<readfile

output:
Code:
 awk 'NR==FNR {T[]; next} {for (t in T) sub (t.*, t)} 1' readfile source
 awk 'NR==FNR {T[]; next} {for (t in T) sub (t.*, t)} 1' readfile source
 awk 'NR==FNR {T[]; next} {for (t in T) sub (t.*, t)} 1' readfile source

When I run s2.sh without echo:

Code:
#!bin/bash
#test1
http://www.aa.bb
#test2
http://www.11.rr.cd
#test3
http://www.22.qq.fc
#!bin/bash
#test1
http://www.aa.bb
#test2
http://www.11.rr.cd
#test3
http://www.22.qq.fc
#!bin/bash
#test1
http://www.aa.bb
#test2
http://www.11.rr.cd
#test3
http://www.22.qq.fc

Sorted out with grep + sed + paste commands. Awk is more complicated for me.

Thank you
Boris
# 12  
Old 01-26-2019
What in the shown result of s2.sh does not satisfy your needs? Looks perfect to me, considering the code you presented.

Quote:
Originally Posted by baris35
No, I do not see difference when I run both seperately.
The difference is the T array index is $1 in the working code, and empty in your error case. I desparately try to understand why "Awk is more complicated for" you and you forgo the efficient complete solutions presented to you in posts #7 or #8, falling back to highly inefficient band aid pseudo solutions.

Consider the case the bumper has fallen off your car. The professional repair shop grabs their MIG welder, welds the screw nuts back to the carrier beam, and with a ratchet screws the bumper back on. Amateurs use chewing gum to fill the gaps, and scotch tape to glue the bumper back.


awk (or perl, sed, etc.) is the MIG welder and the ratchet at your finger tips.
Last edited by RudiC; 01-26-2019 at 08:06 AM..
# 13  
Old 01-26-2019
So, should I tell awk to search which column to be looked up? Sed+grep are like medium frequency welding technology for me. So far, awk seems not-comprehensible, even after your detailed explanation. I need to read more and more..

Thank you for your time.
Boris

--- Post updated at 07:16 AM ---

Quote:
Originally Posted by RudiC
What in the shown result of s2.sh does not satisfy your needs? Looks perfect to me, considering the code you presented.
Hello Rudic,
It requires:
Code:
perl -i -ne 'print unless ${$_}++' output

Don't worry, the problem solved with a bit longer way

Kind regards
Boris
This User Gave Thanks to baris35 For This Post:
nezabudka
# 14  
Old 01-27-2019
I apologize to the author of the topic for rejection.
I think too beautiful to work correctly.
Quote:
Originally Posted by RudiC
Code:
awk '
FNR == NR       {T[$1]
                 next
                }
FNR == 1        {if (FN) close (FN)
                 FN = "report_" FILENAME
                }
                {for (t in T) sub (t".*", t)
                 print > FN
                }
'   readfile source*

I have simplified:
Code:
awk 'NR==FNR {T[$1]; next} {for (t in T) sub (t".*", t)} 1' readfile source

Although the condition of the task to remove the remaining substring after the match.
But logically, it is still necessary to store the address fully in which the entire match is found.
Example
result: http://www._aa.bb
result as I imagine it: http://www.aa.bb.cc
Code:
awk 'NR==FNR {T[$1]; next} {for (t in T) sub(t, "\r"t)
$0 = gensub(/\r([^ ]*).*/, "\\1", 1)} 1' readfile source

Regards to RudiC, yet the decision is very beautiful and concise.
There was something to learn
This User Gave Thanks to nezabudka For This Post:
Neo
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove string perl with first or last word is in a list

Hello, I try to delete all strings if their first or last word is one of this list of words : "the", "i", "in", "there", "this", "with", "on", "we", "that", "of" For example if i have this string in an input file "with me" this string will be removed, Example: input "the european... (2 Replies)
Discussion started by: cyrine
2 Replies

2. Shell Programming and Scripting

Remove last word of a string?

Hello I have a string that may or may not have 4 dots. The string is actualy a file within a folder, where multiple files are located. Files may look like: # ls * creds: 192.168.10.110 someserver shares: 192.168.10.110.Public someserver.sharefolder # I want to fill 2 variables,... (1 Reply)
Discussion started by: sea
1 Replies

3. Shell Programming and Scripting

How to remove first word?

Hi All, I want to remove the first word "cn=" from the below details. Only I want the number like 171345,174144... cn=171345 cn=174144 How can I achieve this. Please suggest. Thanks- P (6 Replies)
Discussion started by: pokhraj_d
6 Replies

4. Shell Programming and Scripting

Remove 1st word and _ from string

var=abc_cde_def_ghi_678.txt Expected output: cde_def_ghi_678.txt Is there a better way to achive this other than cut command? Basically, I need to remove the 1st word and _ from the string. Thanks. (1 Reply)
Discussion started by: vedanta
1 Replies

5. Shell Programming and Scripting

sed command to remove a word from string

Hello All, I am running a command find . -name amp.cfg | cut -c 3- which gives me output something like below rel/prod/amp.cfg rel/fld/amp.cfg deb/detail/amp.cfg deb/err/amp.cfg I want to remove trailing "/amp.cfg" so that i should get output something like... (7 Replies)
Discussion started by: anand.shah
7 Replies

6. Shell Programming and Scripting

want to remove last word.

Hi, I have a file which has the following /u12/data/oracle/abc.dbf /u12/data/oracle/def.dbf /u12/data/oracle/daf.dbf /u12/data/oracledb/fgh.dbf /u12/data/oracledb/fkh.dbf /u12/data/oracledb/kdq.dbf I want to do something like this /u12/data/oracle /u12/data/oracle... (7 Replies)
Discussion started by: javeedkaleem
7 Replies

7. Shell Programming and Scripting

Replace a word in a string starting with another word

Hi All, I have a file in which a number of lines are starting with similar first word but different next words. I want to replace the any nth word(not 1st or 2nd) with another word. Eg:- My file contains are like this:- Ram is a boy. Ram is a good boy. Ram plays cricket. Here I want to... (2 Replies)
Discussion started by: mukeshbaranwal
2 Replies

8. Shell Programming and Scripting

grep part of word or Another word from a string

Hi all, FileOne family balance >>>>> 0 0 0 0 java.io.FileNotFoundException: Settings.xml (No such file or directory) at java.io.FileInputStream.open(Native Method) .. .... ..... ..... java.lang.NullPointerException ... ..... ...... Stacktrace: at... (2 Replies)
Discussion started by: linuxadmin
2 Replies

9. Shell Programming and Scripting

Remove particular word from file

Hi All, If my file is: Wed Sep 9 22:45:14 EDT 2009 sftp> sftp> sftp> sftp> sftp> sftp> sftp> sftp> sftp> sftp> sftp> sftp> sftp> This is log file generated from transfer... sftp> sftp> sftp> sftp> Files placed properly.... sftp> sftp> sftp> How can I remove "sftp>" word from this... (4 Replies)
Discussion started by: darshakraut
4 Replies

10. Shell Programming and Scripting

how to remove first word

Hi, i have a question about to remove first word from a sentence. my script; #!/usr/bin/perl $msgtxt = "this is a test script"; my @ap_txtMsg = split(/ +/, trim_data($msgtxt)); $ap_msgtxt = splice (@ap_txtMsg, 0, 1); print $ap_msgtxt; but the output is first word that i... (1 Reply)
Discussion started by: malaysoul
1 Replies
Login or Register to Ask a Question