Search for a text between two strings in a file using regex


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Search for a text between two strings in a file using regex
# 1  
Old 01-20-2020
Search for a text between two strings in a file using regex

Here is my sample file data:

Quote:
##
## SSL Virtual Host Context

SSLInsecureRenegotiation Off

Seconds before invalid OCSP responses are expired from the cache

<VirtualHost _default_:443>
SSLInsecureRenegotiation Off
RewriteOptions Inherit
SSLInsecureRenegotiation Off
RewriteOptions Inherit

# General setup for the virtual host
DocumentRoot "/web/htdocs"
ServerName www.example.com:443
ServerAdmin you@example.com
ErrorLog "|/web/bin/rotatelogs /web/logs/sslerror_log.%Y-%m-%d 86400 combined"
TransferLog "/web/logs/access_log"

# SSL Engine Switch:
SSLEngine on

SSLCertificateFile "/web/conf/server.crt"

# ECC keys, when in use, can also be configured in parallel
<FilesMatch "\.(cgi|shtml|phtml|php)$">
SSLOptions +StdEnvVars
</FilesMatch>
<Directory "/web/cgi-bin">
AllowOverrideList None

# SSLInsecureRenegotiation Off

</Directory>

CustomLog "/web/logs/ssl_request_log" \
"%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"

SSLInsecureRenegotiation On
# BEGIN ANSIBLE MANAGED BLOCK
<FilesMatch "^.*\.(css|html?|js|pdf|txt|xml|xsl|gif|ico|jpe?g|png)$">
Require all granted
</FilesMatch>
# END ANSIBLE MANAGED BLOCK
</VirtualHost>
My requirement is to have a regex expression that is able to search for visible starting string "SSLInsecureRenegotiation Off" between strings "<VirtualHost " and "</VirtualHost>".

In the sample data two lines should be matched.

Below is what I tried but it does not match the two lines within the Virtual Tags.

Code:
(?<=<VirtualHost ).*^[^#]*SSLInsecureRenegotiation.*Off.*(?=</VirtualHost>)

Can you please suggest how ?
# 2  
Old 01-20-2020
This would be easy if you did not try to make these text processing tasks "one liners" and just write
the (few lines) code to process the file one line at a time, using any programming language you like.

Basically, if you just processed this text in a loop, reading each line at a time, matching flags and setting patterns, you could have easily processed this file. (Or read the file into an array of lines of text.)

The issue, as I see it, is you (not only you, but many) are falling into the "trap" of looking for "one liners" instead of just writing a small program of a few lines which does the trick.

You are not the only person who falls in to the trap of thinking that everything has to be a "one liner" but this will cause you to waste time when you could write a few lines of code in any programming language and most shell scripts to:
  • Read the file into an array of lines.
  • Process each line and search for your beginning <VirtualHost tag and set a flag.
  • When the flag is set, search and match the other string(s) (SSLInsecureRenegotiation ... blah blah) and put the match(es) in an array.
  • Stop processing after the end tag </VirtualHost is matched.

This is only a few lines of code and is very easy for you (or anyone with minimal programming skills) to write and you could have easily written this code in the time it takes to search for a "one liners" to do the job.

I'm not trying to give you a hard time and I like your posts; but I'm just saying. For a guy with nearly 1000 posts here; you should just write a handful of lines of code and process this versus wasting your time searching for the perfect "one-liner" REGEX.

Cheers.
# 3  
Old 01-20-2020
I can't see what program your regex was written for. Pls learn to mention ALL relevant details in your spec. For sed, try
Code:
sed -n '/<VirtualHost/,/<\/VirtualHost>/ {/^[^#]*SSLInsecureRenegotiation.*Off/p}' file
SSLInsecureRenegotiation Off
SSLInsecureRenegotiation Off

# 4  
Old 01-20-2020
Quote:
Originally Posted by RudiC
I can't see what program your regex was written for. Pls learn to mention ALL relevant details in your spec. For sed, try
Code:
sed -n '/<VirtualHost/,/<\/VirtualHost>/ {/^[^#]*SSLInsecureRenegotiation.*Off/p}' file
SSLInsecureRenegotiation Off
SSLInsecureRenegotiation Off

I need regex solution for grep and not sed as i m using a tool called Ansible which has a module called as "lineinfile" module which i guess uses underlying python to interpret the regex.

I use regex101.com to test the sample data before introducing it to my Ansible code.

Last edited by mohtashims; 01-20-2020 at 01:04 PM..
# 5  
Old 01-21-2020
Ansible could execute any shell command you like. You do not have to use lineinfile.

See: shell - Execute shell commands on targets - Ansible Documentation
# 6  
Old 01-21-2020
Quote:
Originally Posted by stomp
Ansible could execute any shell command you like. You do not have to use lineinfile.

See: shell - Execute shell commands on targets - Ansible Documentation
Exactly, but frankly was too busy to keep going back and forth with a system admin who only sees things his way, right or wrong Smilie making incorrect arguments to justify what he wants to do, versus all the possibilities which can be done, quicker and easier! So, I was trying to avoid days and days of questions and posting back and forth, making a simple task which could be coded in few minutes, more complex because of artificial constraints; instead of writing a few simple line of "easy code" and be done with the task in 10 minutes Smilie

In the examples in the link stomp kindly shared, the script can be "anything" including calling python (for example) if someone liked python Smilie ... or anything under the sun, just about.

Code:
name: Execute the command in remote shell; stdout goes to the specified file on the remote.
  shell: somescript.sh >> somelog.txt

- name: Change the working directory to somedir/ before executing the command.
  shell: somescript.sh >> somelog.txt
  args:
    chdir: somedir/

# You can also use the 'args' form to provide the options.
- name: This command will change the working directory to somedir/ and will only run when somedir/somelog.txt doesn't exist.
  shell: somescript.sh >> somelog.txt
  args:
    chdir: somedir/
    creates: somelog.txt

# You can also use the 'cmd' parameter instead of free form format.
- name: This command will change the working directory to somedir/.
  shell:
    cmd: ls -l | grep log
    chdir: somedir/

- name: Run a command that uses non-posix shell-isms (in this example /bin/sh doesn't handle redirection and wildcards together but bash does)
  shell: cat < /tmp/*txt
  args:
    executable: /bin/bash

- name: Run a command using a templated variable (always use quote filter to avoid injection)
  shell: cat {{ myfile|quote }}

# You can use shell to run other executables to perform actions inline
- name: Run expect to wait for a successful PXE boot via out-of-band CIMC
  shell: |
    set timeout 300
    spawn ssh admin@{{ cimc_host }}

    expect "password:"
    send "{{ cimc_password }}\n"

    expect "\n{{ cimc_name }}"
    send "connect host\n"

    expect "pxeboot.n12"
    send "\n"

    exit 0
  args:
    executable: /usr/bin/expect
  delegate_to: localhost

# Disabling warnings
- name: Using curl to connect to a host via SOCKS proxy (unsupported in uri). Ordinarily this would throw a warning.
  shell: curl --socks5 localhost:9000 http://www.ansible.com
  args:
    warn: no

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need grep regex to extract multiline text between two strings

I have a file conatining the below: --- 10.9.16.116: /tmp/5835113081224811756.jar: hash: e6df90d38fa86f0e289f73d79cd2cfd2a29954eb /tmp/4603745991442278706.jar: hash: e6df90d38fa86f0e289f73d79cd2cfd2a29954eb 10.9.14.126: /tmp/conf/extra/httpd-ssl.conf: hash:... (1 Reply)
Discussion started by: mohtashims
1 Replies

2. UNIX for Beginners Questions & Answers

Using strings in one file as regex to search field of another file

I have a data file, file1.txt, such as the following: 1,2 "TWRTW", "TWRH/" 1,2 "JHGH", "HGJ 1,2 "JWMM", "JWMM" 1,2 "W", "W" 1,2 "LJLH", "LJLH/" 1,3 "W", "W" 1,3 "HJH", "HJJ I have another file, file2.txt, that contains... (3 Replies)
Discussion started by: jvoot
3 Replies

3. Shell Programming and Scripting

Read in search strings from text file, search for string in second text file and output to CSV

Hi guys, I have a text file named file1.txt that is formatted like this: 001 , ID , 20000 002 , Name , Brandon 003 , Phone_Number , 616-234-1999 004 , SSNumber , 234-23-234 005 , Model , Toyota 007 , Engine ,V8 008 , GPS , OFF and I have file2.txt formatted like this: ... (2 Replies)
Discussion started by: An0mander
2 Replies

4. Shell Programming and Scripting

Search strings from array in second file

I have a file search_strings.txt filled with search strings which have a blank in between and look like this: S. g. Erh. o. J. v. d. Chijs g. Ehr.I would like to search the strings in the second given Textfile.txt and it shall return the column number. Can anybody help with the correct... (3 Replies)
Discussion started by: sdf
3 Replies

5. Shell Programming and Scripting

Search replace strings between single quotes in a text file

Hi There... I need to serach and replace a strings in a text file. My file has; books.amazon='Let me read' and the output needed is books.amazon=NONFOUND pls if anybody know this can be done in script sed or awk.. i have a list of different strings to be repced by NONFOUND.... (7 Replies)
Discussion started by: Hiano
7 Replies

6. Shell Programming and Scripting

Best Alternative to Search Text strings in directory

Hi All, We have a file "Customers.lst". It contains list of all the Customers. There is directory which has number of text files and each file containing name of defaulter customers. We want to search for all the customers available in "Customers.lst" file against the list of files... (8 Replies)
Discussion started by: arunorcl
8 Replies

7. Shell Programming and Scripting

Search multiple Strings in a File

Hi I want to search multiple strings in a file . But the search should start with "From" Keyword and end with before "Where" keyword. Please suggest me. Thanks (2 Replies)
Discussion started by: sboss
2 Replies

8. Shell Programming and Scripting

Search complicated strings on file

Can someone help me? I been figuring out how I can search and extract a complicated search string from a file. The whole string is delimited by a period. And the file where I'm searching is composed of differnt string such as that. For example, I have this search string: and I have a file... (3 Replies)
Discussion started by: Orbix
3 Replies

9. Shell Programming and Scripting

search file between last occurence of 2 strings

I need to extract the last block of /== START OF SQLPLUS ==/ and /== END OF SQLPLUS ==/. The logifle is written to several times in a day using >> to append. I need a solution using grep/sed. logfile looks like this START OF LOGFILE /== START OF SQLPLUS ==/ ERROR /== END OF SQLPLUS... (5 Replies)
Discussion started by: hanton
5 Replies

10. Shell Programming and Scripting

How to search multiple strings in a file

Hi All, I want to search all the ksh scripts that has following details. 1. Search for "exit 0" 2. Search for "sqlldr" or sqlplus" 3. In the above files i want to search for all the script that has no "case" in it. Please advice. Thanks, Deep (2 Replies)
Discussion started by: deepakpv
2 Replies
Login or Register to Ask a Question