Sponsored Content
Top Forums Shell Programming and Scripting Multi-line filtering based on multi-line pattern in a file Post 302641621 by Finja on Wednesday 16th of May 2012 10:12:46 AM
Old 05-16-2012
Multi-line filtering based on multi-line pattern in a file

I have a file with data records separated by multiple equals signs, as below.

==========
RECORD 1
==========
RECORD 2
DATA LINE
==========
RECORD 3
==========
RECORD 4
DATA LINE
==========
RECORD 5
DATA LINE
==========

I need to filter out all data from this file where the record has no data lines. So in the example above, I would need RECORD 1 and RECORD 3 to be filtered out, plus the extra rows of equals signs separating them. So I would like the output to be:

==========
RECORD 2
DATA LINE
==========
RECORD 4
DATA LINE
==========
RECORD 5
DATA LINE
==========

I can get done whan I need on single lines, but I am stuck trying to match the pattern over multiple lines, and then deleting multiple lines.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Multi line document to single lines based on occurance of string

Hi Guys, I am new to awk and sed, i am working multiline document, i want to make make that document into SINGLE lines based on occurace of string "dwh". here's the sample of my problem.. dwh123 2563 4562 4236 1236 78956 12394 4552 dwh192 2656 46536 231326 65652 6565 23262 16625623... (5 Replies)
Discussion started by: victor369
5 Replies

2. Shell Programming and Scripting

Search for a multi-line strings in a file

Hello I need to search for a mult-line strngs(with spaces in between and qoted) in a file1 and replace that text with Fixed string globally in file1. The strng to search for is in file2. The file is big with some 20K records. so speed and effciency is required file1: (where srch & rplc will... (7 Replies)
Discussion started by: Hiano
7 Replies

3. Shell Programming and Scripting

Print between multi line pattern

Hi, I have a file with text like this .SET WIDTH 10000 .SET MAXERROR 1 insert into new_db SELECT * FROM some_db ; +---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+- .SET WIDTH 10000... (3 Replies)
Discussion started by: sol_nov
3 Replies

4. Shell Programming and Scripting

sed multiple multi line blocks of text containing pattern

Hi, I have a log file which has sessionids in it, each block in the log starts with a date entry, a block may be a single line or multiple lines. I need to sed (or awk) out the lines/blocks with that start with a date and include the session id. The files are large at several Gb. My... (3 Replies)
Discussion started by: andyatit
3 Replies

5. Shell Programming and Scripting

Replacing lines matching a multi-line pattern (sed/perl/awk)

Dear Unix Forums, I am hoping you can help me with a pattern matching problem. What am I trying to do? I want to replace multiple lines of a text file (that match a multi-line pattern) with a single line of text. These patterns can span several lines and do not always have the same number of... (10 Replies)
Discussion started by: thefang
10 Replies

6. Shell Programming and Scripting

Multi line extraction based on condition

Hi I have some data in a file as below ****************************** Class 1A Students absent are : 1. ABC 2. CDE 3. CPE ****************************** Class 2A Students absent are : ****************************** Class 3A Students absent are : (6 Replies)
Discussion started by: reldb
6 Replies

7. Shell Programming and Scripting

How to find all the multi line pattern and redirecting it to a file?

I've a file like this {multi line ....... ....... pattern} { some other stuff ......... } {multi line ....... ....... pattern} { some other stuff ......... } and so on (2 Replies)
Discussion started by: aamir_raihan
2 Replies

8. Shell Programming and Scripting

Help with reformat single-line multi-fasta into multi-line multi-fasta

Input File: >Seq1 ASDADAFASFASFADGSDGFSDFSDFSDFSDFSDFSDFSDFSDFSDFSDFSD >Seq2 SDASDAQEQWEQeqAdfaasd >Seq3 ASDSALGHIUDFJANCAGPATHLACJHPAUTYNJKG ...... Desired Output File >Seq1 ASDADAFASF ASFADGSDGF SDFSDFSDFS DFSDFSDFSD FSDFSDFSDF SD >Seq2 (4 Replies)
Discussion started by: patrick87
4 Replies

9. Shell Programming and Scripting

Multi line log files to single line format

I want to read the log file which was generate from other command . And the output was having multi line in log files for job name and server name. But i need to make all the logs on one line Source file 07/15/2018 17:02:00 TRANSLOG_1700 Server0005_SQL ... (2 Replies)
Discussion started by: ranjancom2000
2 Replies

10. UNIX for Beginners Questions & Answers

Search a multi-line shell command output and execute logic based on result

The following is a multi-line shell command example: $cargo build Compiling prawn v0.1.0 (/Users/ag/rust/prawn) error: failed to resolve: could not find `setup_panix` in `human_panic` --> src/main.rs:14:22 | 14 | human_panic::setup_panix!(); | ... (2 Replies)
Discussion started by: yogi
2 Replies
LDIF(5) 							File Formats Manual							   LDIF(5)

NAME
ldif - LDAP Data Interchange Format DESCRIPTION
The LDAP Data Interchange Format (LDIF) is used to represent LDAP entries and change records in text form. LDAP tools, such as ldapadd(1) and ldapsearch(1), read and write LDIF entry records. ldapmodify(1) reads LDIF change records. This manual page provides a basic description of LDIF. A formal specification of LDIF is published in RFC 2849. ENTRY RECORDS
LDIF entry records are used to represent directory entries. The basic form of an entry record is: dn: <distinguished name> <attrdesc>: <attrvalue> <attrdesc>: <attrvalue> <attrdesc>:: <base64-encoded-value> <attrdesc>:< <URL> ... The value may be specified as UTF-8 text or as base64 encoded data, or a URI may be provided to the location of the attribute value. A line may be continued by starting the next line with a single space or tab, e.g., dn: cn=Barbara J Jensen,dc=exam ple,dc=com Lines beginning with a sharp sign ('#') are ignored. Multiple attribute values are specified on separate lines, e.g., cn: Barbara J Jensen cn: Babs Jensen If an value contains a non-printing character, or begins with a space or a colon ':', the <attrtype> is followed by a double colon and the value is encoded in base 64 notation. e.g., the value " begins with a space" would be encoded like this: cn:: IGJlZ2lucyB3aXRoIGEgc3BhY2U= If the attribute value is located in a file, the <attrtype> is followed by a ':<' and a file: URI. e.g., the value contained in the file /tmp/value would be listed like this: cn:< file:///tmp/value Other URI schemes (ftp,http) may be supported as well. Multiple entries within the same LDIF file are separated by blank lines. ENTRY RECORD EXAMPLE
Here is an example of an LDIF file containing three entries. dn: cn=Barbara J Jensen,dc=example,dc=com cn: Barbara J Jensen cn: Babs Jensen objectclass: person description:< file:///tmp/babs sn: Jensen dn: cn=Bjorn J Jensen,dc=example,dc=com cn: Bjorn J Jensen cn: Bjorn Jensen objectclass: person sn: Jensen dn: cn=Jennifer J Jensen,dc=example,dc=com cn: Jennifer J Jensen cn: Jennifer Jensen objectclass: person sn: Jensen jpegPhoto:: /9j/4AAQSkZJRgABAAAAAQABAAD/2wBDABALD A4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQ ERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVG ... Note that the description in Barbara Jensen's entry is read from file:///tmp/babs and the jpegPhoto in Jennifer Jensen's entry is encoded using base 64. CHANGE RECORDS
LDIF change records are used to represent directory change requests. Each change record starts with line indicating the distinguished name of the entry being changed: dn: <distinguishedname> changetype: <[modify|add|delete|modrdn]> Finally, the change information itself is given, the format of which depends on what kind of change was specified above. For a changetype of modify, the format is one or more of the following: add: <attributetype> <attrdesc>: <value1> <attrdesc>: <value2> ... - Or, for a replace modification: replace: <attributetype> <attrdesc>: <value1> <attrdesc>: <value2> ... - If no attributetype lines are given to replace, the entire attribute is to be deleted (if present). Or, for a delete modification: delete: <attributetype> <attrdesc>: <value1> <attrdesc>: <value2> ... - If no attributetype lines are given to delete, the entire attribute is to be deleted. For a changetype of add, the format is: <attrdesc1>: <value1> <attrdesc1>: <value2> ... <attrdescN>: <value1> <attrdescN>: <value2> For a changetype of modrdn or moddn, the format is: newrdn: <newrdn> deleteoldrdn: 0 | 1 newsuperior: <DN> where a value of 1 for deleteoldrdn means to delete the values forming the old rdn from the entry, and a value of 0 means to leave the val- ues as non-distinguished attributes in the entry. The newsuperior line is optional and, if present, specifies the new superior to move the entry to. For a changetype of delete, no additional information is needed in the record. Note that attribute values may be presented using base64 or in files as described for entry records. Lines in change records may be con- tinued in the manner described for entry records as well. CHANGE RECORD EXAMPLE
The following sample LDIF file contains a change record of each type of change. dn: cn=Babs Jensen,dc=example,dc=com changetype: add objectclass: person objectclass: extensibleObject cn: babs cn: babs jensen sn: jensen dn: cn=Babs Jensen,dc=example,dc=com changetype: modify add: givenName givenName: Barbara givenName: babs - replace: description description: the fabulous babs - delete: sn sn: jensen - dn: cn=Babs Jensen,dc=example,dc=com changetype: modrdn newrdn: cn=Barbara J Jensen deleteoldrdn: 0 newsuperior: ou=People,dc=example,dc=com dn: cn=Barbara J Jensen,ou=People,dc=example,dc=com changetype: delete INCLUDE STATEMENT
The LDIF parser has been extended to support an include statement for referencing other LDIF files. The include statement must be sepa- rated from other records by a blank line. The referenced file is specified using a file: URI and all of its contents are incorporated as if they were part of the original LDIF file. As above, other URI schemes may be supported. For example: dn: dc=example,dc=com objectclass: domain dc: example include: file:///tmp/example.com.ldif dn: dc=example,dc=org objectclass: domain dc: example This feature is not part of the LDIF specification in RFC 2849 but is expected to appear in a future revision of this spec. It is supported by the ldapadd(1), ldapmodify(1), and slapadd(8) commands. SEE ALSO
ldap(3), ldapsearch(1), ldapadd(1), ldapmodify(1), slapadd(8), slapcat(8), slapd-ldif(5). "LDAP Data Interchange Format," Good, G., RFC 2849. ACKNOWLEDGEMENTS
OpenLDAP Software is developed and maintained by The OpenLDAP Project <http://www.openldap.org/>. OpenLDAP Software is derived from Uni- versity of Michigan LDAP 3.3 Release. OpenLDAP 2.4.39 2014/01/26 LDIF(5)
All times are GMT -4. The time now is 12:35 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy