Sponsored Content
Top Forums Shell Programming and Scripting Match paragraph between two patterns, delete the duplicate paragraphs Post 302831055 by sb245 on Wednesday 10th of July 2013 06:28:20 AM
Old 07-10-2013
thanks vidyadhar85,

It removed all the duplicate lines. But the "BEGIN;" and "COMMIT;" were also needed which is needed in the file. The output came in below syntax.

Code:
BEGIN;
replace into domains (name,type) values ('blah blah 1,'MASTER');
COMMIT;
replace into domains (name,type) values ('blah blah 2','MASTER');
replace into domains (name,type) values ('blah blah 3','MASTER');
replace into domains (name,type) values ('blah blah 4','MASTER');

what I need is as below.


Code:
BEGIN;
replace into domains (name,type) values ('blah blah 1,'MASTER');
COMMIT;
BEGIN;
replace into domains (name,type) values ('blah blah 2','MASTER');
COMMIT;
BEGIN;
replace into domains (name,type) values ('blah blah 3','MASTER');
replace into domains (name,type) values ('blah blah 3','MASTER');
replace into domains (name,type) values ('blah blah 3','MASTER');
COMMIT;
BEGIN;
replace into domains (name,type) values ('blah blah 4','MASTER');
replace into domains (name,type) values ('blah blah 4','MASTER');
COMMIT;

Note: In the original file some paragraphs contain single line. Some contain multiple lines. The example I had pasted had multiple lines. Your script works perfectly for the example I had pasted.

TIA
sb
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

removing certain paragraphs for matching patterns

Hi, I have a log file which might have certain paragraphs. Switch not possible Error code 1234 Process number 678 Log not available Error code 567 Process number 874 ..... ...... ...... Now I create an exception file like this. cat text.exp Error code 1234 Process number 874 (7 Replies)
Discussion started by: kaushys
7 Replies

2. Shell Programming and Scripting

print lines which match multiple patterns

Hi, I have a text file as follows: 11:38:11.054 run1_rdseq avg_2-5 999988.0000 1024.0000 11:50:52.053 run3_rdrand 999988.0000 1135.0 128.0417 11:53:18.050 run4_wrrand avg_2-5 999988.0000 8180.5833 11:55:42.051 run4_wrrand avg_2-5 999988.0000 213.8333 11:55:06.053... (2 Replies)
Discussion started by: annazpereira
2 Replies

3. Shell Programming and Scripting

script to match patterns in 2 different files.

I am new to shell scripting and need some help. I googled, but couldn't find a similar scenario. Basically, I need to rename a datafile. This is the scenario - I have a file, readonly.txt that has 2 columns - file# and name. I have another file,missing_files.txt that has id and name. Both the... (3 Replies)
Discussion started by: mathews
3 Replies

4. Shell Programming and Scripting

Find files that do not match specific patterns

Hi all, I have been searching online to find the answer for getting a list of files that do not match certain criteria but have been unsuccessful. I have a directory that has many jpg files. What I need to do is get a list of the files that do not match both of the following patterns (I have... (21 Replies)
Discussion started by: nikos-koutax
21 Replies

5. Shell Programming and Scripting

Using AWK to match CSV files with duplicate patterns

Dear awk users, I am trying to use awk to match records across two moderately large CSV files. File1 is a pattern file with 173,200 lines, many of which are repeated. The order in which these lines are displayed is important, and I would like to preserve it. File2 is a data file with 456,000... (3 Replies)
Discussion started by: isuewing
3 Replies

6. Shell Programming and Scripting

Match 2 different patterns and print the lines

Hi, i have been trying to extract multiple lines based on two different patterns as below:- file1 @jkm|kdo|aas012|192.2.3.1 blablbalablablkabblablabla sjfdsakfjladfjefhaghfagfkafagkjsghfalhfk fhajkhfadjkhfalhflaffajkgfajkghfajkhgfkf jahfjkhflkhalfdhfwearhahfl @jkm|sdf|wud08q|168.2.1.3... (8 Replies)
Discussion started by: redse171
8 Replies

7. UNIX for Dummies Questions & Answers

Match patterns from another file and tag

Hi all, I have a file , which has 6 tab delimited fields, with $3 and $4 subfielded with spaces. I wamt to match cols $2,$3,$4 of tmp1 with tmp2, ..and then flag the 5th col if found. tmp1 1756 Xerm XermA XermB XermC XermD AA TT AA GG A 1 1763 Xerm XermA XermB XermC... (3 Replies)
Discussion started by: senhia83
3 Replies

8. Shell Programming and Scripting

Match 2 patterns together

How can I quickly print out lines in a datafile which has presence of both patterns in a row of another file. Maybe awk can do it much faster than bash. Patternfile ID1 PAT11 PAT12 ID1 PAT21 PAT22 ID2 PAT31 PAT32 datafile headerline... (2 Replies)
Discussion started by: abh.kumar
2 Replies

9. Shell Programming and Scripting

Find duplicate values in specific column and delete all the duplicate values

Dear folks I have a map file of around 54K lines and some of the values in the second column have the same value and I want to find them and delete all of the same values. I looked over duplicate commands but my case is not to keep one of the duplicate values. I want to remove all of the same... (4 Replies)
Discussion started by: sajmar
4 Replies

10. Shell Programming and Scripting

awk to print match or non-match and select fields/patterns for non-matches

In the awk below I am trying to output those lines that Match between file1 and file2, those Missing in file1, and those missing in file2. Using each $1,$2,$4,$5 value as a key to match on, that is if those 4 fields are found in both files the match, but if those 4 fields are not found then missing... (0 Replies)
Discussion started by: cmccabe
0 Replies
Pod::Elemental::Selectors(3pm)				User Contributed Perl Documentation			    Pod::Elemental::Selectors(3pm)

NAME
Pod::Elemental::Selectors - predicates for selecting elements VERSION
version 0.102362 OVERVIEW
Pod::Elemental::Selectors provides a number of routines to check for Pod::Elemental paragraphs with given qualities. SELECTORS
Selectors are predicates: they examine paragraphs and return either true or false. All the selectors have (by default) names like: "s_whatever". They expect zero or more parameters to affect the selection. If these parameters are given, but no paragraph, a callback will be returned that will expect a paragraph. If a paragraph is given, the selector will return immediately. For example, the "s_command" selector expects a parameter that can be the name of the command desired. Both of the following uses are valid: # create and use a callback: my $selector = s_command('head1'); my @headers = grep { $selector->($_) } @paragraphs; # just check a paragraph right now: if ( s_command('head1', $paragraph) ) { ... } The selectors can be imported individually or as the "-all" group, and can be renamed with Sub::Exporter features. (Selectors cannot yet be curried by Sub::Exporter.) s_blank my $callback = s_blank; if( s_blank($para) ) { ... } "s_blank" tests whether a paragraph is a Generic::Blank element. s_flat my $callback = s_flat; if( s_flat($para) ) { ... } "s_flat" tests whether a paragraph does Pod::Elemental::Flat -- in other words, is content-only. s_node my $callback = s_node; if( s_node($para) ) { ... } "s_node" tests whether a paragraph does Pod::Elemental::Node -- in other words, whether it may have children. s_command my $callback = s_command; my $callback = s_command( $command_name); my $callback = s_command(@command_names); if( s_command(undef, $para) ) { ... } if( s_command( $command_name, $para) ) { ... } if( s_command(@command_names, $para) ) { ... } "s_command" tests whether a paragraph does Pod::Elemental::Command. If a command name (or a reference to an array of command names) is given, the tested paragraph's command must match one of the given command names. AUTHOR
Ricardo SIGNES <rjbs@cpan.org> COPYRIGHT AND LICENSE
This software is copyright (c) 2012 by Ricardo SIGNES. This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself. perl v5.14.2 2012-05-05 Pod::Elemental::Selectors(3pm)
All times are GMT -4. The time now is 06:25 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy