Sponsored Content
Top Forums UNIX for Beginners Questions & Answers awk to remove pattern and lines above pattern Post 303033192 by cmccabe on Monday 1st of April 2019 09:34:47 AM
Old 04-01-2019
awk to remove pattern and lines above pattern

In the awk below I am trying to remove all lines above and including the pattern Test or Test2. Each block is seperated by a newline and Test2 also appears in the lines to keep but it will always have additional text after it. The Test to remove will not. The awk executed until the || was added but only removed the first instance of Test, that is the first 3 lines including the line with Test in it. Thank you Smilie.

file
Code:
xxx-xxx 00-0002_Last,First-Id
xxx-xxx 00-0003_Las,Fir-Id
xxx-xxx 00-0004_La,Fi-Id
R_2019_03_00_00_00_00_xxxx_zz-000-000-Test

xxx-xxx 00-0006-yyyy
xxx-xxx 190326-yyyy
R_2019_03_00_00_00_00_xxxx_zz-000-zzz-v5.6_Test2_Additional_xxxx_xxx_xxx_xx_xxx

xxx-xxx Negative_Test
xxx-xxx Positive_Test
R_2019_03_00_00_00_00_xxxx_zz-000-000-v5.6_Test2

xxx-xxx 00-0000_L,F
xxx-xxx 00-0001_L2,F2
R_2019_03_00_00_00_00_xxxx_zz-000-000-Test

xxx-xxx xx-xxxx_Test,Test1
R_2019_03_00_00_00_00_xxxx_zz-000-000_Test2

xxx-xxx 00-0005-yyyy
R_2019_03_00_00_00_00_xxxx_zz-000-zzz-v5.6_Test2_Additional_xxxx_xxx_xxx_xx_xxx

desired
Code:
xxx-xxx 00-0006-yyyy
xxx-xxx 190326-yyyy
R_2019_03_00_00_00_00_xxxx_zz-000-zzz-v5.6_Additional_xxxx_xxx_xxx_xx_xxx

xxx-xxx 00-0005-yyyy
R_2019_03_00_00_00_00_xxxx_zz-000-zzz-v5.6_Additional_xxxx_xxx_xxx_xx_xxx

awk
Code:
awk -F '\n' '/Test || Test2/$ && getline, 0' file


Last edited by cmccabe; 04-01-2019 at 10:39 AM.. Reason: fixed format
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove lines from batch of files which match pattern

I need to remove all lines which does not match the pattern from a text file (batch of text files). I also need to keep the header line which is the first line of the file. Please can you provide an example for that. I used this to solve half of my work. I was unable to keep the first line of... (3 Replies)
Discussion started by: talktobog
3 Replies

2. Shell Programming and Scripting

NAWK to remove lines that matches a specific pattern

Hi, I have requirement that I need to split my input file into two files based on a search pattern "abc" For eg. my input file has below content abc defgh zyx I need file 1 with abc and file2 with defgh zyx I can use grep command to acheive this. But with grep I need... (8 Replies)
Discussion started by: sbhuvana20
8 Replies

3. Shell Programming and Scripting

Searching for pattern and remove the lines

Hi , I want to remove the specific pattern and remove those lines from file using shell script. i want to remove these lines <?xml version='1.0' encoding='UTF-8'?> <row_set> </row_set> my input file has content like this. file name: sample.xml <?xml version='1.0'... (4 Replies)
Discussion started by: nukala_2
4 Replies

4. Shell Programming and Scripting

Remove multiple lines that match pattern

Not sure how I can accomplish this. I would like to remove all interfaces that have the commands I would like to see: switchport port-security, spanning-tree portfast. One line is no problem. interface FastEthernet0/8 spanning-tree portfast interface FastEthernet0/9 spanning-tree... (4 Replies)
Discussion started by: mrlayance
4 Replies

5. Shell Programming and Scripting

sed -- Find pattern -- print remainder -- plus lines up to pattern -- Minus pattern

The intended result should be : PDF converters 'empty line' gpdftext and pdftotext?xml version="1.0"?> xml:space="preserve"><note-content version="0.1" xmlns:/tomboy/link" xmlns:size="http://beatniksoftware.com/tomboy/size">PDF converters gpdftext and pdftotext</note-content>... (9 Replies)
Discussion started by: Klasform
9 Replies

6. Shell Programming and Scripting

awk remove/grab lines from file with pattern from other file

Sorry for the weird title but i have the following problem. We have several files which have between 10000 and about 500000 lines in them. From these files we want to remove lines which contain a pattern which is located in another file (around 20000 lines, all EAN codes). We also want to get... (28 Replies)
Discussion started by: SDohmen
28 Replies

7. Shell Programming and Scripting

awk to combine lines from line with pattern match to a line that ends in a pattern

I am trying to combine lines with these conditions: 1. First line starts with text of "libname VALUE db2 datasrc" where VALUE can be any text. 2. If condition1 is met then continue to combine lines through a line that ends with a semicolon. 3. Ignore case when matching patterns and remove any... (5 Replies)
Discussion started by: Wes Kem
5 Replies

8. Shell Programming and Scripting

How to remove the lines with this pattern?

Hello everyone, I have a sample data like this: Glyma.10G051100 Glyma.02G036000 89.91 228 23 0 1 228 1 228 1e-78 294 Glyma.10G051100 Glyma.09G023700 87.28 228 29 0 1 228 1 228 1e-68 261 Glyma.10G285200 Glyma.20G103800 96.33 1663 55 4 1 1657 1 1663 0.0 2728 Glyma.10G285200 Glyma.05G093700 95.02... (2 Replies)
Discussion started by: nengcheng
2 Replies

9. Shell Programming and Scripting

sed to remove next 15 lines after pattern in Solaris

Team, I am trying to use sed to delete 15 lines, after pattern patch, which includes the pattern as well in Solaris. I used the below command, as we do it Linux, but it's not working as expected in Solaris. I am getting the error as "garbled".sed '/\/table/,+15d' status.html sed: command... (8 Replies)
Discussion started by: Nagaraj R
8 Replies

10. UNIX for Beginners Questions & Answers

awk with sed to combine lines and remove specific odd # pattern from line

In the awk piped to sed below I am trying to format file by removing the odd xxxx_digits and whitespace after, then move the even xxxx_digit to the line above it and add a space between them. There may be multiple lines in file but they are in the same format. The Filename_ID line is the last line... (4 Replies)
Discussion started by: cmccabe
4 Replies
pure-authd(8)							     Pure-FTPd							     pure-authd(8)

NAME
pure-authd - External authentication agent for Pure-FTPd. SYNTAX
pure-authd [-p </path/to/pidfile>] [-u uid] [-g gid] [-B] <-s /path/to/socket> -r /program/to/run DESCRIPTION
pure-authd is a daemon that forks an authentication program, waits for an authentication reply, and feed them to an application server. pure-authd listens to a local Unix socket. A new connection to that socket should feed pure-authd the following structure : account:xxx password:xxx localhost:xxx localport:xxx peer:xxx end (replace xxx with appropriate values) . localhost, localport and peer are numeric IP addresses and ports. peer is the IP address of the remote client. These arguments are passed to the authentication program, as environment variables : AUTHD_ACCOUNT AUTHD_PASSWORD AUTHD_LOCAL_IP AUTHD_LOCAL_PORT AUTHD_REMOTE_IP AUTHD_ENCRYPTED The authentication program should take appropriate actions to fetch account info according to these arguments, and reply to the standard output a structure like the following one : auth_ok:1 uid:42 gid:21 dir:/home/j end auth_ok:xxx If xxx is 0, the user was not found (the next authentication method passed to pure-ftpd will be tried) . If xxx is -1, the user was found, but there was a fatal authentication error : user is root, password is wrong, account has expired, etc (next authentication methods will not be tried) . If xxx is 1, the user was found and successfully authenticated. uid:xxx The system uid to be assigned to that user. Must be > 0. gid:xxx The primary system gid. Must be > 0. dir:xxx The absolute path to the home directory. Can contain /./ for a chroot jail. slow_tilde_expansion:xxx (optional, default is 1) When the command 'cd ~user' is issued, it's handy to go to that user's home directory, as expected in a shell environment. But fetching account info can be an expensive operation for non-system accounts. If xxx is 0, 'cd ~user' will expand to the system user home directory. If xxx is 1, 'cd ~user' won't expand. You should use 1 in most cases with external authentication, when your FTP users don't match system users. You can also set xxx to 1 if you're using slow nss_* system authentication modules. throttling_bandwidth_ul:xxx (optional) The allocated bandwidth for uploads, in bytes per second. throttling_bandwidth_dl:xxx (optional) The allocated bandwidth for downloads, in bytes per second. user_quota_size:xxx (optional) The maximal total size for this account, in bytes. user_quota_files:xxx (optional) The maximal number of files for this account. ratio_upload:xxx (optional) radio_download:xxx (optional) The user must match a ratio_upload:ratio_download ratio. Only one authentication program is forked at a time. It must return quickly. OPTIONS
-u <uid> Have the daemon run with that uid. -g <gid> Have the daemon run with that gid. -B Fork in background (daemonization). -s </path/to/socket> Set the full path to the local Unix socket. -R </path/to/program> Set the full path to the authentication program. -h Output help information and exit. EXAMPLES
To run this program the standard way type: pure-authd -s /var/run/ftpd.sock -r /usr/bin/my-auth-program & pure-ftpd -lextauth:/var/run/ftpd.sock & /usr/bin/my-auth-program can be as simple as : #! /bin/sh echo 'auth_ok:1' echo 'uid:42' echo 'gid:21' echo 'dir:/home/j' echo 'end' AUTHORS
Frank DENIS <j at pureftpd dot org> SEE ALSO
ftp(1), pure-ftpd(8) pure-ftpwho(8) pure-mrtginfo(8) pure-uploadscript(8) pure-statsdecode(8) pure-pw(8) pure-quotacheck(8) pure-authd(8) RFC 959, RFC 2389, RFC 2228 and RFC 2428. Pure-FTPd team 1.0.36 pure-authd(8)
All times are GMT -4. The time now is 11:09 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy