Sponsored Content
Top Forums UNIX for Advanced & Expert Users Remove lines if the first character is "|" Post 302588118 by pkiula on Friday 6th of January 2012 07:46:55 PM
Old 01-06-2012
Remove lines if the first character is "|"

Hi.

I have a huge file (350 million lines).
I need to delete all lines in it that:

1. Begin with a pipe character -- '|'
2. Or have less than 5 pipe characters in the line

Have been searching for some SED/AWK help for this (which is faster btw on such a ginormous file?) but only found this, but this works with a number not a pipe character.

PHP Code:
On this website
 /
unix-advanced-expert-users/160025-deleting-lines-if-ith-character-x.html 

Much appreciate any pointers.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Command Character size limit in the "sh" and "bourne" shell

Hi!!.. I would like to know what is maximum character size for a command in the "sh" or "bourne" shell? Thanks in advance.. Roshan. (1 Reply)
Discussion started by: Roshan1286
1 Replies

2. UNIX for Advanced & Expert Users

Command Character size limit in the "sh" and "bourne" shell

Hi!!.. I would like to know what is maximum character size for a command in the "sh" or "bourne" shell? Thanks in advance.. Roshan. (1 Reply)
Discussion started by: Roshan1286
1 Replies

3. Shell Programming and Scripting

Command Character size limit in the "sh" and "bourne" shell

Hi!!.. I would like to know what is maximum character size for a command in the "sh" or "bourne" shell? Thanks in advance.. Roshan. (1 Reply)
Discussion started by: Roshan1286
1 Replies

4. Shell Programming and Scripting

How to print range of lines using sed when pattern has special character "["

Hi, My input has much more lines, but few of them are below pin(IDF) { direction : input; drc_pinsigtype : signal; pin(SELDIV6) { direction : input; drc_pinsigtype : ... (3 Replies)
Discussion started by: nehashine
3 Replies

5. Shell Programming and Scripting

Appending character "0" in vi editor for all lines between 1 and 40.

I have to append character "0" for lines between 1 and 40 in a file. I tried the following code. :s/^0,1,40/g Input: Output: (2 Replies)
Discussion started by: pinnacle
2 Replies

6. Shell Programming and Scripting

Remove ":" and join lines in outline file

I have a vim outliner file like this: Title title 2 :Testing now :testing 2 :testing 3 title 3 :testing :ttt :ttg Is there a way to use a script or command to remove... (7 Replies)
Discussion started by: jostber
7 Replies

7. Shell Programming and Scripting

how to use "cut" or "awk" or "sed" to remove a string

logs: "/home/abc/public_html/index.php" "/home/abc/public_html/index.php" "/home/xyz/public_html/index.php" "/home/xyz/public_html/index.php" "/home/xyz/public_html/index.php" how to use "cut" or "awk" or "sed" to get the following result: abc abc xyz xyz xyz (8 Replies)
Discussion started by: timmywong
8 Replies

8. UNIX for Dummies Questions & Answers

Grep : Filter/Move All The Lines Containing Not More Than One "X" Character Into A Text File

Hi All It's me again with another huge txt files. :confused: What I have: - I have 33 huge txt files in a folder. - I have thousands of line in this txt file which contain many the letter "x" in them. - Some of them have more than one "x" character in the line. What I want to achieve:... (8 Replies)
Discussion started by: Nexeu
8 Replies

9. Shell Programming and Scripting

Remove word after special character "/"

Hi There, I have one requirement to remove word after character "/". Input file is 2017-07-12|02|user1l|domain1/userl|0 2017-07-12|02|user2|domain1/user2|5 2017-07-12|02|user3|domain2/user3|0 2017-07-12|02|user4|domain1/user4|432 and require OP file is ... (2 Replies)
Discussion started by: anshu ranjan
2 Replies

10. UNIX for Beginners Questions & Answers

Remove a word that ends with special character "!"

Hi all. I want to use sed to remove a word that ends with "!" in the first page of a file. The word I want to remove is: "DNA!". I have search for an answer and nothing of what I found helped me. ~faizlo (2 Replies)
Discussion started by: faizlo
2 Replies
PIPE(2) 							System Calls Manual							   PIPE(2)

NAME
pipe - create an interprocess channel SYNOPSIS
#include <u.h> #include <libc.h> int pipe(int fd[2]) DESCRIPTION
Pipe creates a buffered channel for interprocess I/O communication. Two file descriptors are returned in fd. Data written to fd[1] is available for reading from fd[0] and data written to fd[0] is available for reading from fd[1]. After the pipe has been established, cooperating processes created by subsequent fork(2) calls may pass data through the pipe with read and write calls. The bytes placed on a pipe by one write are contiguous even if many processes are writing. Write boundaries are preserved: each read terminates when the read buffer is full or after reading the last byte of a write, whichever comes first. The number of bytes available to a read(2) is reported in the Length field returned by fstat or dirfstat on a pipe (see stat(2)). When all the data has been read from a pipe and the writer has closed the pipe or exited, read(2) will return 0 bytes. Writes to a pipe with no reader will generate a note sys: write on closed pipe. SOURCE
/sys/src/libc/9syscall SEE ALSO
intro(2), read(2), pipe(3) DIAGNOSTICS
Sets errstr. BUGS
If a read or a write of a pipe is interrupted, some unknown number of bytes may have been transferred. When a read from a pipe returns 0 bytes, it usually means end of file but is indistinguishable from reading the result of an explicit write of zero bytes. PIPE(2)
All times are GMT -4. The time now is 11:00 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy