Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Want to remove all lines but not latest 50 lines from a file Post 302868783 by RudiC on Monday 28th of October 2013 06:52:46 PM
Old 10-28-2013
For discussion:
Code:
tac < file | head -5 | tac | tee file >/dev/null

will keep the data stream within the pipes, by the input redirection file is opened for read early by the shell, before tee will write to it. The whole thing is pretty fast, although it will read the entire file. It keeps the inode of file. I'm not sure if and where the caveats are. I tested it with 10 million lines to get them down to 5000 on my Linux system.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

remove lines from file

file: 1 xxxxxxx 2 xxx xxx 5 xxx xxx ... 180 xxxxxx 200 xxx how to remove any lines with the first number range 1-180 (9 Replies)
Discussion started by: bluemoon1
9 Replies

2. Shell Programming and Scripting

To remove the lines in my file

Hi, There seems to some hack attempts in my site. I have attached the index page of my site and I need to remove the below lines from the index page. The below lines are at the center of the file. --> </style> <script>E V A L( unescape(... (5 Replies)
Discussion started by: gsiva
5 Replies

3. UNIX for Dummies Questions & Answers

vi to remove lines in file

All, I have a text file with several entries like below: personname personname.domain.com I know there is a way to use vi to remove only the personname.domain.com line. Can someone help? I believe that it involves /s/g/ something...I just can't remember the exact syntax. Thanks (2 Replies)
Discussion started by: kjbaumann
2 Replies

4. Shell Programming and Scripting

remove lines from file

Hi gurus, i'm trying to remove a number of lines from a large file using the following command: sed '1,5000d' oldfile > newfile Somehow the lines in the old file are not deleted... Am I doing this wrongly? Any suggestions? :confused: Thanks! :) wee (10 Replies)
Discussion started by: lweegp
10 Replies

5. Shell Programming and Scripting

remove : lines from file

A small question I have a test.txt file I have contents as: a:google b:yahoo : c:facebook : d:hotmail How do I remove the line with : my output should be a:google b:yahoo c:facebook d:hotmail (5 Replies)
Discussion started by: aronmelon
5 Replies

6. Shell Programming and Scripting

remove blank lines and merge lines in shell

Hi, I'm not a expert in shell programming, so i've come here to take help from u gurus. I'm trying to tailor a csv file that i got to make it work for the LOAD FROM command. I've a datatable csv of the below format - --in file format xx,xx,xx ,xx , , , , ,,xx, xxxx,, ,, xxx,... (11 Replies)
Discussion started by: dvah
11 Replies

7. Shell Programming and Scripting

Remove lines from file

Hey Gang- I have a list of servers. I want to exclude servers that begin with and end with certain characters. Is there an easy command to do this? Example wvm1234dev wvm1234pro uvm1122dev uvm1122bku uvm1344dev I want to exclude any lines that start with "wvm" OR "uvm" AND end... (7 Replies)
Discussion started by: idiotboy
7 Replies

8. Shell Programming and Scripting

Two files, remove lines from second based on lines in first

I have two files, a keepout.txt and a database.csv. They're unsorted, but could be sorted. keepout: user1 buser3 anuser19 notheruser27 database: user1,2343,"information about",field,blah,34 user2,4231,"mo info",etc,stuff,43 notheruser27,4344,"hiya",thing,more thing,423... (4 Replies)
Discussion started by: esoffron
4 Replies

9. Shell Programming and Scripting

Remove lines that are subsets of other lines in File

Hello everyone, Although it seems easy, I've been stuck with this problem for a moment now and I can't figure out a way to get it done. My problem is the following: I have a file where each line is a sequence of IP addresses, example : 10.0.0.1 10.0.0.2 10.0.0.5 10.0.0.1 10.0.0.2... (5 Replies)
Discussion started by: MisterJellyBean
5 Replies

10. Shell Programming and Scripting

awk to remove lines that do not start with digit and combine line or lines

I have been searching and trying to come up with an awk that will perform the following on a converted text file (original is a pdf). 1. Since the first two lines are (begin with) text they are removed 2. if $1 is a number then all text is merged (combined) into one line until the next... (3 Replies)
Discussion started by: cmccabe
3 Replies
TEE(2)							     Linux Programmer's Manual							    TEE(2)

NAME
tee - duplicating pipe content SYNOPSIS
#define _GNU_SOURCE /* See feature_test_macros(7) */ #include <fcntl.h> ssize_t tee(int fd_in, int fd_out, size_t len, unsigned int flags); DESCRIPTION
tee() duplicates up to len bytes of data from the pipe referred to by the file descriptor fd_in to the pipe referred to by the file descriptor fd_out. It does not consume the data that is duplicated from fd_in; therefore, that data can be copied by a subsequent splice(2). flags is a series of modifier flags, which share the name space with splice(2) and vmsplice(2): SPLICE_F_MOVE Currently has no effect for tee(); see splice(2). SPLICE_F_NONBLOCK Do not block on I/O; see splice(2) for further details. SPLICE_F_MORE Currently has no effect for tee(), but may be implemented in the future; see splice(2). SPLICE_F_GIFT Unused for tee(); see vmsplice(2). RETURN VALUE
Upon successful completion, tee() returns the number of bytes that were duplicated between the input and output. A return value of 0 means that there was no data to transfer, and it would not make sense to block, because there are no writers connected to the write end of the pipe referred to by fd_in. On error, tee() returns -1 and errno is set to indicate the error. ERRORS
EINVAL fd_in or fd_out does not refer to a pipe; or fd_in and fd_out refer to the same pipe. ENOMEM Out of memory. VERSIONS
The tee() system call first appeared in Linux 2.6.17; library support was added to glibc in version 2.5. CONFORMING TO
This system call is Linux-specific. NOTES
Conceptually, tee() copies the data between the two pipes. In reality no real data copying takes place though: under the covers, tee() assigns data in the output by merely grabbing a reference to the input. EXAMPLE
The following example implements a basic tee(1) program using the tee() system call. #define _GNU_SOURCE #include <fcntl.h> #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <errno.h> #include <limits.h> int main(int argc, char *argv[]) { int fd; int len, slen; if (argc != 2) { fprintf(stderr, "Usage: %s <file> ", argv[0]); exit(EXIT_FAILURE); } fd = open(argv[1], O_WRONLY | O_CREAT | O_TRUNC, 0644); if (fd == -1) { perror("open"); exit(EXIT_FAILURE); } do { /* * tee stdin to stdout. */ len = tee(STDIN_FILENO, STDOUT_FILENO, INT_MAX, SPLICE_F_NONBLOCK); if (len < 0) { if (errno == EAGAIN) continue; perror("tee"); exit(EXIT_FAILURE); } else if (len == 0) break; /* * Consume stdin by splicing it to a file. */ while (len > 0) { slen = splice(STDIN_FILENO, NULL, fd, NULL, len, SPLICE_F_MOVE); if (slen < 0) { perror("splice"); break; } len -= slen; } } while (1); close(fd); exit(EXIT_SUCCESS); } SEE ALSO
splice(2), vmsplice(2) COLOPHON
This page is part of release 3.44 of the Linux man-pages project. A description of the project, and information about reporting bugs, can be found at http://www.kernel.org/doc/man-pages/. Linux 2012-05-04 TEE(2)
All times are GMT -4. The time now is 07:03 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy