Sponsored Content
Top Forums Shell Programming and Scripting Two files, remove lines from second based on lines in first Post 302886411 by esoffron on Friday 31st of January 2014 09:27:05 AM
Old 01-31-2014
Craaaaaap. I think I had this on my own but was bitten by a trip through an editor that changed the line endings. So, takeaway for my internet self when I search for this again.

1) Yoda was right! Thanks for the help!
2) Self, it might not be your code but the input. Maybe create your test files before making them up to ask for help...
3) LF line endings vs CR line endings matter, and seem to be what broke this for you. (LF is what you needed and what seems to fix it.)
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove lines, Sorted with Time based columns using AWK & SORT

Hi having a file as follows MediaErr.log 84 Server1 Policy1 Schedule1 master1 05/08/2008 02:12:16 84 Server1 Policy1 Schedule1 master1 05/08/2008 02:22:47 84 Server1 Policy1 Schedule1 master1 05/08/2008 03:41:26 84 Server1 Policy1 ... (1 Reply)
Discussion started by: karthikn7974
1 Replies

2. Shell Programming and Scripting

remove lines based on score criteria

Hi guys, Please guide for Solution. PART-I INPUT FILE (has 2 columns ID and score) TC5584_1 93.9 DV161411_2 79.5 BP132435_5 46.8 EB682112_1 34.7 BP132435_4 29.5 TC13860_2 10.1 OUTPUT FILE (It shudn't contain the line ' BP132435_4 29.5 ' as BP132435 is repeated... (2 Replies)
Discussion started by: smriti_shridhar
2 Replies

3. Shell Programming and Scripting

Remove lines based on contents of another file

So, this issue is driving me nuts! I was hoping to get a lending hand here... I have 2 files: file1.txt contains: this is example1 this is example2 this is example3 this is example4 this is example5 file2.txt contains: example3 example5 Basically, I need a script or command to... (4 Replies)
Discussion started by: bashshadow1979
4 Replies

4. Shell Programming and Scripting

Remove lines from XML based on condition

Hi, I need to remove some lines from an XML file is the value within a tag is empty. Imagine this scenario, <acd><acdID>2</acdID><logon></logon></acd> <acd><acdID></acdID><logon></logon></acd> <acd><acdID></acdID><logon></logon></acd> <acd><acdID></acdID><logon></logon></acd> I... (3 Replies)
Discussion started by: giles.cardew
3 Replies

5. UNIX for Dummies Questions & Answers

remove duplicate lines based on two columns and judging from a third one

hello all, I have an input file with four columns like this with a lot of lines and for example, line 1 and line 5 match because the first 4 characters match and the fourth column matches too. I want to keep the line that has the lowest number in the third column. So I discard line 5.... (5 Replies)
Discussion started by: TheTransporter
5 Replies

6. Shell Programming and Scripting

Remove lines based on column value

Hi All, I just need a quick fix here. I need to delete all lines containing "." in the 6th column. Input: 1 1055498 . G T 5.46 . 1 1902377 . C T 7.80 . 1 1031540 . A G 34.01 PASS 1 ... (2 Replies)
Discussion started by: Hkins552
2 Replies

7. Shell Programming and Scripting

Remove duplicate lines based on field and sort

I have a csv file that I would like to remove duplicate lines based on field 1 and sort. I don't care about any of the other fields but I still wanna keep there data intact. I was thinking I could do something like this but I have no idea how to print the full line with this. Please show any method... (8 Replies)
Discussion started by: cokedude
8 Replies

8. Shell Programming and Scripting

Remove certain lines from file based on start of line except beginning and ending

Hi, I have multiple large files which consist of the below format: I am trying to write an awk or sed script to remove all occurrences of the 00 record except the first and remove all of the 80 records except the last one. Any help would be greatly appreciated. (10 Replies)
Discussion started by: nwalsh88
10 Replies

9. UNIX for Dummies Questions & Answers

Remove lines in a positional file based on string value

Gurus, I am relatively new to Unix scripting and am struck with a problem in my script. I have positional input file which has a FLAG indicator in at position 11 in every record of the file. If the Flag has value =Y, then the record from the input needs to be written to a new file.However if... (3 Replies)
Discussion started by: gsam
3 Replies

10. Shell Programming and Scripting

Remove duplicate lines from file based on fields

Dear community, I have to remove duplicate lines from a file contains a very big ammount of rows (milions?) based on 1st and 3rd columns The data are like this: Region 23/11/2014 09:11:36 41752 Medio 23/11/2014 03:11:38 4132 Info 23/11/2014 05:11:09 4323... (2 Replies)
Discussion started by: Lord Spectre
2 Replies
eol(3pm)						User Contributed Perl Documentation						  eol(3pm)

NAME
PerlIO::eol - PerlIO layer for normalizing line endings VERSION
This document describes version 0.14 of PerlIO::eol, released December 18, 2006. SYNOPSIS
binmode STDIN, ":raw:eol(LF)"; binmode STDOUT, ":raw:eol(CRLF)"; open FH, "+<:raw:eol(LF-Native)", "file"; binmode STDOUT, ":raw:eol(CRLF?)"; # warns on mixed newlines binmode STDOUT, ":raw:eol(CRLF!)"; # dies on mixed newlines use PerlIO::eol qw( eol_is_mixed ); my $pos = eol_is_mixed( "mixed string " ); DESCRIPTION
This layer normalizes any of "CR", "LF", "CRLF" and "Native" into the designated line ending. It works for both input and output handles. If you specify two different line endings joined by a "-", it will use the first one for reading and the second one for writing. For example, the "LF-CRLF" encoding means that all input should be normalized to "LF", and all output should be normalized to "CRLF". By default, data with mixed newlines are normalized silently. Append a "!" to the line ending will raise a fatal exception when mixed newlines are spotted. Append a "?" will raise a warning instead. It is advised to pop any potential ":crlf" or encoding layers before this layer; this is usually done using a ":raw" prefix. This module also optionally exports a "eol_is_mixed" function; it takes a string and returns the position of the first inconsistent line ending found in that string, or 0 if the line endings are consistent. The "CR", "LF", "CRLF" and "NATIVE" constants are also exported at request. AUTHORS
Audrey Tang <autrijus@autrijus.org>. Janitorial help by Gaal Yahas <gaal@forum2.org>. Inspired by PerlIO::nline by Ben Morrow, <PerlIO-eol@morrow.me.uk>. COPYRIGHT
Copyright 2004-2006 by Audrey Tang <audreyt@audreyt.org>. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. See <http://www.perl.com/perl/misc/Artistic.html> perl v5.14.2 2006-12-16 eol(3pm)
All times are GMT -4. The time now is 04:04 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy