Sponsored Content
Top Forums Shell Programming and Scripting Two files, remove lines from second based on lines in first Post 302886409 by esoffron on Friday 31st of January 2014 09:07:36 AM
Old 01-31-2014
Awesome, except it isn't working on my "real" files. It works entirely correctly on the sample data I gave -- your answer is correct.

I wonder if one of the hacks I had tried before might have worked...

In the real world, with messy data, what might be keeping this from working? Some unprintable nonsense? Line ending CR/LF vs CR vs LF crud? Do you have tips for me where this might be brittle?

Thanks!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove lines, Sorted with Time based columns using AWK & SORT

Hi having a file as follows MediaErr.log 84 Server1 Policy1 Schedule1 master1 05/08/2008 02:12:16 84 Server1 Policy1 Schedule1 master1 05/08/2008 02:22:47 84 Server1 Policy1 Schedule1 master1 05/08/2008 03:41:26 84 Server1 Policy1 ... (1 Reply)
Discussion started by: karthikn7974
1 Replies

2. Shell Programming and Scripting

remove lines based on score criteria

Hi guys, Please guide for Solution. PART-I INPUT FILE (has 2 columns ID and score) TC5584_1 93.9 DV161411_2 79.5 BP132435_5 46.8 EB682112_1 34.7 BP132435_4 29.5 TC13860_2 10.1 OUTPUT FILE (It shudn't contain the line ' BP132435_4 29.5 ' as BP132435 is repeated... (2 Replies)
Discussion started by: smriti_shridhar
2 Replies

3. Shell Programming and Scripting

Remove lines based on contents of another file

So, this issue is driving me nuts! I was hoping to get a lending hand here... I have 2 files: file1.txt contains: this is example1 this is example2 this is example3 this is example4 this is example5 file2.txt contains: example3 example5 Basically, I need a script or command to... (4 Replies)
Discussion started by: bashshadow1979
4 Replies

4. Shell Programming and Scripting

Remove lines from XML based on condition

Hi, I need to remove some lines from an XML file is the value within a tag is empty. Imagine this scenario, <acd><acdID>2</acdID><logon></logon></acd> <acd><acdID></acdID><logon></logon></acd> <acd><acdID></acdID><logon></logon></acd> <acd><acdID></acdID><logon></logon></acd> I... (3 Replies)
Discussion started by: giles.cardew
3 Replies

5. UNIX for Dummies Questions & Answers

remove duplicate lines based on two columns and judging from a third one

hello all, I have an input file with four columns like this with a lot of lines and for example, line 1 and line 5 match because the first 4 characters match and the fourth column matches too. I want to keep the line that has the lowest number in the third column. So I discard line 5.... (5 Replies)
Discussion started by: TheTransporter
5 Replies

6. Shell Programming and Scripting

Remove lines based on column value

Hi All, I just need a quick fix here. I need to delete all lines containing "." in the 6th column. Input: 1 1055498 . G T 5.46 . 1 1902377 . C T 7.80 . 1 1031540 . A G 34.01 PASS 1 ... (2 Replies)
Discussion started by: Hkins552
2 Replies

7. Shell Programming and Scripting

Remove duplicate lines based on field and sort

I have a csv file that I would like to remove duplicate lines based on field 1 and sort. I don't care about any of the other fields but I still wanna keep there data intact. I was thinking I could do something like this but I have no idea how to print the full line with this. Please show any method... (8 Replies)
Discussion started by: cokedude
8 Replies

8. Shell Programming and Scripting

Remove certain lines from file based on start of line except beginning and ending

Hi, I have multiple large files which consist of the below format: I am trying to write an awk or sed script to remove all occurrences of the 00 record except the first and remove all of the 80 records except the last one. Any help would be greatly appreciated. (10 Replies)
Discussion started by: nwalsh88
10 Replies

9. UNIX for Dummies Questions & Answers

Remove lines in a positional file based on string value

Gurus, I am relatively new to Unix scripting and am struck with a problem in my script. I have positional input file which has a FLAG indicator in at position 11 in every record of the file. If the Flag has value =Y, then the record from the input needs to be written to a new file.However if... (3 Replies)
Discussion started by: gsam
3 Replies

10. Shell Programming and Scripting

Remove duplicate lines from file based on fields

Dear community, I have to remove duplicate lines from a file contains a very big ammount of rows (milions?) based on 1st and 3rd columns The data are like this: Region 23/11/2014 09:11:36 41752 Medio 23/11/2014 03:11:38 4132 Info 23/11/2014 05:11:09 4323... (2 Replies)
Discussion started by: Lord Spectre
2 Replies
RCSCLEAN(1)						      General Commands Manual						       RCSCLEAN(1)

NAME
rcsclean - clean up working files SYNOPSIS
rcsclean [options] [ file ... ] DESCRIPTION
rcsclean removes files that are not being worked on. rcsclean -u also unlocks and removes files that are being worked on but have not changed. For each file given, rcsclean compares the working file and a revision in the corresponding RCS file. If it finds a difference, it does nothing. Otherwise, it first unlocks the revision if the -u option is given, and then removes the working file unless the working file is writable and the revision is locked. It logs its actions by outputting the corresponding rcs -u and rm -f commands on the standard output. Files are paired as explained in ci(1). If no file is given, all working files in the current directory are cleaned. Filenames matching an RCS suffix denote RCS files; all others denote working files. The number of the revision to which the working file is compared may be attached to any of the options -n, -q, -r, or -u. If no revision number is specified, then if the -u option is given and the caller has one revision locked, rcsclean uses that revision; otherwise rcsclean uses the latest revision on the default branch, normally the root. rcsclean is useful for clean targets in makefiles. See also rcsdiff(1), which prints out the differences, and ci(1), which normally reverts to the previous revision if a file was not changed. OPTIONS
-ksubst Use subst style keyword substitution when retrieving the revision for comparison. See co(1) for details. -n[rev] Do not actually remove any files or unlock any revisions. Using this option will tell you what rcsclean would do without actually doing it. -q[rev] Do not log the actions taken on standard output. -r[rev] This option has no effect other than specifying the revision for comparison. -T Preserve the modification time on the RCS file even if the RCS file changes because a lock is removed. This option can suppress extensive recompilation caused by a make(1) dependency of some other copy of the working file on the RCS file. Use this option with care; it can suppress recompilation even when it is needed, i.e. when the lock removal would mean a change to keyword strings in the other working file. -u[rev] Unlock the revision if it is locked and no difference is found. -V Print RCS's version number. -Vn Emulate RCS version n. See co(1) for details. -xsuffixes Use suffixes to characterize RCS files. See ci(1) for details. -zzone Use zone as the time zone for keyword substitution; see co(1) for details. EXAMPLES
rcsclean *.c *.h removes all working files ending in .c or .h that were not changed since their checkout. rcsclean removes all working files in the current directory that were not changed since their checkout. FILES
rcsclean accesses files much as ci(1) does. ENVIRONMENT
RCSINIT Options prepended to the argument list, separated by spaces. A backslash escapes spaces within an option. The RCSINIT options are prepended to the argument lists of most RCS commands. Useful RCSINIT options include -q, -V, -x, and -z. RCS_MEM_LIMIT An integer lim, measured in kilobytes, specifying the threshold under which commands will try to use memory-based operations for processing the RCS file. (For RCS files of size lim kilobytes or greater, RCS will use the slower standard input/output routines.) Default value is 256. TMPDIR Name of the temporary directory. If not set, the environment variables TMP and TEMP are inspected instead and the first value found is taken; if none of them are set, a host-dependent default is used, typically /tmp. DIAGNOSTICS
The exit status is zero if and only if all operations were successful. Missing working files and RCS files are silently ignored. IDENTIFICATION
Author: Walter F. Tichy. Manual Page Revision: 5.8.1; Release Date: 2012-06-06. Copyright (C) 2010-2012 Thien-Thi Nguyen. Copyright (C) 1990, 1991, 1992, 1993 Paul Eggert. Copyright (C) 1982, 1988, 1989 Walter F. Tichy. SEE ALSO
ci(1), co(1), ident(1), rcs(1), rcsdiff(1), rcsmerge(1), rlog(1), rcsfile(5). Walter F. Tichy, RCS--A System for Version Control, Software--Practice & Experience 15, 7 (July 1985), 637-654. BUGS
At least one file must be given in older Unix versions that do not provide the needed directory scanning operations. GNU RCS 5.8.1 2012-06-06 RCSCLEAN(1)
All times are GMT -4. The time now is 12:34 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy