Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Two files, remove lines from second based on lines in first
Top Forums Shell Programming and Scripting Two files, remove lines from second based on lines in first Post 302886407 by esoffron on Friday 31st of January 2014 08:49:19 AM
Old 01-31-2014
Two files, remove lines from second based on lines in first
I have two files, a keepout.txt and a database.csv. They're unsorted, but could be sorted.

keepout:
Code:
user1
buser3
anuser19
notheruser27

database:
Code:
user1,2343,"information about",field,blah,34
user2,4231,"mo info",etc,stuff,43
notheruser27,4344,"hiya",thing,more thing,423
user5,6666,"test text",info,stuff,833

Output would be
Code:
user2,4231,"mo info",etc,stuff,43
user5,6666,"test text",info,stuff,833

Make sense? If a line in keepout matches a field (would be nice to just be first field) in database then I want to dump the line in the output.

I've tried awky ways and greppy ways but can't get it just right. It's not entire lines that are matching, just the lines. Kind of like a loopy grep -v.

Thanks for your help!
Last edited by Franklin52; 01-31-2014 at 01:41 PM.. Reason: Please use code tags
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove lines, Sorted with Time based columns using AWK & SORT

Hi having a file as follows MediaErr.log 84 Server1 Policy1 Schedule1 master1 05/08/2008 02:12:16 84 Server1 Policy1 Schedule1 master1 05/08/2008 02:22:47 84 Server1 Policy1 Schedule1 master1 05/08/2008 03:41:26 84 Server1 Policy1 ... (1 Reply)
Discussion started by: karthikn7974
1 Replies

2. Shell Programming and Scripting

remove lines based on score criteria

Hi guys, Please guide for Solution. PART-I INPUT FILE (has 2 columns ID and score) TC5584_1 93.9 DV161411_2 79.5 BP132435_5 46.8 EB682112_1 34.7 BP132435_4 29.5 TC13860_2 10.1 OUTPUT FILE (It shudn't contain the line ' BP132435_4 29.5 ' as BP132435 is repeated... (2 Replies)
Discussion started by: smriti_shridhar
2 Replies

3. Shell Programming and Scripting

Remove lines based on contents of another file

So, this issue is driving me nuts! I was hoping to get a lending hand here... I have 2 files: file1.txt contains: this is example1 this is example2 this is example3 this is example4 this is example5 file2.txt contains: example3 example5 Basically, I need a script or command to... (4 Replies)
Discussion started by: bashshadow1979
4 Replies

4. Shell Programming and Scripting

Remove lines from XML based on condition

Hi, I need to remove some lines from an XML file is the value within a tag is empty. Imagine this scenario, <acd><acdID>2</acdID><logon></logon></acd> <acd><acdID></acdID><logon></logon></acd> <acd><acdID></acdID><logon></logon></acd> <acd><acdID></acdID><logon></logon></acd> I... (3 Replies)
Discussion started by: giles.cardew
3 Replies

5. UNIX for Dummies Questions & Answers

remove duplicate lines based on two columns and judging from a third one

hello all, I have an input file with four columns like this with a lot of lines and for example, line 1 and line 5 match because the first 4 characters match and the fourth column matches too. I want to keep the line that has the lowest number in the third column. So I discard line 5.... (5 Replies)
Discussion started by: TheTransporter
5 Replies

6. Shell Programming and Scripting

Remove lines based on column value

Hi All, I just need a quick fix here. I need to delete all lines containing "." in the 6th column. Input: 1 1055498 . G T 5.46 . 1 1902377 . C T 7.80 . 1 1031540 . A G 34.01 PASS 1 ... (2 Replies)
Discussion started by: Hkins552
2 Replies

7. Shell Programming and Scripting

Remove duplicate lines based on field and sort

I have a csv file that I would like to remove duplicate lines based on field 1 and sort. I don't care about any of the other fields but I still wanna keep there data intact. I was thinking I could do something like this but I have no idea how to print the full line with this. Please show any method... (8 Replies)
Discussion started by: cokedude
8 Replies

8. Shell Programming and Scripting

Remove certain lines from file based on start of line except beginning and ending

Hi, I have multiple large files which consist of the below format: I am trying to write an awk or sed script to remove all occurrences of the 00 record except the first and remove all of the 80 records except the last one. Any help would be greatly appreciated. (10 Replies)
Discussion started by: nwalsh88
10 Replies

9. UNIX for Dummies Questions & Answers

Remove lines in a positional file based on string value

Gurus, I am relatively new to Unix scripting and am struck with a problem in my script. I have positional input file which has a FLAG indicator in at position 11 in every record of the file. If the Flag has value =Y, then the record from the input needs to be written to a new file.However if... (3 Replies)
Discussion started by: gsam
3 Replies

10. Shell Programming and Scripting

Remove duplicate lines from file based on fields

Dear community, I have to remove duplicate lines from a file contains a very big ammount of rows (milions?) based on 1st and 3rd columns The data are like this: Region 23/11/2014 09:11:36 41752 Medio 23/11/2014 03:11:38 4132 Info 23/11/2014 05:11:09 4323... (2 Replies)
Discussion started by: Lord Spectre
2 Replies

LEARN ABOUT DEBIAN

smpteframe

SMPTEFrame(3)							    libltcsmpte 						     SMPTEFrame(3)

NAME

       SMPTEFrame -

       Raw 80 bit SMPTE frame.

SYNOPSIS

       #include <ltcsmpte.h>

   Data Fields
       unsigned int frameUnits:4
       unsigned int user1:4
       unsigned int frameTens:2
       unsigned int dfbit:1
       unsigned int colFrm:1
       unsigned int user2:4
       unsigned int secsUnits:4
       unsigned int user3:4
       unsigned int secsTens:3
       unsigned int biphaseMarkPhaseCorrection:1
       unsigned int user4:4
       unsigned int minsUnits:4
       unsigned int user5:4
       unsigned int minsTens:3
       unsigned int binaryGroupFlagBit1:1
       unsigned int user6:4
       unsigned int hoursUnits:4
       unsigned int user7:4
       unsigned int hoursTens:2
       unsigned int reserved:1
       unsigned int binaryGroupFlagBit2:1
       unsigned int user8:4
       unsigned int syncWord:16

Detailed Description
       Raw 80 bit SMPTE frame.

       Definition at line 140 of file ltcsmpte.h.

Field Documentation
   unsigned int SMPTEFrame::binaryGroupFlagBit1
       Definition at line 160 of file ltcsmpte.h.

   unsigned int SMPTEFrame::binaryGroupFlagBit2
       Definition at line 168 of file ltcsmpte.h.

   unsigned int SMPTEFrame::biphaseMarkPhaseCorrection
       Definition at line 153 of file ltcsmpte.h.

   unsigned int SMPTEFrame::colFrm
       Definition at line 146 of file ltcsmpte.h.

   unsigned int SMPTEFrame::dfbit
       Examples:
	   tests/decoder.c.

       Definition at line 145 of file ltcsmpte.h.

   unsigned int SMPTEFrame::frameTens
       Definition at line 144 of file ltcsmpte.h.

   unsigned int SMPTEFrame::frameUnits
       Definition at line 141 of file ltcsmpte.h.

   unsigned int SMPTEFrame::hoursTens
       Definition at line 166 of file ltcsmpte.h.

   unsigned int SMPTEFrame::hoursUnits
       Definition at line 163 of file ltcsmpte.h.

   unsigned int SMPTEFrame::minsTens
       Definition at line 159 of file ltcsmpte.h.

   unsigned int SMPTEFrame::minsUnits
       Definition at line 156 of file ltcsmpte.h.

   unsigned int SMPTEFrame::reserved
       Definition at line 167 of file ltcsmpte.h.

   unsigned int SMPTEFrame::secsTens
       Definition at line 152 of file ltcsmpte.h.

   unsigned int SMPTEFrame::secsUnits
       Definition at line 149 of file ltcsmpte.h.

   unsigned int SMPTEFrame::syncWord
       Definition at line 171 of file ltcsmpte.h.

   unsigned int SMPTEFrame::user1
       Definition at line 142 of file ltcsmpte.h.

   unsigned int SMPTEFrame::user2
       Definition at line 147 of file ltcsmpte.h.

   unsigned int SMPTEFrame::user3
       Definition at line 150 of file ltcsmpte.h.

   unsigned int SMPTEFrame::user4
       Definition at line 154 of file ltcsmpte.h.

   unsigned int SMPTEFrame::user5
       Definition at line 157 of file ltcsmpte.h.

   unsigned int SMPTEFrame::user6
       Definition at line 161 of file ltcsmpte.h.

   unsigned int SMPTEFrame::user7
       Definition at line 164 of file ltcsmpte.h.

   unsigned int SMPTEFrame::user8
       Definition at line 169 of file ltcsmpte.h.

Author
       Generated automatically by Doxygen for libltcsmpte from the source code.

Version 0.4.4							  Fri Apr 27 2012						     SMPTEFrame(3)
All times are GMT -4. The time now is 10:32 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy