Top Forums UNIX for Dummies Questions & Answers Need advice! Removing multiple entries in a single file! Post 302377838 by Scott on Saturday 5th of December 2009 12:22:10 PM
Code:
$ cat Test
awk '
  ($1 == $5) && ($2 == $6) && ($3 == $7) && ($4 == $8) { next }
  1
' file1

$ ./Test
1g12 A 14 19 2OAY A 326 331 AAAASA
1l7v A 68 73 1XVW B 72 77 AALAIS
1l7v A 68 73 1XXU A 65 70 AALAIS
1l7v A 68 73 1XXU B 65 70 AALAIS
1l7v A 68 73 1XXU C 65 70 AALAIS
1l7v A 68 73 1XXU D 65 70 AALAIS
1j1n A 439 444 1FUI B 360 365 ADVRTY

 
Test Your Knowledge in Computers #582
Difficulty: Medium
A buffered input or output stream is one in which data that is read or written is actually stored in memory in a buffer until such a time as the buffer is flushed and the accumulated data is passed to or from the actual file.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Single to multiple line file

I am working with single line file with 589744523 characters having 542 "^M" (line feed) character. I want to make 542 different lines file from the single line file thr. shell program only (it can be done thr vi command) rd anil sorry for duplicate post previously, actually i don,t know... (6 Replies)
Discussion started by: anil_kut
6 Replies

2. Shell Programming and Scripting

Removing duplicate records in a file based on single column

Hi, I want to remove duplicate records including the first line based on column1. For example inputfile(filer.txt): ------------- 1,3000,5000 1,4000,6000 2,4000,600 2,5000,700 3,60000,4000 4,7000,7777 5,999,8888 expected output: ---------------- 3,60000,4000 4,7000,7777... (5 Replies)
Discussion started by: G.K.K
5 Replies

3. Shell Programming and Scripting

Removing duplicate records in a file based on single column explanation

I was reading this thread. It looks like a simpler way to say this is to only keep uniq lines based on field or column 1. https://www.unix.com/shell-programming-scripting/165717-removing-duplicate-records-file-based-single-column.html Can someone explain this command please? How are there no... (5 Replies)
Discussion started by: cokedude
5 Replies

4. Shell Programming and Scripting

Removing part of a file name and appending into a single file

I have two files like ABC_DEF_yyyyymmdd_hhmiss_XXX.txt and ABC_DEF_yyyyymmdd_hhmiss_YYY.txt. The date part is going to be changing everytime. How do i remove this date part of the file and create a single file like ABC_DEF_XXX.txt. (8 Replies)
Discussion started by: varlax
8 Replies

5. Shell Programming and Scripting

Awk match multiple columns in multiple lines in single file

Hi, Input 7488 7389 chr1.fa chr1.fa 3546 9887 chr5.fa chr9.fa 7387 7898 chrX.fa chr3.fa 7488 7389 chr21.fa chr3.fa 7488 7389 chr1.fa chr1.fa 3546 9887 chr9.fa chr5.fa 7898 7387 chrX.fa chr3.fa Desired Output 7488 7389 chr1.fa chr1.fa 2 3546 9887 chr5.fa chr9.fa 2... (2 Replies)
Discussion started by: jacobs.smith
2 Replies

6. Shell Programming and Scripting

Shell scripting - need to arrange the columns from multiple file into a single file

Hi friends please help me on below, i have 5 files like below file1 is x 10 y 20 z 15 file2 is x 100 z 245 file3 is y 78 z 23 file4 is x 100 (3 Replies)
Discussion started by: siva kumar
3 Replies

7. Shell Programming and Scripting

Execution of loop :Splitting a single file into multiple .dat file

hdr=$(cut c1 $path$file|head -1)#extract headerH trl=$(cut c|path$file|tail -1)#extract trailer T SplitFile=$(cut c 50-250 $path 1$newfile |sed$/ *$// head -1)# to trim white space and extract table name If; then # start loop if it is a header While read I #read file Do... (4 Replies)
Discussion started by: SwagatikaP1
4 Replies

8. Shell Programming and Scripting

Reducing multiple entries in a tri-lingual dictionary to single entries

Dear all, I am editing a tri-lingual dictionary for open source which has the following data structure English headwords <Tab>Devanagari Headwords<Tab>PersoArabic headwords as in the example below to mark, to number अंगणु (اَنگَڻُ) The English headword entry has at times more than one word,... (2 Replies)
Discussion started by: gimley
2 Replies

9. Shell Programming and Scripting

Removing multiple lines from input file, if multiple lines match a pattern.

GM, I have an issue at work, which requires a simple solution. But, after multiple attempts, I have not been able to hit on the code needed. I am assuming that sed, awk or even perl could do what I need. I have an application that adds extra blank page feeds, for multiple reports, when... (7 Replies)
Discussion started by: jxfish2
7 Replies

10. UNIX for Beginners Questions & Answers

Output file name and file contents of multiple files to a single file

I am trying to consolidate multiple information files (<hostname>.Linux.nfslist) into one file so that I can import it into Excel. I can get the file contents with cat *Linux.nfslist >> nfslist.txt. I need each line prefaced with the hostname. I am unsure how to do this. --- Post updated at... (5 Replies)
Discussion started by: Kentlee65
5 Replies
Test::LectroTest::RegressionTesting(3pm)		User Contributed Perl Documentation		  Test::LectroTest::RegressionTesting(3pm)

NAME
Test::LectroTest::RegressionTesting - How to do regression testing (for free!) SYNOPSIS
use Test::LectroTest regressions => "regressions.txt"; # -- OR -- use Test::LectroTest playback_failures => "regression_suite_for_my_module.txt", record_failures => "failures_in_the_field.txt"; # -- OR -- use Test::LectroTest::Compat regressions => "regressions.txt"; # -- OR -- use Test::LectroTest::Compat playback_failures => "regression_suite_for_my_module.txt", record_failures => "failures_in_the_field.txt"; DESCRIPTION
Say that LectroTest uncovers a bug in your software by finding a random test case that proves one of your properties to be false. If you apply a fix for the bug, how can you be sure that LectroTest will re-test the property using the exact same test case that "broke" it before, just to be certain the bug really is fixed? And how can you be sure that future changes to your code will not reintroduce the same bug without your knowing it? For situations like these, LectroTest can record failure-causing test cases to a file, and it can play those test cases back as part of its normal testing strategy. The easiest way to take advantage of this feature is to set the regressions parameter when you "use" Test::LectroTest or Test::LectroTest::Compat: use Test::LectroTest regressions => "regressions.txt"; This tells LectroTest to use the file "regressions.txt" for both recording and playing back failures. If you want to record and play back from separate files, use the record_failures and playback_failures options: use Test::LectroTest::Compat playback_failures => "regression_suite_for_my_module.txt", record_failures => "failures_in_the_field.txt"; Here is how it works: 1. When testing a property named N, LectroTest will check for a play-back file. If the file exists, LectroTest will search it for test cases associated with N. If any such test cases exist, LectroTest will play them back before and in addition to performing the usual, random testing of the property. 2. When performing the usual, random testing of a property named N, if a failure occurs (i.e., LectroTest finds a counterexample), LectroTest will record the test case that caused the failure to the recording file, associating the test case with the name N. NOTE: If you pass any of the recording or playback parameters to Test::LectroTest::Compat, you must have version 0.3500 or greater of Test::LectroTest installed. (Module authors, update your modules' build dependencies accordingly.) The Test::LectroTest module itself, however, has always ignored unfamiliar parameters, and thus these options are backward compatible with older versions. SEE ALSO
Test::LectroTest gives a quick overview of automatic, specification-based testing with LectroTest. This module accepts failure recording and play-back options. Test::LectroTest::Compat lets you mix LectroTest with the popular family of Test::Builder-based modules such as Test::Simple and Test::More. This module accepts failure recording and play-back options. LECTROTEST HOME
The LectroTest home is http://community.moertel.com/LectroTest. There you will find more documentation, presentations, mailing-list archives, a wiki, and other helpful LectroTest-related resources. It's also the best place to ask questions. AUTHOR
Tom Moertel (tom@moertel.com) COPYRIGHT and LICENSE Copyright (c) 2004-06 by Thomas G Moertel. All rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.12.3 2007-08-30 Test::LectroTest::RegressionTesting(3pm)

Featured Tech Videos

All times are GMT -4. The time now is 02:34 AM.
Unix & Linux Forums Content Copyright 1993-2020. All Rights Reserved.
Privacy Policy