Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Need advice! Removing multiple entries in a single file! Post 302377838 by Scott on Saturday 5th of December 2009 12:22:10 PM
Old 12-05-2009
Code:
$ cat Test
awk '
  ($1 == $5) && ($2 == $6) && ($3 == $7) && ($4 == $8) { next }
  1
' file1

$ ./Test
1g12 A 14 19 2OAY A 326 331 AAAASA
1l7v A 68 73 1XVW B 72 77 AALAIS
1l7v A 68 73 1XXU A 65 70 AALAIS
1l7v A 68 73 1XXU B 65 70 AALAIS
1l7v A 68 73 1XXU C 65 70 AALAIS
1l7v A 68 73 1XXU D 65 70 AALAIS
1j1n A 439 444 1FUI B 360 365 ADVRTY

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Single to multiple line file

I am working with single line file with 589744523 characters having 542 "^M" (line feed) character. I want to make 542 different lines file from the single line file thr. shell program only (it can be done thr vi command) rd anil sorry for duplicate post previously, actually i don,t know... (6 Replies)
Discussion started by: anil_kut
6 Replies

2. Shell Programming and Scripting

Removing duplicate records in a file based on single column

Hi, I want to remove duplicate records including the first line based on column1. For example inputfile(filer.txt): ------------- 1,3000,5000 1,4000,6000 2,4000,600 2,5000,700 3,60000,4000 4,7000,7777 5,999,8888 expected output: ---------------- 3,60000,4000 4,7000,7777... (5 Replies)
Discussion started by: G.K.K
5 Replies

3. Shell Programming and Scripting

Removing duplicate records in a file based on single column explanation

I was reading this thread. It looks like a simpler way to say this is to only keep uniq lines based on field or column 1. https://www.unix.com/shell-programming-scripting/165717-removing-duplicate-records-file-based-single-column.html Can someone explain this command please? How are there no... (5 Replies)
Discussion started by: cokedude
5 Replies

4. Shell Programming and Scripting

Removing part of a file name and appending into a single file

I have two files like ABC_DEF_yyyyymmdd_hhmiss_XXX.txt and ABC_DEF_yyyyymmdd_hhmiss_YYY.txt. The date part is going to be changing everytime. How do i remove this date part of the file and create a single file like ABC_DEF_XXX.txt. (8 Replies)
Discussion started by: varlax
8 Replies

5. Shell Programming and Scripting

Awk match multiple columns in multiple lines in single file

Hi, Input 7488 7389 chr1.fa chr1.fa 3546 9887 chr5.fa chr9.fa 7387 7898 chrX.fa chr3.fa 7488 7389 chr21.fa chr3.fa 7488 7389 chr1.fa chr1.fa 3546 9887 chr9.fa chr5.fa 7898 7387 chrX.fa chr3.fa Desired Output 7488 7389 chr1.fa chr1.fa 2 3546 9887 chr5.fa chr9.fa 2... (2 Replies)
Discussion started by: jacobs.smith
2 Replies

6. Shell Programming and Scripting

Shell scripting - need to arrange the columns from multiple file into a single file

Hi friends please help me on below, i have 5 files like below file1 is x 10 y 20 z 15 file2 is x 100 z 245 file3 is y 78 z 23 file4 is x 100 (3 Replies)
Discussion started by: siva kumar
3 Replies

7. Shell Programming and Scripting

Execution of loop :Splitting a single file into multiple .dat file

hdr=$(cut -c1 $path$file|head -1)#extract header”H” trl=$(cut -c|path$file|tail -1)#extract trailer “T” SplitFile=$(cut -c 50-250 $path 1$newfile |sed'$/ *$//' head -1')# to trim white space and extract table name If; then # start loop if it is a header While read I #read file Do... (4 Replies)
Discussion started by: SwagatikaP1
4 Replies

8. Shell Programming and Scripting

Reducing multiple entries in a tri-lingual dictionary to single entries

Dear all, I am editing a tri-lingual dictionary for open source which has the following data structure English headwords <Tab>Devanagari Headwords<Tab>PersoArabic headwords as in the example below to mark, to number अंगणु (اَنگَڻُ) The English headword entry has at times more than one word,... (2 Replies)
Discussion started by: gimley
2 Replies

9. Shell Programming and Scripting

Removing multiple lines from input file, if multiple lines match a pattern.

GM, I have an issue at work, which requires a simple solution. But, after multiple attempts, I have not been able to hit on the code needed. I am assuming that sed, awk or even perl could do what I need. I have an application that adds extra blank page feeds, for multiple reports, when... (7 Replies)
Discussion started by: jxfish2
7 Replies

10. UNIX for Beginners Questions & Answers

Output file name and file contents of multiple files to a single file

I am trying to consolidate multiple information files (<hostname>.Linux.nfslist) into one file so that I can import it into Excel. I can get the file contents with cat *Linux.nfslist >> nfslist.txt. I need each line prefaced with the hostname. I am unsure how to do this. --- Post updated at... (5 Replies)
Discussion started by: Kentlee65
5 Replies
POSIX_MADVISE(3)					     Linux Programmer's Manual						  POSIX_MADVISE(3)

NAME
posix_madvise - give advice about patterns of memory usage SYNOPSIS
#include <sys/mman.h> int posix_madvise(void *addr, size_t len, int advice); Feature Test Macro Requirements for glibc (see feature_test_macros(7)): posix_madvise(): _POSIX_C_SOURCE >= 200112L DESCRIPTION
The posix_madvise() function allows an application to advise the system about its expected patterns of usage of memory in the address range starting at addr and continuing for len bytes. The system is free to use this advice in order to improve the performance of memory accesses (or to ignore the advice altogether), but calling posix_madvise() shall not affect the semantics of access to memory in the speci- fied range. The advice argument is one of the following: POSIX_MADV_NORMAL The application has no special advice regarding its memory usage patterns for the specified address range. This is the default behavior. POSIX_MADV_SEQUENTIAL The application expects to access the specified address range sequentially, running from lower addresses to higher addresses. Hence, pages in this region can be aggressively read ahead, and may be freed soon after they are accessed. POSIX_MADV_RANDOM The application expects to access the specified address range randomly. Thus, read ahead may be less useful than normally. POSIX_MADV_WILLNEED The application expects to access the specified address range in the near future. Thus, read ahead may be beneficial. POSIX_MADV_DONTNEED The application expects that it will not access the specified address range in the near future. RETURN VALUE
On success, posix_madvise() returns 0. On failure, it returns a positive error number. ERRORS
EINVAL addr is not a multiple of the system page size or len is negative. EINVAL advice is invalid. ENOMEM Addresses in the specified range are partially or completely outside the caller's address space. VERSIONS
Support for posix_madvise() first appeared in glibc version 2.2. CONFORMING TO
POSIX.1-2001. NOTES
POSIX.1 permits an implementation to generate an error if len is 0. On Linux, specifying len as 0 is permitted (as a successful no-op). In glibc, this function is implemented using madvise(2). However, since glibc 2.6, POSIX_MADV_DONTNEED is treated as a no-op, because the corresponding madvise(2) value, MADV_DONTNEED, has destructive semantics. SEE ALSO
madvise(2), posix_fadvise(2) COLOPHON
This page is part of release 4.15 of the Linux man-pages project. A description of the project, information about reporting bugs, and the latest version of this page, can be found at https://www.kernel.org/doc/man-pages/. Linux 2017-09-15 POSIX_MADVISE(3)
All times are GMT -4. The time now is 10:56 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy