Sponsored Content
Top Forums Shell Programming and Scripting remove of duplicate line from a file Post 302552637 by itkamaraj on Monday 5th of September 2011 03:24:50 AM
Old 09-05-2011
Code:
 
$nawk '{if($0~/^deep/ && count==0){count++}else{print $0;}}' input.txt > output.txt
ram
sham
deepram
sita
kumar

$cat input.txt
deepak
ram
sham
deepram
sita
kumar

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Remove Duplicate line

Hi, I have a scenario here where I have created a flatfile with the below mentioned information. File as you can see is dispalyed in three columns 1st column is FileNameString 2nd column is Report_Name (this has spaces) 3rd column is Flag Result file needed is, removal of duplicate... (1 Reply)
Discussion started by: Student37
1 Replies

2. UNIX for Dummies Questions & Answers

Remove duplicate entry in one line

Can anyone help me how can i print only the unique entry in a line? MI_AP MI_AP MI_CM MI_MF RC_NAP MBS_AP SF_RAN MBS_AP NT_CAR so that it will on output the one unique entry per line. MI_AP MI_CM MI_MF RC_NAP MBS_AP SF_RAN NT_CAR I can't find the same situation on the knowledge... (5 Replies)
Discussion started by: kharen11
5 Replies

3. Shell Programming and Scripting

remove duplicate words in a line

Hi, Please help! I have a file having duplicate words in some line and I want to remove the duplicate words. The order of the words in the output file doesn't matter. INPUT_FILE pink_kite red_pen ball pink_kite ball yellow_flower white no white no cloud nine_pen pink cloud pink nine_pen... (6 Replies)
Discussion started by: sam_2921
6 Replies

4. Shell Programming and Scripting

awk script to remove duplicate rows in line

i have the long file more than one ns and www and mx in the line like . i need the first ns record and first www and first mx from line . the records are seperated with tthe ; i am try ing in awk scripting not getiing the solution. ... (4 Replies)
Discussion started by: kiranmosarla
4 Replies

5. Shell Programming and Scripting

Remove duplicate line on condition

Hi Ive been scratching over this for some time with no solution. I have a file like this 1 bla bla 1 2 bla bla 2 4 bla bla 3 5 bla bla 1 6 bla bla 1 I want to remove consecutive occurrences of lines like bla bla 1, but the first column may be different. Any ideasss?? (23 Replies)
Discussion started by: jamie_123
23 Replies

6. UNIX for Dummies Questions & Answers

Remove Duplicate Two Line Pairs?

So I have a bunch of files that look like this >gi|33332323 MMKCRGVIMVVEKVMKRDGRIVPFDESRIRWAVQ--- >gi|45235353 MMKCR----VEKMRDVFFDESIRWAVQ They go on...sequences are much longer but all in two line (fasta) format. I want to remove duplicate pairs of ID(GI) number and sequence. I tried... (12 Replies)
Discussion started by: bakere19
12 Replies

7. Shell Programming and Scripting

Remove duplicate entries from the same line

Hello, I have a file which have several duplicate entries on the same line: File ID source 1 GM GF GM 2 GM GF GM GF GM GF GM GF GM GF 3 GM GF GM SF GM GF GM SF 4 FF FF FF FF 5 FF GM FF ... (2 Replies)
Discussion started by: nans
2 Replies

8. Shell Programming and Scripting

Remove duplicate line starting with a pattern

HI, I have the below input file /* ----------------- cmdsDlyStartFWJ -----------------*/ UNIX_JOB CMDS065J RUN ANY CMDNAME sleep 5 AGENT CMDSHP USER proddata RUN MON,TUE,WED,THU,FRI DELAYSUB 02:00 /* "Triggers daily file watcher jobs" */ ENVAR... (5 Replies)
Discussion started by: varun22486
5 Replies

9. Shell Programming and Scripting

Remove sections based on duplicate first line

Hi, I have a file with many sections in it. Each section is separated by a blank line. The first line of each section would determine if the section is duplicate or not. if the section is duplicate then remove the entire section from the file. below is the example of input and output.... (5 Replies)
Discussion started by: ahmedwaseem2000
5 Replies

10. UNIX for Dummies Questions & Answers

Using awk to remove duplicate line if field is empty

Hi all, I've got a file that has 12 fields. I've merged 2 files and there will be some duplicates in the following: FILE: 1. ABC, 12345, TEST1, BILLING, GV, 20/10/2012, C, 8, 100, AA, TT, 100 2. ABC, 12345, TEST1, BILLING, GV, 20/10/2012, C, 8, 100, AA, TT, (EMPTY) 3. CDC, 54321, TEST3,... (4 Replies)
Discussion started by: tugar
4 Replies
RAM(4)							     Kernel Interfaces Manual							    RAM(4)

NAME
ram - ram disk driver SYNOPSIS
/sys/conf/SYSTEM: NRAM ram_size # RAM disk size (512-byte blocks) major device number(s): block: 3 minor device encoding: must be zero (0) DESCRIPTION
The ram pseudo-device provides a very fast extended memory store. It's use is intended for file systems like /tmp and applications which need to access a reasonably large amount of data quickly. The amount of memory dedicated to the ram device is controlled by the NRAM definition in units of 512-byte blocks. This is also patchable in the system binary through the variable ram_size (though a patched system would have to be rebooted before any change took effect; see adb(1)). This makes it easy to test the effects of different ram disk sizes on system performance. It's important to note that any space given to the ram device is permanently allocated at system boot time. Dedicating too much memory can adversely affect system performance by forcing the system to swap heavily as in a memory poor environment. The block file accesses the ram disk via the system's buffering mechanism through a buffer sharing arrangement with the buffer cache. It may be read and written without regard to physical disk records. There is no `raw' interface since no speed advantage is gained by such an interface with the ram disk. DISK SUPPORT
The ram driver does not support pseudo-disks (partitions). The special files refer to the entire `drive' as a single sequentially addressed file. A typical use for the ram disk would be to mount /tmp on it. Note that if this arrangement is recorded in /etc/fstab then /etc/rc will have to be modified slightly to do a mkfs(8) on the ram disk before the standard file system checks are done. FILES
/dev/ram block file /dev/MAKEDEV script to create special files /dev/MAKEDEV.local script to localize special files SEE ALSO
hk(4), ra(4), rl(4), rk(4), rp(4), rx(4), si(4), xp(4) dtab(5), autoconfig(8) DIAGNOSTICS
ram: no space. There is not enough memory to allocate the space needed by the ram disk. The ram disk is disabled. Any attempts to access it will return an error. ram: not allocated. No memory was allocated to the ram disk and an attempt was made to open it. Either not enough memory was available at boot time or the kernel variable ram_size was set to zero. BUGS
The ram driver is only available under 2.11BSD. 3rd Berkeley Distribution Januray 27, 1996 RAM(4)
All times are GMT -4. The time now is 02:23 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy