Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Remove duplicate entry in one line
Top Forums UNIX for Dummies Questions & Answers Remove duplicate entry in one line Post 302125485 by Shell_Life on Thursday 5th of July 2007 02:56:27 PM
Old 07-05-2007
It works now -- great!
In any event, here is another solution in shell only:
Code:
while read mLine
do
  echo $mLine | tr ' ' '\n' > Temp
  cat -n Temp | tr '\t' '|' | sort -t'|' -k2 | uniq +7 | sort | sed 's/.*|//'
| paste -d' ' -s -
done < input_file

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Remove Duplicate line

Hi, I have a scenario here where I have created a flatfile with the below mentioned information. File as you can see is dispalyed in three columns 1st column is FileNameString 2nd column is Report_Name (this has spaces) 3rd column is Flag Result file needed is, removal of duplicate... (1 Reply)
Discussion started by: Student37
1 Replies

2. Shell Programming and Scripting

remove duplicate words in a line

Hi, Please help! I have a file having duplicate words in some line and I want to remove the duplicate words. The order of the words in the output file doesn't matter. INPUT_FILE pink_kite red_pen ball pink_kite ball yellow_flower white no white no cloud nine_pen pink cloud pink nine_pen... (6 Replies)
Discussion started by: sam_2921
6 Replies

3. Shell Programming and Scripting

awk script to remove duplicate rows in line

i have the long file more than one ns and www and mx in the line like . i need the first ns record and first www and first mx from line . the records are seperated with tthe ; i am try ing in awk scripting not getiing the solution. ... (4 Replies)
Discussion started by: kiranmosarla
4 Replies

4. Shell Programming and Scripting

remove of duplicate line from a file

I have a file a.txt having content like deepak ram sham deepram sita kumar I Want to delete the first line containing "deep" ... I tried using... grep -i 'deep' a.txt It gives me 2 rows...I want to delete the first one.. + need to know the command to delete the line from... (5 Replies)
Discussion started by: saluja.deepak
5 Replies

5. Shell Programming and Scripting

Remove duplicate line on condition

Hi Ive been scratching over this for some time with no solution. I have a file like this 1 bla bla 1 2 bla bla 2 4 bla bla 3 5 bla bla 1 6 bla bla 1 I want to remove consecutive occurrences of lines like bla bla 1, but the first column may be different. Any ideasss?? (23 Replies)
Discussion started by: jamie_123
23 Replies

6. UNIX for Dummies Questions & Answers

Remove Duplicate Two Line Pairs?

So I have a bunch of files that look like this >gi|33332323 MMKCRGVIMVVEKVMKRDGRIVPFDESRIRWAVQ--- >gi|45235353 MMKCR----VEKMRDVFFDESIRWAVQ They go on...sequences are much longer but all in two line (fasta) format. I want to remove duplicate pairs of ID(GI) number and sequence. I tried... (12 Replies)
Discussion started by: bakere19
12 Replies

7. Shell Programming and Scripting

Remove duplicate entries from the same line

Hello, I have a file which have several duplicate entries on the same line: File ID source 1 GM GF GM 2 GM GF GM GF GM GF GM GF GM GF 3 GM GF GM SF GM GF GM SF 4 FF FF FF FF 5 FF GM FF ... (2 Replies)
Discussion started by: nans
2 Replies

8. Shell Programming and Scripting

Remove duplicate line starting with a pattern

HI, I have the below input file /* ----------------- cmdsDlyStartFWJ -----------------*/ UNIX_JOB CMDS065J RUN ANY CMDNAME sleep 5 AGENT CMDSHP USER proddata RUN MON,TUE,WED,THU,FRI DELAYSUB 02:00 /* "Triggers daily file watcher jobs" */ ENVAR... (5 Replies)
Discussion started by: varun22486
5 Replies

9. Shell Programming and Scripting

Remove sections based on duplicate first line

Hi, I have a file with many sections in it. Each section is separated by a blank line. The first line of each section would determine if the section is duplicate or not. if the section is duplicate then remove the entire section from the file. below is the example of input and output.... (5 Replies)
Discussion started by: ahmedwaseem2000
5 Replies

10. UNIX for Dummies Questions & Answers

Using awk to remove duplicate line if field is empty

Hi all, I've got a file that has 12 fields. I've merged 2 files and there will be some duplicates in the following: FILE: 1. ABC, 12345, TEST1, BILLING, GV, 20/10/2012, C, 8, 100, AA, TT, 100 2. ABC, 12345, TEST1, BILLING, GV, 20/10/2012, C, 8, 100, AA, TT, (EMPTY) 3. CDC, 54321, TEST3,... (4 Replies)
Discussion started by: tugar
4 Replies

LEARN ABOUT NETBSD

itesio

ITESIO(4)						   BSD Kernel Interfaces Manual 						 ITESIO(4)

NAME

     itesio -- ITE IT87xxF Super I/O driver

SYNOPSIS

     itesio0 at isa? port 0x2e
     itesio1 at isa? port 0x4e

DESCRIPTION

     The itesio driver provides support for the Environment Controller and the Watchdog Timer on the IT87xxF Super I/Os, and may be used to moni-
     tor hardware sensors or setting up a watchdog timeout value for the system.

     The itesio driver has 15 sensors:

	   Sensor    Units    Typical Use
	   Temp0     uK       CPU Temp
	   Temp1     uK       System Temp
	   Temp2     uK       Aux Temp
	   VIN0      uV DC    Vcore A
	   VIN1      uV DC    Vcore B
	   VIN2      uV DC    +3.3V
	   VIN3      uV DC    +5V
	   VIN4      uV DC    +12V
	   VIN5      uV DC    -12V
	   VIN6      uV DC    -5V
	   VIN7      uV DC    STANDBY
	   VBAT      uV DC    VBAT
	   Fan0      RPM      CPU Fan
	   Fan1      RPM      System Fan
	   Fan2      RPM      Aux Fan

     The itesio Watchdog Timer is configurable via the wdogctl(8) utility and has a resolution between 1 and 65535 seconds.  The Watchdog Timer is
     not supported on the IT8705 Super I/O.

SEE ALSO

     envsys(4), envstat(8), wdogctl(8)

HISTORY

     The itesio driver first appeared in OpenBSD 3.4 and was then ported to NetBSD 4.0.

AUTHORS

     The itesio driver was written by Julien Bordet <zejames@greyhats.org> and Juan Romero Pardines <xtraeme@netbsd.org>.

BUGS

     Interrupt support is unimplemented.

BSD
								   April 2, 2008							       BSD
All times are GMT -4. The time now is 10:25 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy