Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: How to remove duplicates using for loop?
Top Forums Shell Programming and Scripting How to remove duplicates using for loop? Post 302984880 by Don Cragun on Wednesday 2nd of November 2016 03:52:39 AM
Old 11-02-2016
Is this a homework assignment? Homework and coursework questions can only be posted in the Homework & Coursework forum following the special homework rules.

Please review the rules, which you agreed to when you registered, if you have not already done so.

If you did not post homework, please explain the company you work for and the nature of the problem you are working on. Then, tell us what shell and operating system you're using. And, show us (in CODE tags) what you have tried to use to solve this problem on your own.

If you did post homework in the main forums, please review the guidelines for posting homework and repost.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

fastest way to remove duplicates.

I have searched the FAQ - by using sort, duplicates, etc.... but I didn't get any articles or results on it. Currently, I am using: sort -u file1 > file2 to remove duplicates. For a file size of 1giga byte approx. time taken to remove duplicates is 1hr 21 mins. Is there any other faster way... (15 Replies)
Discussion started by: radhika
15 Replies

2. Shell Programming and Scripting

Remove duplicates

Hello Experts, I have two files named old and new. Below are my example files. I need to compare and print the records that only exist in my new file. I tried the below awk script, this script works perfectly well if the records have exact match, the issue I have is my old file has got extra... (4 Replies)
Discussion started by: forumthreads
4 Replies

3. Shell Programming and Scripting

Script to remove duplicates

Hi I need a script that removes the duplicate records and write it to a new file for example I have a file named test.txt and it looks like abcd.23 abcd.24 abcd.25 qwer.25 qwer.26 qwer.98 I want to pick only $1 and compare with the next record and the output should be abcd.23... (6 Replies)
Discussion started by: antointoronto
6 Replies

4. Shell Programming and Scripting

remove duplicates and sort

Hi, I'm using the below command to sort and remove duplicates in a file. But, i need to make this applied to the same file instead of directing it to another. Thanks (6 Replies)
Discussion started by: dvah
6 Replies

5. Shell Programming and Scripting

bash - remove duplicates

I need to use a bash script to remove duplicate files from a download list, but I cannot use uniq because the urls are different. I need to go from this: http://***/fae78fe/file1.wmv http://***/39du7si/file1.wmv http://***/d8el2hd/file2.wmv http://***/h893js3/file2.wmv to this: ... (2 Replies)
Discussion started by: locoroco
2 Replies

6. UNIX for Dummies Questions & Answers

Remove duplicates from a file

Can u tell me how to remove duplicate records from a file? (11 Replies)
Discussion started by: saga20
11 Replies

7. Shell Programming and Scripting

Help with merge and remove duplicates

Hi all, I need some help to remove duplicates from a file before merging. I have got 2 files: file1 has data in format 4300 23456 4301 2357 the 4 byte values on the right hand side is uniq, and are not repeated anywhere in the file file 2 has data in same format but is not in... (10 Replies)
Discussion started by: roy121
10 Replies

8. Shell Programming and Scripting

Remove duplicates

I have a file with the following format: fields seperated by "|" title1|something class|long...content1|keys title2|somhing class|log...content1|kes title1|sothing class|lon...content1|kes title3|shing cls|log...content1|ks I want to remove all duplicates with the same "title field"(the... (3 Replies)
Discussion started by: dtdt
3 Replies

9. Shell Programming and Scripting

Sort and Remove duplicates

Here is my task : I need to sort two input files and remove duplicates in the output files : Sort by 13 characters from 97 Ascending Sort by 1 characters from 96 Ascending If duplicates are found retain the first value in the file the input files are variable length, convert... (4 Replies)
Discussion started by: ysvsr1
4 Replies

10. Shell Programming and Scripting

Remove duplicates

Hi I have a below file structure. 200,1245,E1,1,E1,,7611068,KWH,30, ,,,,,,,, 200,1245,E1,1,E1,,7611070,KWH,30, ,,,,,,,, 300,20140223,0.001,0.001,0.001,0.001,0.001 300,20140224,0.001,0.001,0.001,0.001,0.001 300,20140225,0.001,0.001,0.001,0.001,0.001 300,20140226,0.001,0.001,0.001,0.001,0.001... (1 Reply)
Discussion started by: tejashavele
1 Replies

LEARN ABOUT PHP

httppower

httppower(8)							     powerman							      httppower(8)

NAME

       httppower - communicate with HTTP based power distribution units

SYNOPSIS

       httppower [--url URL]

DESCRIPTION

       httppower  is  a  helper program for powerman which enables it to communicate with HTTP based power distribution units.	It is run interac-
       tively by the powerman daemon.

OPTIONS

       -u, --url URL
	      Set the base URL.

INTERACTIVE COMMANDS

       The following commands are accepted at the httppower> prompt:

       auth user:pass
	      Authenticate to the base URL with specified user and password, using ``basic'' HTTP authentication which sends the user and password
	      over the network in plain text.

       seturl URL
	      Set the base URL.  Overrides the command line option.

       get [URL-suffix]
	      Send an HTTP GET to the base URL with the optional URL-suffix appended.

       post [URL-suffix] key=val[&key=val]...
	      Send an HTTP POST to the base URL with the optional URL-suffix appended, and key-value pairs as argument.

FILES

       /usr/sbin/httppower
       /etc/powerman/powerman.conf

ORIGIN

       PowerMan was originally developed by Andrew Uselton on LLNL's Linux clusters.  This software is open source and distributed under the terms
       of the GNU GPL.

SEE ALSO

       powerman(1), powermand(8), httppower(8), plmpower(8), vpcd(8), powerman.conf(5), powerman.dev(5), powerman-devices(7).

       http://sourceforge.net/projects/powerman

powerman-2.3.5							    2009-02-09							      httppower(8)
All times are GMT -4. The time now is 04:19 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy