Sponsored Content
Top Forums Shell Programming and Scripting How to remove duplicates using for loop? Post 302984892 by gandolf989 on Wednesday 2nd of November 2016 10:09:38 AM
Old 11-02-2016
It seems like a really inefficient method of removing duplicates. Therefore it must be homework... ;-)
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

fastest way to remove duplicates.

I have searched the FAQ - by using sort, duplicates, etc.... but I didn't get any articles or results on it. Currently, I am using: sort -u file1 > file2 to remove duplicates. For a file size of 1giga byte approx. time taken to remove duplicates is 1hr 21 mins. Is there any other faster way... (15 Replies)
Discussion started by: radhika
15 Replies

2. Shell Programming and Scripting

Remove duplicates

Hello Experts, I have two files named old and new. Below are my example files. I need to compare and print the records that only exist in my new file. I tried the below awk script, this script works perfectly well if the records have exact match, the issue I have is my old file has got extra... (4 Replies)
Discussion started by: forumthreads
4 Replies

3. Shell Programming and Scripting

Script to remove duplicates

Hi I need a script that removes the duplicate records and write it to a new file for example I have a file named test.txt and it looks like abcd.23 abcd.24 abcd.25 qwer.25 qwer.26 qwer.98 I want to pick only $1 and compare with the next record and the output should be abcd.23... (6 Replies)
Discussion started by: antointoronto
6 Replies

4. Shell Programming and Scripting

remove duplicates and sort

Hi, I'm using the below command to sort and remove duplicates in a file. But, i need to make this applied to the same file instead of directing it to another. Thanks (6 Replies)
Discussion started by: dvah
6 Replies

5. Shell Programming and Scripting

bash - remove duplicates

I need to use a bash script to remove duplicate files from a download list, but I cannot use uniq because the urls are different. I need to go from this: http://***/fae78fe/file1.wmv http://***/39du7si/file1.wmv http://***/d8el2hd/file2.wmv http://***/h893js3/file2.wmv to this: ... (2 Replies)
Discussion started by: locoroco
2 Replies

6. UNIX for Dummies Questions & Answers

Remove duplicates from a file

Can u tell me how to remove duplicate records from a file? (11 Replies)
Discussion started by: saga20
11 Replies

7. Shell Programming and Scripting

Help with merge and remove duplicates

Hi all, I need some help to remove duplicates from a file before merging. I have got 2 files: file1 has data in format 4300 23456 4301 2357 the 4 byte values on the right hand side is uniq, and are not repeated anywhere in the file file 2 has data in same format but is not in... (10 Replies)
Discussion started by: roy121
10 Replies

8. Shell Programming and Scripting

Remove duplicates

I have a file with the following format: fields seperated by "|" title1|something class|long...content1|keys title2|somhing class|log...content1|kes title1|sothing class|lon...content1|kes title3|shing cls|log...content1|ks I want to remove all duplicates with the same "title field"(the... (3 Replies)
Discussion started by: dtdt
3 Replies

9. Shell Programming and Scripting

Sort and Remove duplicates

Here is my task : I need to sort two input files and remove duplicates in the output files : Sort by 13 characters from 97 Ascending Sort by 1 characters from 96 Ascending If duplicates are found retain the first value in the file the input files are variable length, convert... (4 Replies)
Discussion started by: ysvsr1
4 Replies

10. Shell Programming and Scripting

Remove duplicates

Hi I have a below file structure. 200,1245,E1,1,E1,,7611068,KWH,30, ,,,,,,,, 200,1245,E1,1,E1,,7611070,KWH,30, ,,,,,,,, 300,20140223,0.001,0.001,0.001,0.001,0.001 300,20140224,0.001,0.001,0.001,0.001,0.001 300,20140225,0.001,0.001,0.001,0.001,0.001 300,20140226,0.001,0.001,0.001,0.001,0.001... (1 Reply)
Discussion started by: tejashavele
1 Replies
hdf_set_valuef(3)						  util/neo_hdf.h						 hdf_set_valuef(3)

NAME
hdf_set_valuef - Set the value of a named node SYNOPSIS
#include <util/neo_hdf.h> NEOERR* hdf_set_valuef (HDF *hdf, const char *fmt, ...) ATTRIBUTE_PRINTF(2,3); ARGUMENTS
hdf -> the pointer to the hdf dataset fmt -> the name=value printf(3) format string DESCRIPTION
hdf_set_valuef is a convenience function that wraps hdf_set_value. Due to limitations of C, the fmt is in the format "name=value", where we will first format the entire string, and then break it at the first (from the left) equal sign (=) and use the left portion as the name and the right portion as the value. This function is somewhat inefficient in that it first allocates the full name=value, and then the call to hdf_set_value duplicates the value portion, and then we free the name=value. Currently, we don't strip whitespace from the key or value. In the future, this function might work more like reading a single line of an HDF string or file, allowing for attributes and sym- links to be specified... maybe. RETURN VALUE
None SEE ALSO
hdf_set_symlink(3), hdf_get_valuef(3), compareFunc)(3), hdf_dump_str(3), hdf_get_int_value(3), hdf_read_string_ignore(3), hdf_copy(3), hdf_get_node(3), hdf_set_value(3), hdf_obj_child(3), hdf_read_string(3), hdf_set_buf(3), hdf_register_fileload(3), hdf_get_obj(3), hdf_get_attr(3), hdf_obj_top(3), hdf_dump_format(3), hdf_destroy(3), hdf_init(3), hdf_obj_value(3), hdf_write_string(3), hdf_set_val- uevf(3), hdf_sort_obj(3), hdf_write_file_atomic(3), hdf_obj_attr(3), hdf_set_valuef(3), hdf_get_copy(3), hdf_write_file(3), hdf_get_val- uevf(3), hdf_set_int_value(3), hdf_dump(3), hdf_get_value(3), hdf_set_copy(3), hdf_obj_next(3), hdf_search_path(3), hdf_obj_name(3), hdf_set_attr(3), hdf_get_child(3), hdf_remove_tree ClearSilver 12 July 2007 hdf_set_valuef(3)
All times are GMT -4. The time now is 05:57 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy