11-02-2016
It seems like a really inefficient method of removing duplicates. Therefore it must be homework... ;-)
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I have searched the FAQ - by using sort, duplicates, etc.... but I didn't get any articles or results on it.
Currently, I am using:
sort -u file1 > file2 to remove duplicates. For a file size of 1giga byte approx. time taken to remove duplicates is 1hr 21 mins.
Is there any other faster way... (15 Replies)
Discussion started by: radhika
15 Replies
2. Shell Programming and Scripting
Hello Experts,
I have two files named old and new. Below are my example files. I need to compare and print the records that only exist in my new file. I tried the below awk script, this script works perfectly well if the records have exact match, the issue I have is my old file has got extra... (4 Replies)
Discussion started by: forumthreads
4 Replies
3. Shell Programming and Scripting
Hi
I need a script that removes the duplicate records and write it to a new file
for example I have a file named test.txt and it looks like
abcd.23
abcd.24
abcd.25
qwer.25
qwer.26
qwer.98
I want to pick only $1 and compare with the next record and the output should be
abcd.23... (6 Replies)
Discussion started by: antointoronto
6 Replies
4. Shell Programming and Scripting
Hi,
I'm using the below command to sort and remove duplicates in a file. But, i need to make this applied to the same file instead of directing it to another.
Thanks (6 Replies)
Discussion started by: dvah
6 Replies
5. Shell Programming and Scripting
I need to use a bash script to remove duplicate files from a download list, but I cannot use uniq because the urls are different.
I need to go from this:
http://***/fae78fe/file1.wmv
http://***/39du7si/file1.wmv
http://***/d8el2hd/file2.wmv
http://***/h893js3/file2.wmv
to this:
... (2 Replies)
Discussion started by: locoroco
2 Replies
6. UNIX for Dummies Questions & Answers
Can u tell me how to remove duplicate records from a file? (11 Replies)
Discussion started by: saga20
11 Replies
7. Shell Programming and Scripting
Hi all,
I need some help to remove duplicates from a file before merging.
I have got 2 files:
file1 has data in format
4300 23456
4301 2357
the 4 byte values on the right hand side is uniq, and are not repeated anywhere in the file
file 2 has data in same format but is not in... (10 Replies)
Discussion started by: roy121
10 Replies
8. Shell Programming and Scripting
I have a file with the following format:
fields seperated by "|"
title1|something class|long...content1|keys
title2|somhing class|log...content1|kes
title1|sothing class|lon...content1|kes
title3|shing cls|log...content1|ks
I want to remove all duplicates with the same "title field"(the... (3 Replies)
Discussion started by: dtdt
3 Replies
9. Shell Programming and Scripting
Here is my task :
I need to sort two input files and remove duplicates in the output files :
Sort by 13 characters from 97 Ascending
Sort by 1 characters from 96 Ascending
If duplicates are found retain the first value in the file
the input files are variable length, convert... (4 Replies)
Discussion started by: ysvsr1
4 Replies
10. Shell Programming and Scripting
Hi I have a below file structure.
200,1245,E1,1,E1,,7611068,KWH,30, ,,,,,,,,
200,1245,E1,1,E1,,7611070,KWH,30, ,,,,,,,,
300,20140223,0.001,0.001,0.001,0.001,0.001
300,20140224,0.001,0.001,0.001,0.001,0.001
300,20140225,0.001,0.001,0.001,0.001,0.001
300,20140226,0.001,0.001,0.001,0.001,0.001... (1 Reply)
Discussion started by: tejashavele
1 Replies
LEARN ABOUT DEBIAN
hdf_set_valuef
hdf_set_valuef(3) util/neo_hdf.h hdf_set_valuef(3)
NAME
hdf_set_valuef - Set the value of a named node
SYNOPSIS
#include <util/neo_hdf.h>
NEOERR* hdf_set_valuef (HDF *hdf, const char *fmt, ...)
ATTRIBUTE_PRINTF(2,3);
ARGUMENTS
hdf -> the pointer to the hdf dataset
fmt -> the name=value printf(3) format string
DESCRIPTION
hdf_set_valuef is a convenience function that wraps hdf_set_value. Due to limitations of C, the fmt is in the format "name=value", where
we will first format the entire string, and then break it at the first (from the left) equal sign (=) and use the left portion as the name
and the right portion as the value. This function is somewhat inefficient in that it first allocates the full name=value, and then the
call to hdf_set_value duplicates the value portion, and then we free the name=value. Currently, we don't strip whitespace from the key or
value. In the future, this function might work more like reading a single line of an HDF string or file, allowing for attributes and sym-
links to be specified... maybe.
RETURN VALUE
None
SEE ALSO
hdf_set_symlink(3), hdf_get_valuef(3), compareFunc)(3), hdf_dump_str(3), hdf_get_int_value(3), hdf_read_string_ignore(3), hdf_copy(3),
hdf_get_node(3), hdf_set_value(3), hdf_obj_child(3), hdf_read_string(3), hdf_set_buf(3), hdf_register_fileload(3), hdf_get_obj(3),
hdf_get_attr(3), hdf_obj_top(3), hdf_dump_format(3), hdf_destroy(3), hdf_init(3), hdf_obj_value(3), hdf_write_string(3), hdf_set_val-
uevf(3), hdf_sort_obj(3), hdf_write_file_atomic(3), hdf_obj_attr(3), hdf_set_valuef(3), hdf_get_copy(3), hdf_write_file(3), hdf_get_val-
uevf(3), hdf_set_int_value(3), hdf_dump(3), hdf_get_value(3), hdf_set_copy(3), hdf_obj_next(3), hdf_search_path(3), hdf_obj_name(3),
hdf_set_attr(3), hdf_get_child(3), hdf_remove_tree
ClearSilver 12 July 2007 hdf_set_valuef(3)