Sponsored Content
Full Discussion: Avoid Duplicates in a file
Top Forums UNIX for Dummies Questions & Answers Avoid Duplicates in a file Post 302178074 by lazytech on Monday 24th of March 2008 02:33:06 PM
Old 03-24-2008
My suggestion would be to write a Perl script incorporating Regular Expression to solve your issue. It may be easier to pull out your unique values that way.
 

10 More Discussions You Might Find Interesting

1. Ubuntu

Avoid creating temporary files on editing a file in Ubuntu

Hi, My ubuntu flavor always create temporary files having filename followed by ~ on editing. For eg: if I am editing a file called "sip.c", automatically a temporary (bkup) file is getting created with the name "sip.c~". How to avoid this file creation? (7 Replies)
Discussion started by: royalibrahim
7 Replies

2. Shell Programming and Scripting

How to avoid a temp file

Hi all. I want to check the free space on a given FS and process the output. Right now, I'm using a temp file to avoid using df twice. This is what I'm doing #!/usr/bin/ksh ... df -k $FS_NAME > $TMP_FILE 2>&1 if ]; then RESULT="CRITICAL - $(cat $TMP_FILE)" else cat $TMP_FILE | ...... (3 Replies)
Discussion started by: fox1212
3 Replies

3. HP-UX

Parameter to avoid file being deleted by SAM

Good afternoon. I am a newbie. We just had a potentially big problem (negated to having good backups). Basically, there is an option in SAM, to delete all the data from the system that a user ever created. Lo and behold, silly me, I choose that option, and all sorts of needed files... (5 Replies)
Discussion started by: instant000
5 Replies

4. Shell Programming and Scripting

Avoid file creation in a script...achive same result

Guys following lines help me in getting numbers from PID column ,to be thrown into first column of a CSV file. COLUMNS=2047 /usr/bin/ps -eo pid,ppid,uid,user,args | grep -v "PID" > /tmp/masterPID.txt cat /tmp/masterPID.txt|while read line do PID=`echo $line|awk '{print $1}'` echo "$PID"... (4 Replies)
Discussion started by: ak835
4 Replies

5. Shell Programming and Scripting

avoid open file to check field.

Hi Everyone, # cat a.txt 94,aqqc,62345907, 5,aeec,77, # cat 1.pl #!/usr/bin/perl use strict; use warnings; use Date::Manip; open(my $FA, "/root/a.txt") or die "$!"; while(<$FA>) { chomp; my @tmp=split(/\,/, $_); if (index($tmp, "qq") ne -1) { ... (4 Replies)
Discussion started by: jimmy_y
4 Replies

6. Shell Programming and Scripting

How to avoid Newline character in generated text file?

Hi All, Just need small help in resolving the special new line character in generated output file. In one of my shell script I am using following lines to get the spool file (i.e. sfile.txt) and AAA_XXXX_p111_n222.txt AAA_YYYY_p111_n222.txt Here assuming v_pnum="p111" v_nid="n222" ... (1 Reply)
Discussion started by: shekharjchandra
1 Replies

7. UNIX for Dummies Questions & Answers

CSV file:Find duplicates, save original and duplicate records in a new file

Hi Unix gurus, Maybe it is too much to ask for but please take a moment and help me out. A very humble request to you gurus. I'm new to Unix and I have started learning Unix. I have this project which is way to advanced for me. File format: CSV file File has four columns with no header... (8 Replies)
Discussion started by: arvindosu
8 Replies

8. Programming

[c]Why first file is creating after the second. How to avoid

Hi, My Code is as below: nbECRITS = fwrite(strstr(data->buffer, ";") + 1, sizeof(char), (data->buffsize) - LEN_NOM_FIC, fic_sortie); fclose(fic_sortie); sprintf(PATH_BALISE, "%s.balise", PATH); fic_balise_data = fopen(PATH_BALISE, "a+"); if (fic_balise_data == NULL) {... (1 Reply)
Discussion started by: ezee
1 Replies

9. Shell Programming and Scripting

Avoid overwriting backup file when multiple entries need to replace in one file input from another

Hello, I have been working on script in which search and replace the multiple pattern. 1. update_params.sh read the multiple pattern from input file ParamMapping.txt(old_entry|New_entry) and passing this values one by one to change_text.sh 2. change_text.sh read... (0 Replies)
Discussion started by: ketanraut
0 Replies

10. Shell Programming and Scripting

Write only changes to file - avoid duplicates

I want to create a file, to save a list of fail2ban blocked ip addresses. So I thought I'd create a loop that will check with fail2ban every minute, and write the ip addresses to a file. while true; do echo $(fail2ban-client status asterisk-iptables | grep 'IP list' | sed 's/.*://g' | sed -e... (4 Replies)
Discussion started by: aristosv
4 Replies
Bio::Graphics::Wiggle::Loader(3pm)			User Contributed Perl Documentation			Bio::Graphics::Wiggle::Loader(3pm)

SYNOPSIS
my $loader = Bio::Graphics::Wiggle::Loader->new('/base/directory/for/wigfiles','wibfilename'); my $fh = IO::File->new('uploaded_file.txt'); $loader->load($fh); my $gff3_file = $loader->featurefile('gff3',$method,$source); my $featurefile = $loader->featurefile('featurefile'); USAGE
This module loads Bio::Graphics::Wiggle files from source files that use Jim Kent's "WIG" format: http://genome.ucsc.edu/google/goldenPath/help/wiggle.html Several data sets can be grouped together in a single WIG source file. The load() method accepts the path to a WIG source file, and will create one or more .wib ("wiggle binary") databases of quantitative data in the directory indicated when you created the loader. Call the featurefile() method to return a text file in either GFF3 or Bio::Graphics::FeatureFile format, suitable for loading into a gbrowse database. METHODS $loader = Bio::Graphics::Wiggle::Loader->new('/base/directory' [,'my_data']) Create a new loader. The first argument specifies the base directory in which the loaded .wib files will be created. The second argument specifies the base name for the created .wib files, or "track" if not specified. $loader->load($fh) Load the data from a source WIG file opened on a filehandle. $data = $loader->featurefile($type [,$method,$source]) Return the data corresponding to a GFF3 or Bio::Graphics::FeatureFile. The returned file will have one feature per WIG track, and a properly formatted "wigfile" attribute that directs Bio::Graphics to the location of the quantitative data. $type is one of "gff3" or "featurefile". In the case of "gff3", you may specify an optional method and source for use in describing each feature. In the case of "featurefile", the returned file will contain GBrowse stanzas that describe a reasonable starting format to display the data. $loader->allow_sampling(1) If allow_sampling() is passed a true value, then very large files (more than 5 MB) will undergo a sampling procedure to find their minimum and maximum values and standard deviation. Otherwise, file will be read in its entirety to generate those statistics. EXTENSIONS Several extensions to the WIG format "track" declaration are recognized. transform=<transform> Specify a transform to be performed on all numeric data within this track prior to loading into the binary wig file. Currently, the following three declarations are recognized: transform=logtransform y' = 0 for y == 0 y' = log(y) for y > 0 y' = -log(-y) for y < 0 transform=logsquared y' = log(y**2) for y != 0 y' = 0 for y == 0 transform=none y' = y (no transform - the default) trim=<trim> Specify a trimming function to be performed on the data prior to scaling. Currently, the following trim functions are recognized: trim=stdev1 trim to plus/minus 1 standard deviation of the mean trim=stdev2 trim to plus/minus 2 standard deviations of the mean (default) trim=stdevN trim to plus/minus N standard deviations of the mean trim=none no trimming Example entended track declaration: track type=wiggle_0 name="example" description="20 degrees, 2 hr" trim=stdev2 transform=logsquared SEE ALSO
Bio::Graphics::Wiggle, Bio::Graphics::Glyph::wiggle_xyplot, Bio::Graphics::Glyph::wiggle_density, Bio::Graphics::Panel, Bio::Graphics::Glyph, Bio::Graphics::Feature, Bio::Graphics::FeatureFile AUTHOR
Lincoln Stein <lstein@cshl.org>. Copyright (c) 2007 Cold Spring Harbor Laboratory This package and its accompanying libraries is free software; you can redistribute it and/or modify it under the terms of the GPL (either version 1, or at your option, any later version) or the Artistic License 2.0. Refer to LICENSE for the full license text. In addition, please see DISCLAIMER.txt for disclaimers of warranty. perl v5.14.2 2012-02-20 Bio::Graphics::Wiggle::Loader(3pm)
All times are GMT -4. The time now is 06:26 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy