Sponsored Content
Full Discussion: Remove dupes in a large file
Top Forums Shell Programming and Scripting Remove dupes in a large file Post 303024642 by RudiC on Saturday 13th of October 2018 04:31:28 PM
Old 10-13-2018
Hi MadeInGermany,


mind to explain that approach? Is that because X[$0]++ becomes a number and consumes a "float" 's space, whereas X[$0] has just an index but points to nowhere?
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

remove a large number of user from oracle

Hi on solaris and oracle 10g2, I have number of users created in Oracle, I wonder if I have a list of the usernames will it be possible to remove the users quickly ? I want to keep the users access to system but oracle. some thing like shell script may be ?:confused: I am trying to... (4 Replies)
Discussion started by: upengan78
4 Replies

2. Shell Programming and Scripting

Sed or awk script to remove text / or perform calculations from large CSV files

I have a large CSV files (e.g. 2 million records) and am hoping to do one of two things. I have been trying to use awk and sed but am a newbie and can't figure out how to get it to work. Any help you could offer would be greatly appreciated - I'm stuck trying to remove the colon and wildcards in... (6 Replies)
Discussion started by: metronomadic
6 Replies

3. Shell Programming and Scripting

remove a specific line in a LARGE file

Hi guys, i have a really big file, and i want to remove a specific line. sed -i '5d' fileThis doesn't really work, it takes a lot of time... The whole script is supposed to remove every word containing less than 5 characters and currently looks like this: #!/bin/bash line="1"... (2 Replies)
Discussion started by: blubbiblubbkekz
2 Replies

4. Shell Programming and Scripting

Remove Duplicate Filenames in 2 very large directories

Hello Gurus, O/S RHEL4 I have a requirement to compare two linux based directories for duplicate filenames and remove them. These directories are close to 2 TB each. I have tried running a: Prompt>diff -r data1/ data2/ I have tried this as well: jason@jason-desktop:~$ cat script.sh ... (7 Replies)
Discussion started by: jaysunn
7 Replies

5. Shell Programming and Scripting

How to remove a subset of data from a large dataset based on values on one line

Hello. I was wondering if anyone could help. I have a file containing a large table in the format: marker1 marker2 marker3 marker4 position1 position2 position3 position4 genotype1 genotype2 genotype3 genotype4 with marker being a name, position a numeric... (2 Replies)
Discussion started by: davegen
2 Replies

6. UNIX for Dummies Questions & Answers

Filtering F-Dupes

Is there an easy way to tell FDupes what filetypes to look at or ignore? (0 Replies)
Discussion started by: furashgf
0 Replies

7. Shell Programming and Scripting

Removing Dupes from huge file- awk/perl/uniq

Hi, I have the following command in place nawk -F, '!a++' file > file.uniq It has been working perfectly as per requirements, by removing duplicates by taking into consideration only first 3 fields. Recently it has started giving below error: bash-3.2$ nawk -F, '!a++'... (17 Replies)
Discussion started by: makn
17 Replies

8. Shell Programming and Scripting

remove large portion of web page code between two tags

Hi everybody, I am trying to remove bunch of lines from web pages between two tags: one is <h1> and the other is <table it looks like <h1>Anniversary cards roses</h1> many lines here <table summary="Free anniversary greeting cards." cellspacing="8" cellpadding="8" width="70%">my goal... (5 Replies)
Discussion started by: georgi58
5 Replies

9. Shell Programming and Scripting

Removing dupes within 2 delimited areas in a large dictionary file

Hello, I have a very large dictionary file which is in text format and which contains a large number of sub-sections. Each sub-section starts with the following header : #DATA #VALID 1 and ends with a footer as shown below #END The data between the Header and the Footer consists of... (6 Replies)
Discussion started by: gimley
6 Replies

10. Shell Programming and Scripting

Modify script to remove dupes with two delimiters

Hello, I have a script which removes duplicates in a database with a single delimiter = The script is given below: # script to remove dupes from a row with structure word=word BEGIN{FS="="} {for(i=1;i<=NF;i++){a++;}for(i in a){b=b"="i}{sub("=","",b);$0=b;b="";delete a}}1 How do I modify... (6 Replies)
Discussion started by: gimley
6 Replies
DH_INSTALLINIT(1)						     Debhelper							 DH_INSTALLINIT(1)

NAME
dh_installinit - install upstart jobs or init scripts into package build directories SYNOPSIS
dh_installinit [debhelperoptions] [--name=name] [-n] [-R] [-r] [-d] [-O] [--params] DESCRIPTION
dh_installinit is a debhelper program that is responsible for installing upstart job files or init scripts with associated defaults files into package build directories, and in the former case providing compatibility handling for non-upstart systems. It also automatically generates the postinst and postrm and prerm commands needed to set up the symlinks in /etc/rc*.d/ and to start and stop the init scripts. FILES
debian/package.upstart If this exists, it is installed into etc/init/package.conf in the package build directory. debian/package.init Otherwise, if this exists, it is installed into etc/init.d/package in the package build directory. debian/package.default If this exists, it is installed into etc/default/package in the package build directory. OPTIONS
-n, --noscripts Do not modify postinst/postrm/prerm scripts. -o, --onlyscripts Only modify postinst/postrm/prerm scripts, do not actually install any init script, default files, or upstart job. May be useful if the init script or upstart job is shipped and/or installed by upstream in a way that doesn't make it easy to let dh_installinit find it. If no upstart job file is installed in the target directory when dh_installinit --onlyscripts is called, this program will assume that an init script is being installed and not provide the compatibility symlinks or upstart dependencies. -R, --restart-after-upgrade Do not stop the init script until after the package upgrade has been completed. This is different than the default behavior, which stops the script in the prerm, and starts it again in the postinst. This can be useful for daemons that should not have a possibly long downtime during upgrade. But you should make sure that the daemon will not get confused by the package being upgraded while it's running before using this option. -r, --no-restart-on-upgrade Do not stop init script on upgrade. --no-start Do not start the init script on install or upgrade, or stop it on removal. Only call update-rc.d. Useful for rcS scripts. -d, --remove-d Remove trailing d from the name of the package, and use the result for the filename the upstart job file is installed as in etc/init/ , or for the filename the init script is installed as in etc/init.d and the default file is installed as in etc/default/ . This may be useful for daemons with names ending in d. (Note: this takes precedence over the --init-script parameter described below.) -uparams --update-rcd-params=params -- params Pass params to update-rc.d(8). If not specified, defaults will be passed to update-rc.d(8). --name=name Install the upstart job file or the init script (and default file) using the filename name instead of the default filename, which is the package name. When this parameter is used, dh_installinit looks for and installs files named debian/package.name.upstart, debian/package.name.init and debian/package.name.default, instead of the usual debian/package.upstart, debian/package.init and debian/package.default. --init-script=scriptname Use scriptname as the filename the init script is installed as in etc/init.d/ (and also use it as the filename for the defaults file, if it is installed). If you use this parameter, dh_installinit will look to see if a file in the debian/ directory exists that looks like package.scriptname and if so will install it as the init script in preference to the files it normally installs. This parameter is deprecated, use the --name parameter instead. This parameter will be ignored completely for upstart jobs. --upstart-only Only install an upstart job file, and do not include maintainer script code to replace an init script with that upstart job. This parameter is intended for use when the "package.upstart" file is new and only to be used on Upstart-based systems. --error-handler=function Call the named shell function if running the init script fails. The function should be provided in the prerm and postinst scripts, before the #DEBHELPER# token. NOTES
Note that this command is not idempotent. dh_prep(1) should be called between invocations of this command. Otherwise, it may cause multiple instances of the same text to be added to maintainer scripts. SEE ALSO
debhelper(7) This program is a part of debhelper. AUTHORS
Joey Hess <joeyh@debian.org> Steve Langasek <steve.langasek@canonical.com> Scott James Remnant <scott@canonical.com> 8.9.0ubuntu2.1 2012-06-12 DH_INSTALLINIT(1)
All times are GMT -4. The time now is 02:22 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy