Sponsored Content
Top Forums Shell Programming and Scripting How to Remove duplicate value from file? Post 302770748 by mohan sharma on Monday 18th of February 2013 08:00:28 AM
Old 02-18-2013
but i want to keep all the record of same BIC code with duplicate branch code if XXX is not present among all the branch code of same BIC code..

---------- Post updated at 06:30 PM ---------- Previous update was at 06:10 PM ----------

Thanks zazzybob
but , if i am executing this query its running but output is not coming on the screen ..can u provide me in details or can u give me a query with output should store in file.
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove Duplicate Lines in File

I am doing KSH script to remove duplicate lines in a file. Let say the file has format below. FileA 1253-6856 3101-4011 1827-1356 1822-1157 1822-1157 1000-1410 1000-1410 1822-1231 1822-1231 3101-4011 1822-1157 1822-1231 and I want to simply it with no duplicate line as file... (5 Replies)
Discussion started by: Teh Tiack Ein
5 Replies

2. UNIX for Dummies Questions & Answers

Remove Duplicate lines from File

I have a log file "logreport" that contains several lines as seen below: 04:20:00 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping 06:38:08 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping 07:11:05 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but... (18 Replies)
Discussion started by: Nysif Steve
18 Replies

3. Shell Programming and Scripting

Sort and Remove Duplicate on file

How do we sort and remove duplicate on column 1,2 retaining the record with maximum date (in feild 3) for the file with following format. aaa|1234|2010-12-31 aaa|1234|2010-11-10 bbb|345|2011-01-01 ccc|346|2011-02-01 bbb|345|2011-03-10 aaa|1234|2010-01-01 Required Output ... (5 Replies)
Discussion started by: mabarif16
5 Replies

4. Shell Programming and Scripting

Formatting a file - Remove Duplicate

Hi I have a file in the following format. Basically the file contains tablename and their aliases: TABLE1 TABLE1 A TABLE2 TABLE2 B TABLE3 TABLE4 TABLE4 C TABLE4 Upon formatting an sql statement I am getting such output. Problem: Whenever a tablename appears with alias, it has... (5 Replies)
Discussion started by: freakygs
5 Replies

5. Shell Programming and Scripting

remove of duplicate line from a file

I have a file a.txt having content like deepak ram sham deepram sita kumar I Want to delete the first line containing "deep" ... I tried using... grep -i 'deep' a.txt It gives me 2 rows...I want to delete the first one.. + need to know the command to delete the line from... (5 Replies)
Discussion started by: saluja.deepak
5 Replies

6. Shell Programming and Scripting

How do I remove the duplicate lines in this file?

Hey guys, need some help to fix this script. I am trying to remove all the duplicate lines in this file. I wrote the following script, but does not work. What is the problem? The output file should only contain five lines: Later! (5 Replies)
Discussion started by: Ernst
5 Replies

7. Shell Programming and Scripting

Remove the duplicate content in a file

Here is the contents of test.txt Dependencies Resolved Changes in packages about to be updated: ChangeLog for: 1:perl-Archive-Extract-0.38-131.el6_4.x86_64, - Resolves: #915692 - CVE-2013-1667 (DoS in rehashing code) Dependencies Resolved Changes in packages about to be updated: ... (5 Replies)
Discussion started by: ashokvpp
5 Replies

8. Shell Programming and Scripting

Remove duplicate lines from a file

Hi, I have a csv file which contains some millions of lines in it. The first line(Header) repeats at every 50000th line. I want to remove all the duplicate headers from the second occurance(should not remove the first line). I don't want to use any pattern from the Header as I have some... (7 Replies)
Discussion started by: sudhakar T
7 Replies

9. Shell Programming and Scripting

Remove duplicate values in a column(not in the file)

Hi Gurus, I have a file(weblog) as below abc|xyz|123|agentcode=sample code abcdeeess,agentcode=sample code abcdeeess,agentcode=sample code abcdeeess|agentadd=abcd stereet 23343,agentadd=abcd stereet 23343 sss|wwq|999|agentcode=sample1 code wqwdeeess,gentcode=sample1 code... (4 Replies)
Discussion started by: ratheeshjulk
4 Replies
OBNAM-BENCHMARK(1)					      General Commands Manual						OBNAM-BENCHMARK(1)

NAME
obnam-benchmark - benchmark obnam SYNOPSIS
obnam-benchmark [--config=FILE] [--description=DESCRIPTION] [--drop-caches] [--dump-config] [--dump-memory-profile=METHOD] [--dump-setting-names] [--file-size=SIZE] [--generate-manpage=TEMPLATE] [--generations=N] [-h] [--help] [--larch-branch=DIR] [--list-config-files] [--log=FILE] [--log-keep=N] [--log-level=LEVEL] [--log-max=SIZE] [--log-mode=MODE] [--no-default-configs] [--obnam-branch=DIR] [--output=FILE] [--profile-name=PROFILE-NAME] [--results=DIR] [--seivot-branch=DIR] [--seivot-log=SEIVOT-LOG] [--sftp-delay=SFTP-DELAY] [--size=PAIR] [--use-sftp-repository] [--use-sftp-root] [--verify] [--version] [--with-encryption] [FILE]... DESCRIPTION
obnam-benchmark benchmarks the obnam(1) backup application, by measuring how much time it takes to do a backup, restore, etc, in various scenarios. obnam-benchmark uses the seivot(1) tool for actually running the benchmarks, but makes some helpful assumptions about things, to make it simpler to run than running seivot directly. Benchmarks are run using two different usage profiles: mailspool (all files are small), and mediaserver (all files are big). For each pro- file, test data of the desired total size is generated, backed up, and then several incremental generations are backed up, each adding some more generated test data. Then other operations are run against the backup repository: restoring, listing the contents of, and removing each generation. The result of the benchmark is a .seivot file per profile, plus a Python profiler file for each run of obnam. These are stored in ../benchmarks. A set of .seivot files can be summarized for comparison with seivots-summary(1). The profiling files can be viewed with the usual Python tools: see the pstats module. The benchmarks are run against a version of obnam checked out from version control. It is not (currently) possible to run the benchmark against an installed version of obnam. Also the larch Python library, which obnam needs, needs to be checked out from version control. The --obnam-branch and --larch-branch options set the locations, if the defaults are not correct. OPTIONS
--config=FILE add FILE to config files --description=DESCRIPTION describe benchmark --drop-caches drop kernel buffer caches --dump-config write out the entire current configuration --dump-memory-profile=METHOD make memory profiling dumps using METHOD, which is one of: none, simple, meliae, or heapy (default: simple) --dump-setting-names write out all names of settings and quit --file-size=SIZE how big should files be? --generate-manpage=TEMPLATE fill in manual page TEMPLATE --generations=N benchmark N generations (default: 5) -h, --help show this help message and exit --larch-branch=DIR use DIR as the larch branch (default: /sbuild-nonexistent/larch/trunk) --list-config-files list all possible config files --log=FILE write log entries to FILE (default is to not write log files at all); use "syslog" to log to system log, or "none" to disable log- ging --log-keep=N keep last N logs (10) --log-level=LEVEL log at LEVEL, one of debug, info, warning, error, critical, fatal (default: debug) --log-max=SIZE rotate logs larger than SIZE, zero for never (default: 0) --log-mode=MODE set permissions of new log files to MODE (octal; default 0600) --no-default-configs clear list of configuration files to read --obnam-branch=DIR use DIR as the obnam branch to benchmark (default: .) --output=FILE write output to FILE, instead of standard output --profile-name=PROFILE-NAME short name for benchmark scenario --results=DIR put results under DIR (../benchmarks) --seivot-branch=DIR use DIR as the seivot branch (default: installed seivot) --seivot-log=SEIVOT-LOG seivot log setting --sftp-delay=SFTP-DELAY add artifical delay to sftp transfers (in milliseconds) --size=PAIR add PAIR to list of sizes to benchmark (e.g., 10g/1m) --use-sftp-repository access the repository over SFTP (requires ssh to localhost to work) --use-sftp-root access the live data over SFTP (requires ssh to localhost to work) --verify verify restores --version show program's version number and exit --with-encryption run benchmark using encryption ENVIRONMENT
TMPDIR This variable must be set. It controls where the temporary files (generated test data) is stored. If this variable was not set, they'd be put into /tmp, which easily fills up, to the detriment of the entire system. Thus. obnam-benchmark requires that the lo- cation is set explicitly. (You can still use /tmp if you want, but you have to set TMPDIR explicitly.) FILES
../benchmarks/ The default directory where results of the benchmark are stored, in a subdirectory named after the branch and revision numbers. EXAMPLE
To run a small benchmark: TMPDIR=/var/tmp obnam-benchmark --size=10m/1m To run a benchmark using existing data: TMPDIR=/var/tmp obnam-benchmark --use-existing=$HOME/Mail To view the currently available benchmark results: seivots-summary ../benchmarks/*/*mail*.seivot | less -S seivots-summary ../benchmarks/*/*media*.seivot | less -S (You need to run seivots-summary once per usage profile.) SEE ALSO
obnam(1), seivot(1), seivots-summary(1). OBNAM-BENCHMARK(1)
All times are GMT -4. The time now is 06:35 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy