Sponsored Content
Top Forums Shell Programming and Scripting How to avoid duplication within 2 files? Post 302323276 by supercops on Saturday 6th of June 2009 08:18:06 AM
Old 06-06-2009
Quote:
Originally Posted by cfajohnson
If you don't mind the order being re-arranged:

Code:
sort -u file1 file2

If you want to preserve the order:

Code:
awk '!x[$0]++' file1 file2


The first one is okay for me ..but when I am trying the second one I am getting

x[$0]++': Event not found .

Could it be because of permission restrictions on me ?>
 

9 More Discussions You Might Find Interesting

1. Windows & DOS: Issues & Discussions

File Duplication

hi all how to find the file duplication in a windows 2000 server as usual replies are sincerely appreciated. thanks raguram R (3 Replies)
Discussion started by: raguramtgr
3 Replies

2. Ubuntu

Avoid creating temporary files on editing a file in Ubuntu

Hi, My ubuntu flavor always create temporary files having filename followed by ~ on editing. For eg: if I am editing a file called "sip.c", automatically a temporary (bkup) file is getting created with the name "sip.c~". How to avoid this file creation? (7 Replies)
Discussion started by: royalibrahim
7 Replies

3. Programming

What is the proper way to combine C++ files (with g++) to avoid link (ld) errors?

Problem background: gcc v 4.1 2 .cpp files, 2 .h files Files: main.cpp a.cpp a.h b.h Organization: main.cpp includes a.h (because it calls a.cpp code) a.cpp includes a.h a.h includes b.h (because a class in a.h uses a b.h class) There is no inheritance between a.h or b.h or any of... (1 Reply)
Discussion started by: johnqsmith
1 Replies

4. Shell Programming and Scripting

Avoid files being archived

hi all, i want to write a shell script which can automatically touch my all files within a folder in an interval of 90 days ...so that i can avoid them being archived. I don't want to manually touch the all files instead i want an automated shell script to do this. Thanks in advance, Om (3 Replies)
Discussion started by: koti
3 Replies

5. Shell Programming and Scripting

How to Avoid intermediate files when pipe does nt work

problem with piping one output to another.Would like to avoid the intermediate file creation.The piping does nt work on places where files have been created and goes in an endless loop. sed -e "s/^\.\///g" $LINE1| sed -e "s/_\(\)/kkk\1/g" > $file1 tr -s '_' ' ' < $file1| \ sort -n -k... (1 Reply)
Discussion started by: w020637
1 Replies

6. Ubuntu

Avoid typing long filenames in terminal, shortcut for selecting files?

You probably know the answer to this, because I know it exists. I have super long filenames with md5 hashes and I sucks to type the whole hash in the console. Because... just because :P What is the shortcut for selecting a file in the current directory? Like you get a sort of loop through the... (1 Reply)
Discussion started by: hellfire1
1 Replies

7. Shell Programming and Scripting

Avoid locking between two files:

Hi frnz, I have requirement as follows.. There are two Modules say A and B...n both have few set of files..we have one script which read these files and load into database(vertica) table..while running script smtimes i encounter an error like one table is getting loaded from A module file... (7 Replies)
Discussion started by: gnnsprapa
7 Replies

8. Linux

De-Duplication Problem

Hi all, I download and install lessfs for deduplication, I copy files in /SharedFiles directory and lessfs work right and not store again copy files, but, when i delete all files in /SharedFiles , not return free space to total space, files not show in /SharedFiles , but not copy new files in... (3 Replies)
Discussion started by: saeedha
3 Replies

9. AIX

How to avoid errors when moving files in a bff?

I am building a bff using mkinstallp. My template file is : Package Name: svr_exForum Package VRMF: 7.2 Update: N Fileset Fileset Name: svr_exForum.rte Fileset VRMF: 7.2 USRLIBLPPFiles Pre-installation Script: /lppdir/lpp/exForum/F_pre_i ... (2 Replies)
Discussion started by: kevinl
2 Replies
comm(1) 							   User Commands							   comm(1)

NAME
comm - select or reject lines common to two files SYNOPSIS
comm [-123] file1 file2 DESCRIPTION
The comm utility reads file1 and file2, which must be ordered in the current collating sequence, and produces three text columns as output: lines only in file1; lines only in file2; and lines in both files. If the input files were ordered according to the collating sequence of the current locale, the lines written will be in the collating sequence of the original lines. If not, the results are unspecified. OPTIONS
The following options are supported: -1 Suppresses the output column of lines unique to file1. -2 Suppresses the output column of lines unique to file2. -3 Suppresses the output column of lines duplicated in file1 and file2. OPERANDS
The following operands are supported: file1 A path name of the first file to be compared. If file1 is -, the standard input is used. file2 A path name of the second file to be compared. If file2 is -, the standard input is used. USAGE
See largefile(5) for the description of the behavior of comm when encountering files greater than or equal to 2 Gbyte ( 2**31 bytes). EXAMPLES
Example 1: Printing a list of utilities specified by files If file1, file2, and file3 each contain a sorted list of utilities, the command example% comm -23 file1 file2 | comm -23 - file3 prints a list of utilities in file1 not specified by either of the other files. The entry: example% comm -12 file1 file2 | comm -12 - file3 prints a list of utilities specified by all three files. And the entry: example% comm -12 file2 file3 | comm -23 -file1 prints a list of utilities specified by both file2 and file3, but not specified in file1. ENVIRONMENT VARIABLES
See environ(5) for descriptions of the following environment variables that affect the execution of comm: LANG, LC_ALL, LC_COLLATE, LC_CTYPE, LC_MESSAGES, and NLSPATH. EXIT STATUS
The following exit values are returned: 0 All input files were successfully output as specified. >0 An error occurred. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWesu | +-----------------------------+-----------------------------+ |CSI |enabled | +-----------------------------+-----------------------------+ |Interface Stability |Standard | +-----------------------------+-----------------------------+ SEE ALSO
cmp(1), diff(1), sort(1), uniq(1), attributes(5), environ(5), largefile(5), standards(5) SunOS 5.10 3 Mar 2004 comm(1)
All times are GMT -4. The time now is 12:41 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy