Sponsored Content
Top Forums Shell Programming and Scripting merge 2 files (without repeating any lines) Post 302139217 by cfajohnson on Friday 5th of October 2007 12:58:08 AM
Old 10-05-2007
Quote:
Originally Posted by bluemoon1
I need to add the content of file1 to file2 - all lines but not those existing in file2 already, so the "cat file1 >> file2" doesn't work.
Code:
awk '!x[$0]++' file2 file1

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Omit repeating lines

Can someone help me with the following 2 objectives? 1) The following command is just an example. It gets a list of all print jobs. From there I am trying to extract the printer name. It works with the following command: lpstat -W "completed" -o | awk -F- '{ print $1}' Problem is, I want... (6 Replies)
Discussion started by: TheCrunge
6 Replies

2. Shell Programming and Scripting

Merging non-repeating columns of lines

Hello, I have file to work with. It has 5 columns. The first three, altogether, constitutes the position. The 4th column contains some values for downstream analysis and the fifth column contains some values that I want to add to 4th column (only if they happen to be in the same position). My... (5 Replies)
Discussion started by: menenuh
5 Replies

3. UNIX for Dummies Questions & Answers

Remove groups of repeating lines

I know uniq exists, but am not sure how to remove repeating lines when they are groups of two different lines repeating themselves, without using sort. I need them to be sorted in the original order, just to remove repeats. cd /media/AUDIO/WAVE/9780743518673/mp3 ~/Desktop/mp3-to-m4b... (1 Reply)
Discussion started by: glev2005
1 Replies

4. Shell Programming and Scripting

remove blank lines and merge lines in shell

Hi, I'm not a expert in shell programming, so i've come here to take help from u gurus. I'm trying to tailor a csv file that i got to make it work for the LOAD FROM command. I've a datatable csv of the below format - --in file format xx,xx,xx ,xx , , , , ,,xx, xxxx,, ,, xxx,... (11 Replies)
Discussion started by: dvah
11 Replies

5. Shell Programming and Scripting

Removing repeating lines from a data frame (AWK)

Hey Guys! I have written a code which combines lots of files into one big file(.csv). However, each of the original files had headers on the first line, and now that I've combined the files the headers are interspersed throughout the new combined data frame. For example, throughout the data... (21 Replies)
Discussion started by: gd9629
21 Replies

6. Shell Programming and Scripting

Printing the lines which are repeating in a files

Hi, I need to find the lines which are repeating in a file cat file1 abcdef 23-1 abcdef 24-1 bcdeff 25-0 ttdcfg 26-0 ttdcfg 20-0 bcdef1 25-0 bcdef2 25-0 bcdef3 25-0 bcdef4 25-0 bcdef4 00-0any help is greatly appreciated. Thanks in advance. In need to find which one are... (3 Replies)
Discussion started by: jpkumar10
3 Replies

7. Shell Programming and Scripting

Compare last 90 logs and print repeating lines with >20

*log files are in date order sample logs... ciscoresets_20120314 ciscoresets_20120313 ciscoresets_20120312 ciscoresets_20120311 ciscoresets_20120310 cat ciscoresets_20120314 SYDGRE04,10,9 SYDGRE04,10,10 SYDGRE04,10,11 SYDGRE04,10,12 SYDGRE04,10,13 SYDGRE04,10,14 SYDGRE04,10,15... (2 Replies)
Discussion started by: slashbash
2 Replies

8. UNIX for Dummies Questions & Answers

Need help combining txt files w/ multiple lines into csv single cell - also need data merge

:confused:Hello -- i just joined the forums. I am a complete noob -- only about 1 week into learning how to program anything... and starting with linux. I am working in Linux terminal. I have a folder with a bunch of txt files. Each file has several lines of html code. I want to combine... (2 Replies)
Discussion started by: jetsetter
2 Replies

9. Shell Programming and Scripting

Deleting Repeating lines from a txt file via script

Hi, I'm having trouble in achieving the following scenario. There is a txt file with thousands of lines and few lines are repeated, which needs to be removed using a script. File.txt 20140522121432,0,12,ram Loc=India From=ram@xxx.com, To=ravi@yyy.com,, 1 2 3 4 . . 30... (18 Replies)
Discussion started by: Gautham
18 Replies

10. Shell Programming and Scripting

Bash repeating lines for some files but not all

The bash below executes and seems to work fine on those files in which . However on those files where there is no additional CNV detected that line repeats multiple times instead of only once. I tried adding an END as all lines are printed but that doesn't help. I can not seem to solve this... (5 Replies)
Discussion started by: cmccabe
5 Replies
MERGE(1)						      General Commands Manual							  MERGE(1)

NAME
merge - three-way file merge SYNOPSIS
merge [ options ] file1 file2 file3 DESCRIPTION
merge incorporates all changes that lead from file2 to file3 into file1. The result ordinarily goes into file1. merge is useful for com- bining separate changes to an original. Suppose file2 is the original, and both file1 and file3 are modifications of file2. Then merge combines both changes. A conflict occurs if both file1 and file3 have changes in a common segment of lines. If a conflict is found, merge normally outputs a warning and brackets the conflict with <<<<<<< and >>>>>>> lines. A typical conflict will look like this: <<<<<<< file A lines in file A ======= lines in file B >>>>>>> file B If there are conflicts, the user should edit the result and delete one of the alternatives. OPTIONS
-A Output conflicts using the -A style of diff3(1), if supported by diff3. This merges all changes leading from file2 to file3 into file1, and generates the most verbose output. -E, -e These options specify conflict styles that generate less information than -A. See diff3(1) for details. The default is -E. With -e, merge does not warn about conflicts. -L label This option may be given up to three times, and specifies labels to be used in place of the corresponding file names in conflict reports. That is, merge -L x -L y -L z a b c generates output that looks like it came from files x, y and z instead of from files a, b and c. -p Send results to standard output instead of overwriting file1. -q Quiet; do not warn about conflicts. -V Print 's version number. DIAGNOSTICS
Exit status is 0 for no conflicts, 1 for some conflicts, 2 for trouble. IDENTIFICATION
Author: Walter F. Tichy. Manual Page Revision: 5.7; Release Date: 1995/06/01. Copyright (C) 1982, 1988, 1989 Walter F. Tichy. Copyright (C) 1990, 1991, 1992, 1993, 1994, 1995 Paul Eggert. SEE ALSO
diff3(1), diff(1), rcsmerge(1), co(1). BUGS
It normally does not make sense to merge binary files as if they were text, but merge tries to do it anyway. GNU
1995/06/01 MERGE(1)
All times are GMT -4. The time now is 06:29 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy