Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Getting non unique lines from concatenated files
Top Forums UNIX for Dummies Questions & Answers Getting non unique lines from concatenated files Post 302507757 by bartus11 on Thursday 24th of March 2011 05:27:12 PM
Old 03-24-2011
This is because * is expanded to all the file names before script is executed, so after expansion it looks like: ./script.sh file1 file2 file3 and as you can see there are more arguments passed now. file2 is stored in $2, file3 is in $3 and so on. But there is also variable containing all the arguments passed - $*. You can see below how it can be used to execute that one-liner on all the files:
Code:
#!/bin/sh
for i in $*; do
perl -lane '$a{$F[4]}++;END{for $i (keys %a){print "$i=$a{$i}"}}' $i
done

After modifying script.sh, execute it again with ./script.sh file*
This User Gave Thanks to bartus11 For This Post:
pawannoel
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Lines Concatenated with awk

Hello, I have a bash shell script and I use awk to print certain columns of one file and direct the output to another file. If I do a less or cat on the file it looks correct, but if I email the file and open it with Outlook the lines outputted by awk are concatenated. Here is my awk line:... (6 Replies)
Discussion started by: xadamz23
6 Replies

2. Shell Programming and Scripting

Comparing 2 files and return the unique lines in first file

Hi, I have 2 files file1 ******** 01-05-09|java.xls| 02-05-08|c.txt| 08-01-09|perl.txt| 01-01-09|oracle.txt| ******** file2 ******** 01-02-09|windows.xls| 02-05-08|c.txt| 01-05-09|java.xls| 08-02-09|perl.txt| 01-01-09|oracle.txt| ******** (8 Replies)
Discussion started by: shekhar_v4
8 Replies

3. UNIX for Advanced & Expert Users

In a huge file, Delete duplicate lines leaving unique lines

Hi All, I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space. I dont know if this works : I want to read each line of the File in a For Loop, and want to... (16 Replies)
Discussion started by: krishnix
16 Replies

4. Shell Programming and Scripting

Compare multiple files and print unique lines

Hi friends, I have multiple files. For now, let's say I have two of the following style cat 1.txt cat 2.txt output.txt Please note that my files are not sorted and in the output file I need another extra column that says the file from which it is coming. I have more than 100... (19 Replies)
Discussion started by: jacobs.smith
19 Replies

5. UNIX for Dummies Questions & Answers

getting unique lines from 2 files

hi i have used comm -13 <(sort 1.txt) <(sort 2.txt) option to get the unique lines that are present in file 2 but not in file 1. but some how i am getting the entire file 2. i would expect few but not all uncommon lines fro my dat. is there anything wrong with the way i used the command? my... (1 Reply)
Discussion started by: anurupa777
1 Replies

6. Shell Programming and Scripting

compare 2 files and return unique lines in each file (based on condition)

hi my problem is little complicated one. i have 2 files which appear like this file 1 abbsss:aa:22:34:as akl abc 1234 mkilll:as:ss:23:qs asc abc 0987 mlopii:cd:wq:24:as asd abc 7866 file2 lkoaa:as:24:32:sa alk abc 3245 lkmo:as:34:43:qs qsa abc 0987 kloia:ds:45:56:sa acq abc 7805 i... (5 Replies)
Discussion started by: anurupa777
5 Replies

7. Shell Programming and Scripting

Print only lines where fields concatenated match strings

Hello everyone, Maybe somebody could help me with an awk script. I have this input (field separator is comma ","): 547894982,M|N|J,U|Q|P,98,101,0,1,1 234900027,M|N|J,U|Q|P,98,101,0,1,1 234900023,M|N|J,U|Q|P,98,54,3,1,1 234900028,M|H|J,S|Q|P,98,101,0,1,1 234900030,M|N|J,U|F|P,98,101,0,1,1... (2 Replies)
Discussion started by: Ophiuchus
2 Replies

8. Shell Programming and Scripting

Look up 2 files and print the concatenated output

file 1 Sun Mar 17 00:01:33 2013 submit , Name="1234" Sun Mar 17 00:01:33 2013 submit , Name="1344" Sun Mar 17 00:01:33 2013 submit , Name="1124" .. .. .. .. Sun Mar 17 00:01:33 2013 submit , Name="8901" file 2 Sun Mar 17 00:02:47 2013 1234 execute SUCCEEDED Sun Mar 17... (24 Replies)
Discussion started by: aravindj80
24 Replies

9. UNIX for Dummies Questions & Answers

Print unique lines without sort or unique

I would like to print unique lines without sort or unique. Unfortunately the server I am working on does not have sort or unique. I have not been able to contact the administrator of the server to ask him to add it for several weeks. (7 Replies)
Discussion started by: cokedude
7 Replies

10. UNIX for Beginners Questions & Answers

Print number of lines for files in directory, also print number of unique lines

I have a directory of files, I can show the number of lines in each file and order them from lowest to highest with: wc -l *|sort 15263 Image.txt 16401 reference.txt 40459 richtexteditor.txt How can I also print the number of unique lines in each file? 15263 1401 Image.txt 16401... (15 Replies)
Discussion started by: spacegoose
15 Replies

LEARN ABOUT REDHAT

merge

MERGE(1)						      General Commands Manual							  MERGE(1)

NAME

       merge - three-way file merge

SYNOPSIS

       merge [ options ] file1 file2 file3

DESCRIPTION

       merge  incorporates all changes that lead from file2 to file3 into file1.  The result ordinarily goes into file1.  merge is useful for com-
       bining separate changes to an original.	Suppose file2 is the original, and both file1 and file3 are modifications of  file2.   Then  merge
       combines both changes.

       A  conflict  occurs  if	both  file1 and file3 have changes in a common segment of lines.  If a conflict is found, merge normally outputs a
       warning and brackets the conflict with <<<<<<< and >>>>>>> lines.  A typical conflict will look like this:

	      <<<<<<< file A
	      lines in file A
	      =======
	      lines in file B
	      >>>>>>> file B

       If there are conflicts, the user should edit the result and delete one of the alternatives.

OPTIONS

       -A     Output conflicts using the -A style of diff3(1), if supported by diff3.  This merges all changes leading from file2  to  file3  into
	      file1, and generates the most verbose output.

       -E, -e These  options  specify conflict styles that generate less information than -A.  See diff3(1) for details.  The default is -E.  With
	      -e, merge does not warn about conflicts.

       -L label
	      This option may be given up to three times, and specifies labels to be used in place of the corresponding  file  names  in  conflict
	      reports.	 That  is, merge -L x -L y -L z a b c generates output that looks like it came from files x, y and z instead of from files
	      a, b and c.

       -p     Send results to standard output instead of overwriting file1.

       -q     Quiet; do not warn about conflicts.  -V Print 's version number.

DIAGNOSTICS

       Exit status is 0 for no conflicts, 1 for some conflicts, 2 for trouble.

IDENTIFICATION

       Author: Walter F. Tichy.
       Manual Page Revision: 5.7; Release Date: 1995/06/01.
       Copyright (C) 1982, 1988, 1989 Walter F. Tichy.
       Copyright (C) 1990, 1991, 1992, 1993, 1994, 1995 Paul Eggert.

SEE ALSO

       diff3(1), diff(1), rcsmerge(1), co(1).

BUGS

       It normally does not make sense to merge binary files as if they were text, but merge tries to do it anyway.

GNU
								    1995/06/01								  MERGE(1)
All times are GMT -4. The time now is 06:28 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy