Sponsored Content
Top Forums Shell Programming and Scripting Script to remove same content from other file Post 302850917 by Mr_47 on Friday 6th of September 2013 03:55:35 AM
Old 09-06-2013
Script to remove same content from other file

Hi/ Hello all Guru here,

I am trying to create script to remove same content from other file, already tested few idea and found that in unix it is limited to sort and uniq. There is many script for removing duplicate content however to delete all same content is non. Need your help and guide . thx

so here the situation

1. a.out a file with content that we need to delete in other file
Code:
more a.out
a
b
c
d


2. b.out a file that

Code:
more b.out
a
a
a
b
b
c
d
d
e
f


3. the result should be like c.out file

Code:
more c.out
e
f


note: the data is around 1.5 million list.

---------- Post updated at 03:55 PM ---------- Previous update was at 03:53 PM ----------

tested and not working and most only to delete duplication and im lost

Code:
sed -e "s/Text_1/TextA/" -e "s/Text1/TextB/" <your_file.txt>your_file_new.txt

cat deleteme.txt deleteme.txt masterlist.txt | sort | uniq -u > newmasterlist.txt

diff file-a file-b --new-line-format="" --old-line-format="%L" --unchanged-line-format="" > file-a

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove content from file

hey all, I have a file with records in following format 8-29-2006 13:01:45|ABC|45 8-29-2006 14:23:12|DEF|21 8-30-2006 00:04:57|ABC|34 I want to remove all yesterday records. Can anyone show me how? Thanks! (10 Replies)
Discussion started by: mpang_
10 Replies

2. Shell Programming and Scripting

remove space from file content

i am a bit new to shell scripting i have a file containing xxxx xx xx but i want to output the content as xxxxxxxx. thus removing the space. any idea how i can do this (4 Replies)
Discussion started by: blackzinga
4 Replies

3. Shell Programming and Scripting

Remove specific content in a file

Hi, I have a file called fl_list consists of files i have to archive. I want to create a exception parm called except_parm, so if it finds the directory it will not archive these files and remove from fl_list. $ cat fl_list /apps/dev/ihub/ready/IA003B/IA003B_Deal_Header_yyyymmdd_hhmmss.txt... (1 Reply)
Discussion started by: k9cheung
1 Replies

4. Shell Programming and Scripting

Help with remove duplicate content and only keep the first content detail

Input data_10 SSA data_2 TYUE data_3 PEOCV data_6 SSAT data_21 SSA data_19 TYUEC data_14 TYUE data_15 SSA data_32 PEOCV . . Desired Output data_10 SSA data_2 TYUE data_3 PEOCV data_6 SSAT data_19 TYUEC (9 Replies)
Discussion started by: patrick87
9 Replies

5. Shell Programming and Scripting

Shell script to remove some content in a file

How can I remove all data that contain domain e.g zzgh@something.com, sdd@something.com.my and gg@something.my in one file? so that i only have data without the domain in the file. Here is the file structure "test.out" more test.out 1 zzztop@b.com 1 zzzulll 1 zzzullll@s.com.my ... (4 Replies)
Discussion started by: Mr_47
4 Replies

6. Shell Programming and Scripting

Remove the file content based on the Header of the file

Hi All, I want to remove the content based on the header information . Please find the example below. File1.txt Name|Last|First|Location|DepId|Depname|DepLoc naga|rr|tion|hyd|1|wer|opr Nava|ra|tin|gen|2|wera|opra I have to search for the DepId and remove the data from the... (5 Replies)
Discussion started by: i150371485
5 Replies

7. Shell Programming and Scripting

Remove some content from a verilog file

Hi, I have a verilog file which looks like module xyz (x, y, z, a, b, c); input x; input y; input z; output a; output b; output c; initial begin ... end always ... ... endmodule (7 Replies)
Discussion started by: matbrow
7 Replies

8. Shell Programming and Scripting

Shell script to monitor new file in a directory and mail the file content

Hi I am looking for a help in designing a bash script on linux which can do below:- 1) Look in a specific directory for any new files 2) Mail the content of the new file Appreciate any help Regards Neha (5 Replies)
Discussion started by: neha0785
5 Replies

9. Shell Programming and Scripting

Remove the duplicate content in a file

Here is the contents of test.txt Dependencies Resolved Changes in packages about to be updated: ChangeLog for: 1:perl-Archive-Extract-0.38-131.el6_4.x86_64, - Resolves: #915692 - CVE-2013-1667 (DoS in rehashing code) Dependencies Resolved Changes in packages about to be updated: ... (5 Replies)
Discussion started by: ashokvpp
5 Replies

10. Shell Programming and Scripting

How to remove exisiting file content from a file and have to append new file content?

hi all, i had the below script x=`cat input.txt |wc -1` awk 'NR>1 && NR<'$x' ' input.txt > output.txt by using above script i am able to remove the head and tail part from the input file and able to append the output to the output.txt but if i run it for second time the output is... (2 Replies)
Discussion started by: hemanthsaikumar
2 Replies
Test::Script(3) 					User Contributed Perl Documentation					   Test::Script(3)

NAME
Test::Script - Basic cross-platform tests for scripts DESCRIPTION
The intent of this module is to provide a series of basic tests for 80% of the testing you will need to do for scripts in the script (or bin as is also commonly used) paths of your Perl distribution. Further, it aims to provide this functionality with perfect platform-compatibility, and in a way that is as unobtrusive as possible. That is, if the program works on a platform, then Test::Script should always work on that platform as well. Anything less than 100% is considered unacceptable. In doing so, it is hoped that Test::Script can become a module that you can safely make a dependency of all your modules, without risking that your module won't on some platform because of the dependency. Where a clash exists between wanting more functionality and maintaining platform safety, this module will err on the side of platform safety. FUNCTIONS
script_compiles script_compiles( 'script/foo.pl', 'Main script compiles' ); The "script_compiles" test calls the script with "perl -c script.pl", and checks that it returns without error. The path it should be passed is a relative unix-format script name. This will be localised when running "perl -c" and if the test fails the local name used will be shown in the diagnostic output. Note also that the test will be run with the same perl interpreter that is running the test script (and not with the default system perl). This will also be shown in the diagnostic output on failure. script_runs script_runs( 'script/foo.pl', 'Main script runs' ); The "script_runs" test executes the script with "perl script.pl" and checks that it returns success. The path it should be passed is a relative unix-format script name. This will be localised when running "perl -c" and if the test fails the local name used will be shown in the diagnostic output. The test will be run with the same perl interpreter that is running the test script (and not with the default system perl). This will also be shown in the diagnostic output on failure. SUPPORT
All bugs should be filed via the bug tracker at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Test-Script <http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Test-Script> For other issues, or commercial enhancement and support, contact the author. AUTHOR
Adam Kennedy <adamk@cpan.org> SEE ALSO
prove, <http://ali.as/> COPYRIGHT
Copyright 2006 - 2009 Adam Kennedy. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. The full text of the license can be found in the LICENSE file included with this module. perl v5.16.2 2009-11-23 Test::Script(3)
All times are GMT -4. The time now is 08:08 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy