the idea is to remove the lines that are repeated more than once , write how many times the line is repeated and the summation field length . I also want to arrange fields to have the following matches
I have huge txt file having millions of trade data.
For e.g
Trade.txt (first 8 lines in the file is header info)
COB_DATE,TRADE_ID,SOURCE_SYSTEM_TRADE_ID,TRADE_GROUP_ID,
TRADE_TYPE,DEALER_NAME,EXTERNAL_COUNTERPARTY_ID,
EXTERNAL_COUNTERPARTY_NAME,DB_COUNTERPARTY_ID,... (6 Replies)
Hi All,
I have huge trade file with milions of trades.I need to remove duplicate records (e.g I have following records)
30/10/2009,trdeId1,..,..
26/10/2009.tradeId1,..,..,,
30/10/2009,tradeId2,..
In the above case i need to filter duplicate recods and I should get following output.... (2 Replies)
Can anyone help me to removing duplicate records from 2 separate files in UNIX?
Please find the sample records for both the files
cat Monday.dat
3FAHP0JA1AR319226MOHMED ATEK 966504453742 SAU2010DE
3LNHL2GC6AR636361HEA DEUK CHOI 821057314531 KOR2010LE
3MEHM0JG7AR652083MUTLAB NAL-NAFISAH... (4 Replies)
Hi,
I want to remove duplicate records including the first line based on column1. For example
inputfile(filer.txt):
-------------
1,3000,5000
1,4000,6000
2,4000,600
2,5000,700
3,60000,4000
4,7000,7777
5,999,8888
expected output:
----------------
3,60000,4000
4,7000,7777... (5 Replies)
Hi All,
I want to remove the rows from File1.csv by comparing a column/field in the File2.csv. If both columns matches then I want that row to be deleted from File1 using shell script(awk). Here is an example on what I need.
File1.csv:
RAJAK,ACTIVE,1
VIJAY,ACTIVE,2
TAHA,ACTIVE,3... (6 Replies)
I was reading this thread. It looks like a simpler way to say this is to only keep uniq lines based on field or column 1.
https://www.unix.com/shell-programming-scripting/165717-removing-duplicate-records-file-based-single-column.html
Can someone explain this command please? How are there no... (5 Replies)
Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted!
1. The problem statement, all variables and given/known data:
You will write a script that will remove all HTML tags from an HTML document and remove any consecutive... (3 Replies)
Friends,
I have data sorted on id like this
id addressl
1 abc
2 abc
2 abc
2 abc
3 aabc
4 abc
4 abc
I want to pick all ids with addressesses leaving out duplicate records. Desired output would be
id address
1 abc
2 abc
3 abc
4 abc (5 Replies)
Hello
I have been trying to remove a row from a file which has the same first three columns as another row - I have tried lots of different combinations of suggestion on this forum but can't get it exactly right.
what I have is
900 - 1000 = 0
900 - 1000 = 2562
1000 - 1100 = 0
1000 - 1100... (7 Replies)
Discussion started by: tinytimmay
7 Replies
LEARN ABOUT SUSE
faith
FAITH(4) BSD Kernel Interfaces Manual FAITH(4)NAME
faith -- IPv6-to-IPv4 TCP relay capturing interface
SYNOPSIS
device faith
DESCRIPTION
The faith interface captures IPv6 TCP traffic, for implementing userland IPv6-to-IPv4 TCP relay like faithd(8).
Each faith interface is created at runtime using interface cloning. This is most easily done with the ifconfig(8) create command or using
the cloned_interfaces variable in rc.conf(5).
Special action will be taken when IPv6 TCP traffic is seen on a router, and the routing table suggests to route it to the faith interface.
In this case, the packet will be accepted by the router, regardless of the list of IPv6 interface addresses assigned to the router. The
packet will be captured by an IPv6 TCP socket, if it has the IN6P_FAITH flag turned on and matching address/port pairs. As a result, faith
will let you capture IPv6 TCP traffic to some specific destination addresses. Userland programs, such as faithd(8) can use this behavior to
relay IPv6 TCP traffic to IPv4 TCP traffic. The program can accept some specific IPv6 TCP traffic, perform getsockname(2) to get the IPv6
destination address specified by the client, and perform application-specific address mapping to relay IPv6 TCP to IPv4 TCP.
The IN6P_FAITH flag on a IPv6 TCP socket can be set by using setsockopt(2), with level IPPROTO_IPV6 and optname IPv6_FAITH.
To handle error reports by ICMPv6, some ICMPv6 packets routed to an faith interface will be delivered to IPv6 TCP, as well.
To understand how faith can be used, take a look at the source code of faithd(8).
As the faith interface implements potentially dangerous operations, great care must be taken when configuring it. To avoid possible misuse,
the sysctl(8) variable net.inet6.ip6.keepfaith must be set to 1 prior to using the interface. When net.inet6.ip6.keepfaith is 0, no packets
will be captured by the faith interface.
The faith interface is intended to be used on routers, not on hosts.
SEE ALSO inet(4), inet6(4), faithd(8)
Jun-ichiro itojun Hagino and Kazu Yamamoto, An IPv6-to-IPv4 transport relay translator, RFC3142.
HISTORY
The FAITH IPv6-to-IPv4 TCP relay translator first appeared in the WIDE hydrangea IPv6 stack.
BSD April 10, 1999 BSD
05-21-2011
Need awk script for removing duplicate records
