I've been searching for the best way to duplicate a line based on a variable in the next line.
Sample Data:
I want to duplicate the lines based on how many times the log reports it was was repeated. So in the above example, line 1 would be duplicated 2 times, line 3 would be duplicated 5 times and line 5 would be duplicated 3 times.
I'm at a loss how to accomplish my goal. Anyone have a suggestion?
Thanks
Last edited by Scrutinizer; 11-23-2012 at 05:56 PM..
Reason: code tags
Hi,
I have research data, which is organized to 100 folders numbered 00-99. I have many sets of 100 folders, for different values of initial parameters. For some reason, the computer that ran the program to gather the data, didn't always create a unique seed for each folder. I anticipated that... (1 Reply)
Hey all, a relative bash/script newbie trying solve a problem.
I've got a text file with lots of lines that I've been able to clean up and format with awk/sed/cut, but now I'd like to remove the lines with duplicate usernames based on time stamp. Here's what the data looks like
2007-11-03... (3 Replies)
Hi,
I'm trying to create an XML sitemap of our dynamic ecommerce sites SEO Friendly URLs and am trying to create the initial page listing.
I have a CSV file that looks like the following and need duplicate the lines based on a value which needs calculating.
... (2 Replies)
hello all,
I have an input file with four columns like this with a lot of lines
and for example, line 1 and line 5 match because the first 4 characters match and the fourth column matches too. I want to keep the line that has the lowest number in the third column. So I discard line 5.... (5 Replies)
Hi I have a file like this. I need to eliminate lines with first column having the same value 10 times.
13 18 1 + chromosome 1, 122638287 AGAGTATGGTCGCGGTTG
13 18 1 + chromosome 1, 128904080 AGAGTATGGTCGCGGTTG
13 18 1 - chromosome 14, 13627938 CAACCGCGACCATACTCT
13 18 1 + chromosome 1,... (5 Replies)
Hi,
In a file, I have to mark duplicate records as 'D' and the latest record alone as 'C'.
In the below file, I have to identify if duplicate records are there or not based on Man_ID, Man_DT, Ship_ID and I have to mark the record with latest Ship_DT as "C" and other as "D" (I have to create... (7 Replies)
I have a csv file that I would like to remove duplicate lines based on field 1 and sort. I don't care about any of the other fields but I still wanna keep there data intact. I was thinking I could do something like this but I have no idea how to print the full line with this. Please show any method... (8 Replies)
Hi experts, I have a tab-delimited file with one column containing values separated by a comma. I wish to duplicate the entire line for every value in that comma-delimited field.
For example:
$cat file
4444 4444 4444 4444
9990 2222,7777 6666 2222 ... (3 Replies)
Dear community,
I have to remove duplicate lines from a file contains a very big ammount of rows (milions?) based on 1st and 3rd columns
The data are like this:
Region 23/11/2014 09:11:36 41752
Medio 23/11/2014 03:11:38 4132
Info 23/11/2014 05:11:09 4323... (2 Replies)
Hi,
I have tried to remove dublicate lines based on first column with pipe delimiter . but i ma not able to get some uniqu lines
Command : sort -t'|' -nuk1 file.txt
Input :
38376KZ|09/25/15|1.057
38376KZ|09/25/15|1.057
02006YB|09/25/15|0.859
12593PS|09/25/15|2.803... (2 Replies)
Discussion started by: parithi06
2 Replies
LEARN ABOUT CENTOS
dnet
DNET(8) BSD System Manager's Manual DNET(8)NAME
dnet -- dumb networking library test program
SYNOPSIS
dnet command args [...]
DESCRIPTION
dnet is a simple test program for the dnet(3) library. It can be used to compose and transmit network datagrams as a Unix-style filter (e.g.
reading from or writing to files and pipes) or modify the local system network configuration (including the ARP cache, firewall ruleset, net-
work interfaces, and routing table).
Payload generation commands
addr address [...]
Convert the address (specified as a hostname, IP address, or MAC address) into its binary representation on standard output.
hex string [...]
Convert the C-style escaped string (shellcode, for instance) into its binary representation on standard output.
rand len
Write len random bytes to standard output.
Packet encapsulation commands
eth [type type] [src mac] [dst mac]
Prepend the data read from standard input with an Ethernet header on standard output. The Ethernet type may be specified as 'arp', 'ip',
or as a hex, octal, or decimal number.
arp [op op] [sha mac] [spa host] [tha mac] [tpa host]
Prepend the data read from standard input with an ARP header on standard output. The ARP op may be specified as 'req', 'rep', 'revreq',
'revrep', or as a hex, octal, or decimal number.
ip [tos num] [id num] [off offset] [ttl num] [proto protocol] [src host] [dst dst]
Prepend the data read from standard input with an IP header on standard output. The fragmentation offset may be specified as a decimal
number (optionally concatenated with '+' to indicate more fragments) or as a hex number. The protocol may be specified by name, or as a
hex, octal, or decimal number.
icmp [type num] [code num]
Prepend the data read from standard input with an ICMP header on standard output.
tcp [sport port] [dport port] [flags flags] [seq num] [ack num] [win num] [urp num]
Prepend the data read from standard input with a TCP header on standard output. A port may be specified by name or hex, octal, or deci-
mal number. The TCP flags may be specified as some combination of the characters in the set 'SAFRPU' or as a hex number.
udp [sport port] [dport port]
Prepend the data read from standard input with a UDP header on standard output. A port may be specified by name or hex, octal, or deci-
mal number.
Packet transmission commands
send [device]
Read a packet from standard input and send it over the network. If no device is specified, the packet is assumed to be an IP datagram
and routed to its destination. Otherwise, the packet is assumed to be an Ethernet frame and is transmitted on the specified interface.
Kernel interface commands
arp show
Display the kernel ARP cache.
arp get host
Display the kernel ARP entry for host.
arp add host mac
Add an ARP entry mapping the mac address for host.
arp delete host
Delete the ARP entry for host.
fw show
Display the kernel firewall ruleset.
fw add|delete action direction device protocol src[:port[-max]] dst[:port[-max]] [type[/code]]
Add a rule to or delete a rule from the active firewall ruleset. The action must be either 'allow' or 'block'. The direction must be
either 'in' or 'out'. The device may specify an interface name, or 'any'. The protocol may be specified by name, or as a decimal num-
ber. For TCP and UDP protocols, a port (or range, if specified with a max value) may be specified in decimal and appended to the source
and/or destination address. For ICMP, a type (and optional code) may be specified in decimal.
intf show
Display the configuration of all network interfaces.
intf get device
Display the configuration for the interface specified by device.
intf set device [alias host] [dst host] [inet host] [link mac] [up|down] [arp|noarp]
Configure the interface specified by device.
route show
Display the kernel routing table.
route get dst
Display the route for the destination dst, specified as a hostname, IP address, or network prefix in CIDR notation.
route add dst gw
Add a route for the destination dst through the gateway gw.
route delete dst
Delete the route for the destination dst.
EXAMPLES
Send a UDP datagram containing random shellcode:
dnet hex "xebx1fx5ex89x76x08x31xc0x88x46x07x89"
"x46x0cxb0x0bx89xf3x8dx4ex08x8dx56x0cxcdx80"
"x31xdbx89xd8x40xcdx80xe8xdcxffxffxff/bin/sh" |
dnet udp sport 555 dport 666 |
dnet ip proto udp src 1.2.3.4 dst 5.6.7.8 | dnet send
Save an ARP request in a file and send it twice:
dnet arp op req sha 0:d:e:a:d:0 spa 10.0.0.3 tpa 10.0.0.4 |
dnet eth type arp src 0:d:e:a:d:0 dst ff:ff:ff:ff:ff:ff > arp.pkt
dnet send fxp0 < arp.pkt
dnet send fxp0 < arp.pkt
Send a fragmented ping packet:
# Create ping packet with IP header, to set ICMP checksum
echo "monkey monkey monkey monkey" | dnet icmp type 8 code 0 |
dnet ip proto icmp src 1.2.3.4 dst 5.6.7.8 > ping.pkt
# Chop off IP header
dd if=ping.pkt of=ping.data bs=20 skip=1
# Fragment IP payload
split -b 24 ping.data p.
# Send fragments
dnet ip id 1 off 0+ proto icmp src 1.2.3.4 dst 5.6.7.8 < p.aa |
dnet send
dnet ip id 1 off 24 proto icmp src 1.2.3.4 dst 5.6.7.8 < p.ab |
dnet send
SEE ALSO dnet(3)AUTHORS
Dug Song <dugsong@monkey.org>
BSD October 17, 2001 BSD