Sponsored Content
Top Forums Shell Programming and Scripting Bash script help - removing certain rows from .csv file Post 302798495 by MrTuxor on Wednesday 24th of April 2013 02:43:12 PM
Old 04-24-2013
Quote:
Originally Posted by Skrynesaver
Code:
tshark -r "$in_pcap" -T fields -e frame.number -e ip.src -e ip.dst -e http.host -e frame.time -e frame.time_relative -E header=y -E separator=, |egrep -v '^[^,]*,[^,]*,[^,]*,,'> "$out_csv"

Thanks a million for this answer!

I went from 600,000+ rows to analyze to less than 3100 and it did everything I wanted it to.

I do have one problem though. If I run this line:
tshark -r test.pcap -T fields -e frame.number -e ip.src -e ip.dst -e http.host -e frame.time -e frame.time_relative -E header=y -E separator=, |egrep -v '^[^,]*,[^,]*,[^,]*,,'> test.csv
Everything works perfectly fine.

However if I run my script mentioned above with the variables $in_pcap and $out_csv, I get the following error message:
./pcapAnalyze: line 9: $out_csv: ambiguous redirect
tshark: Output fields were specified with "-e", but "-Tfields" was not specified.


I wanted to make sure the command ran on its own without the variables to limit the amount of things that could go wrong. After simply replacing the hard coded .pcap and .csv files with variables, I get that error message. The only thing that I changed was implementing the variable... Why is it doing that?

Thanks in advance!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash Script - Removing Year and Text from File Name

Hi, I have files with names in the following naming pattern, Name.of.moviemoretext-moretext.mov or Name of moviemoretext-moretext.mov And I would like to delete the and moretext-moretext leaving just the Name of movie. The Name of movie will always be different and the year will... (4 Replies)
Discussion started by: Monkey Dean
4 Replies

2. Shell Programming and Scripting

Deleting rows from csv file

Hello, I am supposed to process about 100 csv files. But these files have some extra lines at the bottom of the file. these extra lines start with a header for each column and then some values below. These lines are actually a summary of the actual data and not supposed to be processed. These... (8 Replies)
Discussion started by: cobroraj
8 Replies

3. Shell Programming and Scripting

Removing rows from a file

I have a file like below and want to use awk to solve this problem. The record separator is ">". I want to look at each record section enclosed within ">". Find the row with the 2nd and 3rd columns being 0, such as 10 0 0 I need to take the first number which in this case is 10. Then... (15 Replies)
Discussion started by: kristinu
15 Replies

4. Shell Programming and Scripting

Removing rows based on a another file

Hi, I am not sure if this has already been asked (I tried the search but the search was too broad). Basically I want to remove rows based on another file. So file1 looks like this (tab seperated): HHN 3 5 5 HUJ 2 2 1 JJJ 3 1 1 JUN 2 1 3 I have another file (file2)... (2 Replies)
Discussion started by: kylle345
2 Replies

5. Shell Programming and Scripting

Need to modify csv-file with bash script

Hi Guys, I need to write a script, that exports the "moz_places" table of the "places.sqlite"-file (firefox browser history) into a csv-file. That part works. After the export, my csv looks like this: ... 4429;http://www.sqlite.org/sqlite.html;"Command Line Shell For... (11 Replies)
Discussion started by: Sebi0815
11 Replies

6. Shell Programming and Scripting

Please do help: Perl Script to pull out rows from a CSV file

I have CSV file that contains data in the format as shown below: ABC, 67, 56, 67, 78, 89, 76, 55 PDR, 85, 83, 83, 72, 82, 89, 83 MPG, 86, 53, 54, 65, 23, 54, 75 .. .. .. .. I want to create a script that will pull out the rows from the above sheet and paste it into another CSV file.... (12 Replies)
Discussion started by: pankajusc
12 Replies

7. Shell Programming and Scripting

Shell script to count unique rows in a CSV

HI All, I have a CSV file of 30 columns separated by ,. I want to get a count of all unique rows written to a flat file. The CSV file is around 5000 rows The first column is a time stamp and I need to exclude while counting unique Thanks, Ravi (4 Replies)
Discussion started by: Nani369
4 Replies

8. Shell Programming and Scripting

Removing rows and chars from text file

Dear community, maybe I'm asking the moon :rolleyes:, but I'm scratching my head to find a solution for it. :wall: I have a file called query.out (coming from Oracle query), the file is like this: ADDR TOTAL -------------------- ---------- TGROUPAGGR... (16 Replies)
Discussion started by: Lord Spectre
16 Replies

9. Shell Programming and Scripting

Removing Duplicate Rows in a file

Hello I have a file with contents like this... Part1 Field2 Field3 Field4 (line1) Part2 Field2 Field3 Field4 (line2) Part3 Field2 Field3 Field4 (line3) Part1 Field2 Field3 Field4 (line4) Part4 Field2 Field3 Field4 (line5) Part5 Field2 Field3 Field4 (line6) Part2 Field2 Field3 Field4... (7 Replies)
Discussion started by: ekbaazigar
7 Replies

10. Shell Programming and Scripting

Removing top few and bottom few rows in a file

Hi, I have a requirement where I need to delete given number of top and bottom rows in a flat file which has new line as its delimiter. For ex: if top_rows=2 & bottom_rows=1 Then in a given file which looks like: New York DC LA London Tokyo Prague Paris Bombay Sydney... (7 Replies)
Discussion started by: calredd
7 Replies
MERGECAP(1)						  The Wireshark Network Analyzer					       MERGECAP(1)

NAME
mergecap - Merges two or more capture files into one SYNOPSIS
mergecap [ -a ] [ -F <file format> ] [ -h ] [ -s <snaplen> ] [ -T <encapsulation type> ] [ -v ] -w <outfile>|- <infile> [<infile> ...] DESCRIPTION
Mergecap is a program that combines multiple saved capture files into a single output file specified by the -w argument. Mergecap knows how to read pcap capture files, including those of tcpdump, Wireshark, and other tools that write captures in that format. By default, Mergecap writes the capture file in pcap format, and writes all of the packets from the input capture files to the output file. Mergecap is able to detect, read and write the same capture files that are supported by Wireshark. The input files don't need a specific filename extension; the file format and an optional gzip compression will be automatically detected. Near the beginning of the DESCRIPTION section of wireshark(1) or <http://www.wireshark.org/docs/man-pages/wireshark.html> is a detailed description of the way Wireshark handles this, which is the same way Mergecap handles this. Mergecap can write the file in several output formats. The -F flag can be used to specify the format in which to write the capture file, mergecap -F provides a list of the available output formats. Packets from the input files are merged in chronological order based on each frame's timestamp, unless the -a flag is specified. Mergecap assumes that frames within a single capture file are already stored in chronological order. When the -a flag is specified, packets are copied directly from each input file to the output file, independent of each frame's timestamp. The output file frame encapsulation type is set to the type of the input files if all input files have the same type. If not all of the input files have the same frame encapsulation type, the output file type is set to WTAP_ENCAP_PER_PACKET. Note that some capture file formats, most notably pcap, do not currently support WTAP_ENCAP_PER_PACKET. This combination will cause the output file creation to fail. OPTIONS
-a Causes the frame timestamps to be ignored, writing all packets from the first input file followed by all packets from the second input file. By default, when -a is not specified, the contents of the input files are merged in chronological order based on each frame's timestamp. Note: when merging, mergecap assumes that packets within a capture file are already in chronological order. -F <file format> Sets the file format of the output capture file. Mergecap can write the file in several formats; mergecap -F provides a list of the available output formats. The default is to use the file format of the first input file. -h Prints the version and options and exits. -s <snaplen> Sets the snapshot length to use when writing the data. If the -s flag is used to specify a snapshot length, frames in the input file with more captured data than the specified snapshot length will have only the amount of data specified by the snapshot length written to the output file. This may be useful if the program that is to read the output file cannot handle packets larger than a certain size (for example, the versions of snoop in Solaris 2.5.1 and Solaris 2.6 appear to reject Ethernet frames larger than the standard Ethernet MTU, making them incapable of handling gigabit Ethernet captures if jumbo frames were used). -T <encapsulation type> Sets the packet encapsulation type of the output capture file. If the -T flag is used to specify a frame encapsulation type, the encapsulation type of the output capture file will be forced to the specified type, rather than being the type appropriate to the encapsulation type of the input capture files. Note that this merely forces the encapsulation type of the output file to be the specified type; the packet headers of the packets will not be translated from the encapsulation type of the input capture file to the specified encapsulation type (for example, it will not translate an Ethernet capture to an FDDI capture if an Ethernet capture is read and '-T fddi' is specified). -v Causes mergecap to print a number of messages while it's working. -w <outfile>|- Sets the output filename. If the name is '-', stdout will be used. This setting is mandatory. EXAMPLES
To merge two capture files together, 100 seconds apart use: capinfos -aeS a.pcap b.pcap (Let's suppose a.pcap starts at 1009932757 and b.pcap ends at 873660281. 1009932757 - 873660281 - 100 = 136272376 seconds.) editcap -t 136272376 b.pcap b-shifted.pcap mergecap -w compare.pcap a.pcap b-shifted.pcap SEE ALSO
pcap(3), wireshark(1), tshark(1), dumpcap(1), editcap(1), text2pcap(1), pcap-filter(7) or tcpdump(8) NOTES
Mergecap is based heavily upon editcap by Richard Sharpe <sharpe[AT]ns.aus.com> and Guy Harris <guy[AT]alum.mit.edu>. Mergecap is part of the Wireshark distribution. The latest version of Wireshark can be found at <http://www.wireshark.org>. HTML versions of the Wireshark project man pages are available at: <http://www.wireshark.org/docs/man-pages>. AUTHORS
Original Author -------- ------ Scott Renfro <scott[AT]renfro.org> Contributors ------------ Bill Guyton <guyton[AT]bguyton.com> 1.10.3 2013-07-28 MERGECAP(1)
All times are GMT -4. The time now is 10:30 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy