Parsing a CSV file and deleting all rows on condition
Hello list,
I am working on a csv file which contains two fields per record which contain IP addresses. What I am trying to do is find records which have identical fields(IP addresses) which occur 4(four) times, and if they do, delete all records with that specific identical field(ip address).
for example, the csv looks like this:
Code:
field 1,192.168.1.1,field 3, 192.168.1.128
field 1,192.168.1.1,field 3, 192.168.1.128
field 1,192.168.1.1,field 3, 192.168.1.128
field 1,192.168.1.1,field 3, 192.168.1.128
field 1,10.10.10.1,field 3, 192.168.1.128
field 1,172.16.10.1,field 3, 192.168.1.128
field 1,10.10.10.3,field 3, 192.168.1.128
so I want to parse the csv and delete the following rows where the ip address in field 2 is identical for 4 occurrences:
Code:
field 1,192.168.1.1,field 3, 192.168.1.128
field 1,192.168.1.1,field 3, 192.168.1.128
field 1,192.168.1.1,field 3, 192.168.1.128
field 1,192.168.1.1,field 3, 192.168.1.128
any ideas?
thanks.
Last edited by Franklin52; 06-08-2011 at 04:01 AM..
Reason: Please use code tags
I am getting some spaces between the two lines(rows) in file.i want delete that empty rows in the file
example
1 abc xyz
2 def jkl
like i am having lots of rows in a file i want to delete the spce between the two rows give any... (7 Replies)
Hello,
I am supposed to process about 100 csv files. But these files have some extra lines at the bottom of the file. these extra lines start with a header for each column and then some values below. These lines are actually a summary of the actual data and not supposed to be processed. These... (8 Replies)
Hi Guys,
I need help in modifying a large text file containing more than 1-2 lakh rows of data using unix commands. I am quite new to the unix language
the text file contains data in a pipe delimited format
sdfsdfs
sdfsdfsd
START_ROW
sdfsd|sdfsdfsd|sdfsdfasdf|sdfsadf|sdfasdf... (9 Replies)
I have been trying to find a good solution for this seemingly simple task for 2 days, and I'm giving up and posting a thread. I hope someone can help me out!
I'm on HPUX, using sqlplus, mailx, awk, have some other tools available, but can't install stuff that isn't already in place (without a... (6 Replies)
I know this is a complicated question but I will try to illustrate it with some data. I have a data file that looks like the following:
1341 NA06985 0 0 2 46.6432798439
1341 NA06991 NA06993 NA06985 2 48.8478948517
1341 NA06993 0 0 1 45.8022601455
1340 NA06994 0 0 1 48.780669145
1340... (1 Reply)
there are 20 variables and I would like to delete the rows if 13th-20th columns are all NA.
Thank you!
FID IID aspirpre statihos fibrahos ocholhos arbhos betabhos alphbhos cacbhos diurehos numbcig.x toast1 toast2 toast3 toast4 ischoth1 ischoth2 ischoth3 ischoth4
101 101 1 1 1 1 1 2 1 2... (2 Replies)
Hi Friends,
I have come across some files where some of the columns don not have data.
Key, Data1,Data2,Data3,Data4,Data5
A,5,6,,10,,
A,3,4,,3,,
B,1,,4,5,,
B,2,,3,4,,
If we see the above data on Data5 column do not have any row got filled. So remove only that column(Here Data5) and... (4 Replies)
Discussion started by: ks_reddy
4 Replies
LEARN ABOUT REDHAT
pilot-addresses
pilot-addresses(1) General Commands Manual pilot-addresses(1)NAME
pilot-addresses - read and write address book databases to and from a Palm handheld device, such as those made from Palm, Handspring, Han-
dera, TRGPro, Sony or other Palm Compatible Handheld PDA device
SYNOPSIS
pilot-addresses -p <port> [-c category ] [-d category ] [-r file | -w file ]
(Note that some options are not shown above)
DESCRIPTION
pilot-addresses allows the user to read all entries in the Palm address book database, write new entries into the database, and delete a
category or delete all entries in the database.
TARGET DEVICE
The default serial device used to communicate with a Palm is /dev/pilot. If the environment variable $PILOTPORT is set, its value will
override the default. A serial device specified on the command-line will be used regardless of any $PILOTPORT setting.
OPTIONS
Several options exist, including...
-p --port <port>,
Use device file port to communicate with the Palm handheld device. If this is not specified, will look for the $PILOTPORT environ-
ment variable. If both are not found, will fall back to /dev/pilot.
-h --help
Display help synopsis for pilot-addresses
-v --version
Display version of pilot-addresses
-a Augments fields in address book records with additional information. The augmented information is placed before and separated from
the field with a semi-colon, (;).
Augmented information includes:
category_name - placed in front of each record or
["Work" | "Home" | "Fax" | "Other" | "E-mail" | "Main" | "Pager" | "Mobile" ] - placed in front of each phone number field.
Empty fields are not augmented.
-c category
Install records to category category by default. Normally pilot-addresses uses Unfiled as the default category. This option is over-
ridden by the category specified in the record with the -a option.
-d category
Delete all records in the specified category before installing new records.
-D Delete all address book records in all categories. Obviously, be very careful with this one.
-e Escape all special characters with a backslash. This enables you to read and write entries with newline characters in a field or
note.
-q Causes pilot-addresses to be quiet and not prompt you to press the HotSync button.
-r file
Reads records from file and install them to the Palm address book database. (Use the -w file to get a template file for input
records.)
-t delim
Include category in each record, use the delimiter specified to separate all fields of a record. Delimiters are specified as fol-
lows: 3=tab, 2=;, 1=,. This overrides the default delimiter of comma between fields and semi-colon between a field's augmented
information. (Please note that this may generate confusing results when used with the -a option.)
-T Write a header line with field titles as the first line of the data file.
-w file
Get all address book records from the Palm address book database and writes them into file
USAGE
The program will connect to a target device and port, prompt the user to HotSync, and perform the requested read or write operation speci-
fied by the user.
EXAMPLES
To write all address records in a Palm to the file addrbook.csv:
pilot-addresses -w addrbook.csv
or
pilot-addresses -p /dev/irnine -w addrbook.csv
To read the address book records in the file addrbook.csv and install them on a Palm:
pilot-addresses -r addrbook.csv
To read the address book records in the file addrbook.csv and place them into the Palm address book database category Special after first
deleting all current records in the Special category on the palm:
pilot-addresses -c Special -d Special -r addrbook.csv
SEE ALSO pilot-link(7)KNOWN BUGS
pilot-addresses has no known bugs.
REPORTING BUGS
Report bugs at http://bugs.pilot-link.org/
AUTHOR
pilot-addresses originally written by Kenneth Albanowski, manual page was written by Robert Wittig <bob.wittig@gt.org>.
Free Software Foundation Palm Computing Device Tools pilot-addresses(1)