How to delete a columns of a CSV file which has cell values with a string enclosed in " , "? Post: 302902037

Sponsored Content

Top Forums Shell Programming and Scripting How to delete a columns of a CSV file which has cell values with a string enclosed in " , "? Post 302902037 by drl on Saturday 17th of May 2014 05:03:55 PM

05-17-2014

Registered User

Hi.

Here is an example with 5 columns (I usually say fields), and the commands to csvtool will copy 1-2 and 5, effectively deleting columns 3 and 4. Note that some columns contain embedded commas and spaces:

Code:

#!/usr/bin/env bash

# @(#) s1	Demonstrate CSV file manipulation, csvtool.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
LC_ALL=C ; LANG=C ; export LC_ALL LANG
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C csvtool

FILE=${1-data1}

pl " Input data file $FILE:"
cat $FILE

pl " Results, copy columns 1,2,5, omitting 3,4:"
csvtool col 1,2,5 $FILE

exit 0

producing:

Code:

$ ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian 5.0.8 (lenny, workstation) 
bash GNU bash 3.2.39
csvtool - ( /usr/bin/csvtool, 2008-05-22 )

-----
 Input data file data1:
"a1","b1,c1 d1",e1,"f1 g1,h1","i1"
"a2","b2,c2 d2",e2,"f2 g2,h2","i2"
"a3","b3,c3 d3",e3,"f3 g3,h3","i3"

-----
 Results, copy columns 1-2,5, omitting 3,4:
a1,"b1,c1 d1",i1
a2,"b2,c2 d2",i2
a3,"b3,c3 d3",i3

Quote:

Originally Posted by Litu19

Thanks drl...Will I be able to access the commands that are used by this tool ? or this is something that will process the original data and give the result back in a new file. If that is the case then it is not going to serve my purpose.

Sorry, I don't understand this statement / question.

See man page for details ... cheers, drl

drl

View Public Profile for drl

Find all posts by drl

9 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

How to add two values in the same cell of CSV file

I need help to create a csv file with Unix command. In csv file, i need to put two values in the same cell. Rite now, whts happening is, if i put 2 values in the same cell, its comming as " asd, zxc" but i want it in different line but in same cell. asd zxc Please reply me ASAP.

2. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone...

3. UNIX for Dummies Questions & Answers

replace "," with "." only in specific columns of a file?

Hi all, I have this text file containing 9 columns separated by space. The 8th columns contains the numbers. C1 C2 C3 C4 C5 C6 C7 C8 C9 er rt yt gh iu nk il 0.07 xs yt lr ty bg iu zk nh 0,0005 lt ...etc. I want to replace the comma with full stop only in 8th coloumn. the output...

4. Shell Programming and Scripting

how to use "cut" or "awk" or "sed" to remove a string

logs: "/home/abc/public_html/index.php" "/home/abc/public_html/index.php" "/home/xyz/public_html/index.php" "/home/xyz/public_html/index.php" "/home/xyz/public_html/index.php" how to use "cut" or "awk" or "sed" to get the following result: abc abc xyz xyz xyz

5. Shell Programming and Scripting

How to delete a column/columns of a CSV file which has cell values with a string enclosed in " , "?

How can I delete a column from a CSV file which has comma separated value with a string enclosed in double quotes and a comma in between? I have a file 44.csv with 4 lines including the header like the below format: column1, column2, column3, column 4, column5, column6 12,455,"string with...

6. Shell Programming and Scripting

Working with CSV files values enclosed with ""

I have a CSV file as shown below "1","SANTHA","KUMAR","SAM,MILLER","DEVELOPER","81,INDIA" "2","KAPIL","DHAMI","ECO SPORT","DEVELOPER","82,INDIA" File is comma delimited.All the field values are enclosed by double quotes. But while using awk or cut, it interprets the comma which is present in...

7. Shell Programming and Scripting

How to delete the commas in a .CSV file that are enclosed in a string with double quotes?

Okay, I would like to delete all the commas in a .CSV file (TEST.CSV) or at least substitute them with empty space, that are enclosed in double quote. Please see the sample file as below: column 1,column 2,column 3,column 4,column 5,column 6,column 7,column 8,column 9,column 10...

8. Shell Programming and Scripting

Delete all log files older than 10 day and whose first string of the first line is "MSH" or "<?xml"

Dear Ladies & Gents, I have a requirement to delete all the log files in /var/log/test directory that are older than 10 days and their first line begin with "MSH" or "<?xml" or "FHS". I've put together the following BASH script, but it's erroring out: for filename in $(find /var/log/test...

9. Shell Programming and Scripting

Bash script - Print an ascii file using specific font "Latin Modern Mono 12" "regular" "9"

Hello. System : opensuse leap 42.3 I have a bash script that build a text file. I would like the last command doing : print_cmd -o page-left=43 -o page-right=22 -o page-top=28 -o page-bottom=43 -o font=LatinModernMono12:regular:9 some_file.txt where : print_cmd ::= some printing...

LEARN ABOUT DEBIAN

ifpps

ifpps(8)							netsniff-ng-toolkit							  ifpps(8)

NAME

       ifpps - fetch and format kernel network statistics

SYNOPSIS

       ifpps	 -d|--dev <netdev> [-t|--interval <sec>][-p|--promisc][-c|--term]
	    [-C|--csv][-H|--csv-tablehead][-l|--loop][-v|--version][-h|--help]

DESCRIPTION

       A tiny tool to provide top-like reliable networking statistics.	ifpps reads out the 'real' kernel statistics, so it does not give
       erroneous statistics on high I/O load.

OPTIONS

       ifpps --dev eth0
	   Fetch eth0 interface statistics.

       ifpps --dev eth0 --interval 60 --csv
	   Output eth0 interface statistics every minute in CSV format.

OPTIONS

       -h|--help
	   Print help text and lists all options.

       -v|--version
	   Print version.

       -d|--dev <netdev>
	   Device to fetch statistics for i.e., eth0.

       -p|--promisc
	   Put the device in promiscuous mode

       -t|--interval <time>
	   Refresh time in sec (default 1 sec)

       -c|--term
	   Output to terminal

       -C|--csv
	   Output in CSV format.  E.g. post-processing with Gnuplot et al.

       -H|--csv-tablehead
	   Print CSV table head.

       -l|--loop
	   Loop terminal output.

AUTHOR

       Written by Daniel Borkmann <daniel@netsniff-ng.org>

DOCUMENTATION

       Documentation by Emmanuel Roullit <emmanuel@netsniff-ng.org>

BUGS

       Please report bugs to <bugs@netsniff-ng.org>

								    2012-06-29								  ifpps(8)