Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Have problem with extra EOLs in my CSV - need help cleaning out
Top Forums Shell Programming and Scripting Have problem with extra EOLs in my CSV - need help cleaning out Post 302906009 by Corona688 on Monday 16th of June 2014 03:55:59 PM
Old 06-16-2014
Logically, you should fix the thing which is producing your broken CSV's, not kludge them back together after, but:

Code:
awk -F, -v OFS="," 'NR==1 { MINREC=NF; next }
{
        if(PREV) {      $0=PREV $0;     $1=$1   }
        if(NF >= MINREC) { print; PREV=""; }
        else    PREV=$0
}' < inputfile > outputfile

This will fail if , appears inside your quoted sections.
This User Gave Thanks to Corona688 For This Post:
richardsantink
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

manipulating csv to add extra row

hi how do i manipulate .csv file to add an extra row after each row using shell script? I need a blank line added for each 1000 records in my file? I will then need to copy and paste some data in the blank row created. thanks 4 ur support neil (3 Replies)
Discussion started by: neil546
3 Replies

2. Shell Programming and Scripting

SED help - cleaning up code, extra spaces won't go away

Hello, W/in the script I'm working on, I have a need to take a column from a file, and format it so I can have a variable that will egrep for & invert the regex from another file. My solution is this: VAR=`awk -F, '{print $2}' $FAIL | sed 's/-i/\|/g'` VAR2=`echo $VAR | sed 's/... (5 Replies)
Discussion started by: Matthias03
5 Replies

3. UNIX for Dummies Questions & Answers

cleaning up spaces from fixed width file while converting to csv file

Open to a sed/awk/or perl alternative so that i can stick command into my bash script. This is a problem I resolve using a combination of cut commands - but that is getting convoluted. So would really appreciate it if someone could provide a better solution which basically replaces all... (3 Replies)
Discussion started by: svn
3 Replies

4. Shell Programming and Scripting

shell script to remove extra commas from CSV outp file

Name,,,,,,,,,,,,,,,,,,,,Domain,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Contact,Phone,Email,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Location -----------------------,------------------------------------------------,-------,-----,---------------------------------,------------------------------------ ----... (1 Reply)
Discussion started by: sreenath1037
1 Replies

5. Shell Programming and Scripting

Ascii Mode appending extra records to csv file

I am relatively new to this forum and Unix scripting. ksh script: part 1 :will call a PL\SQL program will create 3 CSV file at the unix directory. part 2 : will sftp the files to the EFT server. Once the EFT server receives these file , it will transfer them to a shared windows folders. ... (3 Replies)
Discussion started by: developerpa
3 Replies

6. Shell Programming and Scripting

Adding Extra Commas to a CSV file

Trying in this forum. Not sure if it is permitted.... but in need of help. Please find the requirements in the below link. https://www.unix.com/unix-dummies-questions-answers/191503-add-extra-commas-csv-file-2.html#post302665179 Thanks in Advance. (1 Reply)
Discussion started by: chillblue
1 Replies

7. UNIX for Dummies Questions & Answers

To Add extra commas to a CSV file using 2 files...

Hi , Based on my previous requirement the code works fine for comma as delimiter. Now my Req is widened up a bit .. There will be two set of files .. one with comma as delimiter and other with semi-colon ; as delimiter. Second Sample file. With Double Quotes (Semi-Colon... (1 Reply)
Discussion started by: chillblue
1 Replies

8. UNIX for Dummies Questions & Answers

To Add extra commas to a CSV file.

Hi All, I got this requirement to process a complex CSV file. Eg File. Line 1: Name:,XYz Line 2: Age:,15 Line 3: Grade:,7 Line 4: Line 5: English, Maths, Science,Spanish Line 6:10,11,13,14 As you can see the maximum column is 4 . The file i need to make is Line 1: Name:,XYz,,... (12 Replies)
Discussion started by: chillblue
12 Replies

9. Shell Programming and Scripting

Randomly inserting extra columns into csv file

Hi Tech Guru, I have a test file as below , which needs some more fields to be populated randomly : dks3243;12;20130823;1420;25m;0;syt dks3243;rocy;10 dks3243;kiop;18 sde21p4;77;20151210;8479;7py;9;vfr sde21p4;temp;67 sfq6i01;12;20120123;3412;4rd;7;jui sfq6i01;uymk;90 sfq6i01;kiop;51 ... (8 Replies)
Discussion started by: Lokesha
8 Replies

10. Shell Programming and Scripting

Need to Insert three extra columns in csv file

Hello Experts, I got a requirement i have a input file which am getting from different source,Now i want to add extra 3 columns to this file like BASE,ACTUAL and DATE. Input File Looks like QUAL CHGE TYP LAW COM1 COM2 A 1 X SED HO ASE B 3 Z CDE SE ... (5 Replies)
Discussion started by: ahmed.vaghar
5 Replies

LEARN ABOUT DEBIAN

ifpps

ifpps(8)							netsniff-ng-toolkit							  ifpps(8)

NAME

       ifpps - fetch and format kernel network statistics

SYNOPSIS

       ifpps	 -d|--dev <netdev> [-t|--interval <sec>][-p|--promisc][-c|--term]
	    [-C|--csv][-H|--csv-tablehead][-l|--loop][-v|--version][-h|--help]

DESCRIPTION

       A tiny tool to provide top-like reliable networking statistics.	ifpps reads out the 'real' kernel statistics, so it does not give
       erroneous statistics on high I/O load.

OPTIONS

       ifpps --dev eth0
	   Fetch eth0 interface statistics.

       ifpps --dev eth0 --interval 60 --csv
	   Output eth0 interface statistics every minute in CSV format.

OPTIONS

       -h|--help
	   Print help text and lists all options.

       -v|--version
	   Print version.

       -d|--dev <netdev>
	   Device to fetch statistics for i.e., eth0.

       -p|--promisc
	   Put the device in promiscuous mode

       -t|--interval <time>
	   Refresh time in sec (default 1 sec)

       -c|--term
	   Output to terminal

       -C|--csv
	   Output in CSV format.  E.g. post-processing with Gnuplot et al.

       -H|--csv-tablehead
	   Print CSV table head.

       -l|--loop
	   Loop terminal output.

AUTHOR

       Written by Daniel Borkmann <daniel@netsniff-ng.org>

DOCUMENTATION

       Documentation by Emmanuel Roullit <emmanuel@netsniff-ng.org>

BUGS

       Please report bugs to <bugs@netsniff-ng.org>

								    2012-06-29								  ifpps(8)
All times are GMT -4. The time now is 01:10 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy