Sponsored Content
Top Forums Shell Programming and Scripting Have problem with extra EOLs in my CSV - need help cleaning out Post 302906009 by Corona688 on Monday 16th of June 2014 03:55:59 PM
Old 06-16-2014
Logically, you should fix the thing which is producing your broken CSV's, not kludge them back together after, but:

Code:
awk -F, -v OFS="," 'NR==1 { MINREC=NF; next }
{
        if(PREV) {      $0=PREV $0;     $1=$1   }
        if(NF >= MINREC) { print; PREV=""; }
        else    PREV=$0
}' < inputfile > outputfile

This will fail if , appears inside your quoted sections.
This User Gave Thanks to Corona688 For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

manipulating csv to add extra row

hi how do i manipulate .csv file to add an extra row after each row using shell script? I need a blank line added for each 1000 records in my file? I will then need to copy and paste some data in the blank row created. thanks 4 ur support neil (3 Replies)
Discussion started by: neil546
3 Replies

2. Shell Programming and Scripting

SED help - cleaning up code, extra spaces won't go away

Hello, W/in the script I'm working on, I have a need to take a column from a file, and format it so I can have a variable that will egrep for & invert the regex from another file. My solution is this: VAR=`awk -F, '{print $2}' $FAIL | sed 's/-i/\|/g'` VAR2=`echo $VAR | sed 's/... (5 Replies)
Discussion started by: Matthias03
5 Replies

3. UNIX for Dummies Questions & Answers

cleaning up spaces from fixed width file while converting to csv file

Open to a sed/awk/or perl alternative so that i can stick command into my bash script. This is a problem I resolve using a combination of cut commands - but that is getting convoluted. So would really appreciate it if someone could provide a better solution which basically replaces all... (3 Replies)
Discussion started by: svn
3 Replies

4. Shell Programming and Scripting

shell script to remove extra commas from CSV outp file

Name,,,,,,,,,,,,,,,,,,,,Domain,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Contact,Phone,Email,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Location -----------------------,------------------------------------------------,-------,-----,---------------------------------,------------------------------------ ----... (1 Reply)
Discussion started by: sreenath1037
1 Replies

5. Shell Programming and Scripting

Ascii Mode appending extra records to csv file

I am relatively new to this forum and Unix scripting. ksh script: part 1 :will call a PL\SQL program will create 3 CSV file at the unix directory. part 2 : will sftp the files to the EFT server. Once the EFT server receives these file , it will transfer them to a shared windows folders. ... (3 Replies)
Discussion started by: developerpa
3 Replies

6. Shell Programming and Scripting

Adding Extra Commas to a CSV file

Trying in this forum. Not sure if it is permitted.... but in need of help. Please find the requirements in the below link. https://www.unix.com/unix-dummies-questions-answers/191503-add-extra-commas-csv-file-2.html#post302665179 Thanks in Advance. (1 Reply)
Discussion started by: chillblue
1 Replies

7. UNIX for Dummies Questions & Answers

To Add extra commas to a CSV file using 2 files...

Hi , Based on my previous requirement the code works fine for comma as delimiter. Now my Req is widened up a bit .. There will be two set of files .. one with comma as delimiter and other with semi-colon ; as delimiter. Second Sample file. With Double Quotes (Semi-Colon... (1 Reply)
Discussion started by: chillblue
1 Replies

8. UNIX for Dummies Questions & Answers

To Add extra commas to a CSV file.

Hi All, I got this requirement to process a complex CSV file. Eg File. Line 1: Name:,XYz Line 2: Age:,15 Line 3: Grade:,7 Line 4: Line 5: English, Maths, Science,Spanish Line 6:10,11,13,14 As you can see the maximum column is 4 . The file i need to make is Line 1: Name:,XYz,,... (12 Replies)
Discussion started by: chillblue
12 Replies

9. Shell Programming and Scripting

Randomly inserting extra columns into csv file

Hi Tech Guru, I have a test file as below , which needs some more fields to be populated randomly : dks3243;12;20130823;1420;25m;0;syt dks3243;rocy;10 dks3243;kiop;18 sde21p4;77;20151210;8479;7py;9;vfr sde21p4;temp;67 sfq6i01;12;20120123;3412;4rd;7;jui sfq6i01;uymk;90 sfq6i01;kiop;51 ... (8 Replies)
Discussion started by: Lokesha
8 Replies

10. Shell Programming and Scripting

Need to Insert three extra columns in csv file

Hello Experts, I got a requirement i have a input file which am getting from different source,Now i want to add extra 3 columns to this file like BASE,ACTUAL and DATE. Input File Looks like QUAL CHGE TYP LAW COM1 COM2 A 1 X SED HO ASE B 3 Z CDE SE ... (5 Replies)
Discussion started by: ahmed.vaghar
5 Replies
ns_thread(3aolserver)					    AOLserver Built-In Commands 				     ns_thread(3aolserver)

__________________________________________________________________________________________________________________________________________________

NAME
ns_thread - commands SYNOPSIS
ns_thread begin script ns_thread begindetached script ns_thread get ns_thread getid ns_thread wait tid ns_thread yield _________________________________________________________________ DESCRIPTION
ns_thread begin: begins a new thread which evaluates the specified script and then exits. It returns a thread ID that must eventually be passed to ns_thread wait. (Failing to call ns_thread wait will eventually result in no new threads being created.) ns_thread begindetached: begins a detached thread that doesn't have to be (and can't be) waited for. ns_thread get: gets the thread ID of the current thread. The result is a thread ID that can be passed to ns_thread wait and may look something like "tid532". ns_thread getid: gets the thread integer number for the current thread. The result is a small integer used for identifying threads is a human-read- able way, such as "1" or "1120", for example. ns_thread wait: waits for the specified thread to exit. The tid argument is a thread ID returned by ns_thread begin or ns_thread get. ns_thread yield: causes the current thread to yield. EXAMPLES
This example is similar to the example under the ns_sockselect function of connecting to the 10 servers and waiting to service them with the ns_sockselect command. In this case, though, each connection gets it's own thread. # This is the procedure which is evaluated for each thread and # handles a single connection to host number $i proc getpage {i} { global pages # new thread will start here - first connect to host set host [format "www%2d.foo.com" $i] set fds [ns_sockopen $host 80 set r [lindex $fds 0] set w [lindex $fds 1] # next, send request 0r" puts $w "GET /index.htm HTTP/1.0 flush $w # then read page set pages($i) [read $r] # and close sockets close $w close $r # thread goes away here and other threads waiting # on ns_thread wait will wakeup } # Here's the loop which creates the threads which run getpage. for {set i 1} {$i < 9} {incr i} { set tids($i) [ns_thread begin "getpage $i"] } # wait for the threads to exit and then process the pages for {set i 1} {$i < 9} {incr i} { ns_thread wait $tids($i) # output page ... process the page in $pages($i) put there by other thread ... } Note that the code here is much simpler to follow than the ns_sockselect example; that's the benefit of multithreaded programming. However, it uses more resources as threads need to be created and initialized. This can be a problem if you plan to create many threads. SEE ALSO
KEYWORDS
threads AOLserver 4.0 ns_thread(3aolserver)
All times are GMT -4. The time now is 12:34 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy