Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: New Line in CSV file
Top Forums Shell Programming and Scripting New Line in CSV file Post 302967564 by Don Cragun on Thursday 25th of February 2016 02:02:22 AM
Old 02-25-2016
UNIX format text files have a newline character as a line terminator. DOS format text file have a carriage-return character immediately followed by a newline character with the combination of those two characters being the line terminator.

The first file you asked about (INPUT_GIANT_JAN2016.txt) was not a normal DOS format text file. It only had the carriage-return character as the line terminator (except for the last line in the file that had both carriage-return and newline characters). So to fix it, all we had to do was change the carriage-return characters to newline characters. (That left you with a single empty line at the end of the file after tr changed all of the carriage-return characters to newline characters.)

The second file (INPUT_GUARDIAN_JAN2016.txt) is a normal DOS format text file. So instead of changing carriage-return characters to newline characters, we just need to remove the carriage-return characters:
Code:
tr -d '\r' < INPUT_GUARDIAN_JAN2016.txt > NEW_INPUT_GUARDIAN_JAN2016.txt

If you only have files in these two exact formats, you could handle both with:
Code:
tr '\r' '\n' < input_file | grep -v '^$' > fixed_file

This won't work for general text files (which frequently have empty lines separating paragraphs), but it should work for the samples you have shown us so far.
This User Gave Thanks to Don Cragun For This Post:
aimy
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Checking the Empty line in csv file

Hi all, I have one CSV file(MasterFile.csv) consists of two columns. "./incoming/ABC.CSV","./incoming/ABC_CONTROL.txt" "./incoming/PQR.CSV","./incoming/PQR_CONTROL.txt" "./incoming/123.CSV","./incoming/123_CONTROL.txt" I have written a script to read the MasterFile.csv.Here i want to... (4 Replies)
Discussion started by: sollins
4 Replies

2. Shell Programming and Scripting

how to start looping from the second line in .csv file

I have a .csv file and i use the below while loop to navigate through it But i need to loop from the second line since the first line is the header How will i do it?? please help while IFS=, read Filename Path size readonly do echo "Filename -> ${Filename}" echo "Path -> ${Path}" echo... (8 Replies)
Discussion started by: codeman007
8 Replies

3. Shell Programming and Scripting

How to remove line break in a csv file

Hi Experts, My requirement is to read the csv file and need to remove if any line break in it. sample data: Row1: "Oslo, Symra kino",Oslo,130-7,Symra 1,130-7-91 Row2:"Tønsberg, Brygga Kino SF",Tønsberg,202-1, Tønsberg SF 4,202-1-4 Expected data: Row1: "Oslo, Symra... (6 Replies)
Discussion started by: cnraja
6 Replies

4. Shell Programming and Scripting

Read csv file line by line

Folks , i want to read a csv file line by line till the end of file and filter the text in the line and append everything into a variable. csv file format is :- trousers:shirts,price,50 jeans:tshirts,rate,60 pants:blazer,costprice,40 etc i want to read the first line and get... (6 Replies)
Discussion started by: venu
6 Replies

5. UNIX for Dummies Questions & Answers

Delete first line of a csv file

Hi there How do I delete the first line of a csv file without creating a new file ? Regards, Peter (4 Replies)
Discussion started by: nagileon
4 Replies

6. UNIX for Dummies Questions & Answers

skip first line when doing a read of csv file

Folks, how do i skip the first line in a csv, while doing the read of a csv file in to a variable line by line. eg : do echo $line done < $rpt where rpt is path to csv file The initial 1st line is a garbage that i want to avoid, and start reading from 2nd line ... (2 Replies)
Discussion started by: venu
2 Replies

7. Shell Programming and Scripting

Reading last line of a CSV file

Hi I have a file which I am reading line by line and processing it. But the last line is not getting read in the file loop until I put an enter in the end. #!/bin/ksh -p v_org_id=${P1} FILE=${P2} NEW_FILE_NAME=$APPLPTMP/b1.txt BAKIFS=$IFS IFS=$'\n' exec 0<"$FILE" echo "File to be... (2 Replies)
Discussion started by: Chinky23
2 Replies

8. Shell Programming and Scripting

Split csv file by line count.

I have a very large csv file that I sort by the data that is in the second column. But what I need to do next is split the file in groups of say around 30,000 lines but don't split the data while there is still like data in the in the second column. Here is some of the data. ... (2 Replies)
Discussion started by: GroveTuckey
2 Replies

9. Shell Programming and Scripting

Honey, I broke awk! (duplicate line removal in 30M line 3.7GB csv file)

I have a script that builds a database ~30 million lines, ~3.7 GB .cvs file. After multiple optimzations It takes about 62 min to bring in and parse all the files and used to take 10 min to remove duplicates until I was requested to add another column. I am using the highly optimized awk code: awk... (34 Replies)
Discussion started by: Michael Stora
34 Replies

10. Shell Programming and Scripting

Column with New Line in CSV file

Hello, Got a CSV file which contains 21 columns Need to convert the file to Pipe delimiter and Few columns text data contains new line Example 1,2,3,"ABC" 8" ABC,5,6,7 1,2,3,"ABC" 8" ABC,5,6,7 ( New Line) 1,2,3,""ABC" 8" ABC", 5,6,7 1,2,3,"ABC" ,5,6,7(New line) Expected... (8 Replies)
Discussion started by: krux_rap
8 Replies

LEARN ABOUT DEBIAN

rdf::trine::exporter::csv

RDF::Trine::Exporter::CSV(3pm)				User Contributed Perl Documentation			    RDF::Trine::Exporter::CSV(3pm)

NAME

       RDF::Trine::Exporter::CSV - Export RDF data to CSV

VERSION

       This document describes RDF::Trine::Exporter::CSV version 1.000

SYNOPSIS

	use RDF::Trine::Exporter::CSV;

DESCRIPTION

       The RDF::Trine::Exporter::CSV class provides an API for serializing RDF data to CSV strings and files.

METHODS

       "new ( sep_char => $sep_char, quote => $bool )"
	   Returns a new RDF::Trine::Exporter::CSV object. If $sep_char is provided, it is used as the separator character in CSV serialization,
	   otherwise a comma (",") is used.

       "serialize_iterator_to_file ( $file, $iterator )"
	   Serializes the bindings objects produced by $iterator, printing the results to the supplied filehandle "<$fh">.

       "serialize_iterator_to_string ( $iterator )"
	   Serializes the bindings objects produced by $iterator, returning the result as a string.

BUGS

       Please report any bugs or feature requests to through the GitHub web interface at <https://github.com/kasei/perlrdf/issues>.

AUTHOR

       Gregory Todd Williams  "<gwilliams@cpan.org>"

COPYRIGHT

       Copyright (c) 2006-2012 Gregory Todd Williams. This program is free software; you can redistribute it and/or modify it under the same terms
       as Perl itself.

perl v5.14.2							    2012-06-29					    RDF::Trine::Exporter::CSV(3pm)
All times are GMT -4. The time now is 04:57 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy