Sponsored Content
Operating Systems Linux File conversion and removing special characters from a file in Linux Post 302863485 by dhruuv369 on Monday 14th of October 2013 11:22:23 AM
Old 10-14-2013
File conversion and removing special characters from a file in Linux

I have a .CSV file when I check for the special characters in the file using the command
Code:
cat -vet filename.csv

, i get very lengthy lines with "^@", "^I^@" and "^@^M" characters in between each alphabet in all of the records. Using the code below
Code:
file filename.csv

I get the output as
Quote:
filename.csv: Little-endian UTF-16 Unicode English character data, with very long lines, with CRLF, CR line terminators
I have a script to remove the control M (^M) from the file, whose output returns me an error saying : cannot execute binary file.

I know that ^I represent a tab. I also have a script to convert ^I to comma delimited file but Can anyone help me format the file with respect to the error and also ^@.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing special characters in file

I have file special.txt with the following data. <header info> 123$ty5%98&0asd 1@356fgbv78 09*&^5jkns43( ...........some more rows. In my output file, I want to eliminate all the special characters in my file and I want all other data. need some help. (6 Replies)
Discussion started by: srivsn
6 Replies

2. Shell Programming and Scripting

removing special characters @ EOL

How to remove special chracters @ END OF EACH LINE in a file file1.txt: 0003073413^M 0003073351^M 0003073379^M 0003282724^M 0003323334^M 0003217159^M 0003102760^M 0002228911^M I used the below command but it is not working ? perl -pi -e 's/^M\/g' file1.txt (6 Replies)
Discussion started by: ali560045
6 Replies

3. AIX

Removing a filename which has special characters passed from a pipe with xargs

Hi, On AIX 5200-07-00 I have a find command as following to delete files from a certain location that are more than 7 days old. I am being told that I cannot use -exec option to delete files from these directories. Having said that I am more curious to know how this can be done. an sample... (3 Replies)
Discussion started by: jerardfjay
3 Replies

4. Solaris

removing special characters, white spaces from a field in a file

what my code is doing, it is executing a sql file and the resullset of the query is getting stored in the text file in a fixed format. for that fixed format i have used the following code:: Code: awk -F":"... (2 Replies)
Discussion started by: priyanka3006
2 Replies

5. Shell Programming and Scripting

Removing special characters

Dear Friends, I want to remove text between two patters. Problem is, it has random special characters like \ / | * ` ~ ! $ etc. These random special characters has no fixed length. But these special characters are appearing between a fixed pattern e.g. DM&^%#|#!\/?CT Expected output... (14 Replies)
Discussion started by: anushree.a
14 Replies

6. Shell Programming and Scripting

Removing Special Character from File.

Hi, My file has this special character "^M" I would like to remove this characters. eg: abc,abc,^M i tried using sed but doesnt work. i used octal dump command to see special character it returns following: 015 \r Appreciate your reply. (6 Replies)
Discussion started by: pinnacle
6 Replies

7. UNIX for Dummies Questions & Answers

awk for removing special characters and extra commas

Hi, I have a .csv file which as empty lines with comma and some special characters in 3rd column as below. Source data 1,2,3,4,%#,6 ,,,,,, 1,2,3,4,5,6 Target Data 1,2,3,4,5,6I need to remove blank lines and special charcters I am trying to get this using the below awk awk -F","... (2 Replies)
Discussion started by: shruthidwh
2 Replies

8. Shell Programming and Scripting

Removing special characters - Control M

I have developed a small script to remove the Control M characters that get embedded when we move any file from Windows to Unix. For some reason, its not working in all scenarios. Some times I still see the ^M not being removed. Is there anything missing in the script: cd ${inputDir}... (7 Replies)
Discussion started by: vskr72
7 Replies

9. Shell Programming and Scripting

Removing blank/white spaces and special characters

Hello All , 1. I am trying to do a task where I need to remove Blank spaces from my file , I am usingawk '{$1=$1}{print}' file>file1Input :- ;05/12/1990 ;31/03/2014 ; Output:- ;05/12/1990 ;31/03/2014 ;This command is not removing all spaces from... (6 Replies)
Discussion started by: himanshu sood
6 Replies

10. UNIX for Beginners Questions & Answers

Replacing string/special characters using a 'conversion' table

Hi, Does anyone know if there is a script or program available out there that uses a conversion table to replace special characters from a file? I am trying to remove some special characters from a file but there are several unprintable/control characters that some I need to remove but some I... (2 Replies)
Discussion started by: newbie_01
2 Replies
CSV2LATEX(1)						      General Commands Manual						      CSV2LATEX(1)

NAME
csv2latex -- convert a csv file into a LaTeX document SYNOPSIS
csv2latex [--nohead] [--longtable] [--noescape] [--guess] [--separator c|s|t|p|l] [--block q|d|n] [--lines #] [--position l|c|r] [--colorrows 0-1] [--reduce 1|2|3|4] [--repeatheader] [--nohlines] [--novlines] [file] DESCRIPTION
This manual page documents the csv2latex program. csv2latex is a program that reads a "comma separated values" (csv) file and outputs a LaTeX file with one or more tabular environments to display the printable values of the csv file. The LaTeX code is flushed on the standard output. So-called "comma separated values" files are common formats for exchanging two-dimensinal tables between programs such as spreadsheets edi- tors, to represent almost any kind of data. By default, a csv file is made of printable data separated by commas (`,'), each comma repre- senting a `cell' separator, and each line representing a row. By extension, cell separators can be represented by tabs if the comma is con- sidered as printable data. Moreover, some non true csv files can be assumed as two-dimensional tables as well. In some circumstances, if the printable data includes the cell separator of the exchange format, the latter can use a second extra character to embrace the printable data into a block (e.g: quoted text). Thus, it is still possible to parse the file by using the block delimiter (used twice to embrace the cell) instead of the separator. csv2latex aims to parse various csv formats plus formats that fits into the above definiton, assuming the data is text, and to produce a yet simple LaTeX file using the "tabular" environment for a table-style layout. Some options of output will also use macros provided by extra LaTeX packages that are commonly included in the main LaTeX distributions. OPTIONS
This program follows the usual GNU command line syntax, with long options starting with two dashes (`-'). A summary of options is included below. -h --help Show summary of options. -v --version Show version of program. -n --nohead Do not output the LaTeX document header. This is useful when the output is to be included as a separate file into the master document. -t --longtable uses the 'longtable' package instead of the 'tabular' one. This is useful when the input is long, with --lines 0 option. This option uses the extra `longtable' LaTeX package. If you also use --nohead option, do not forget to add the following line into the header of your master document: "usepackage{longtable}". -x --noescape Do not escape TeX control characters from the input. This is useful when the input contains already TeX code. -g --guess Try to guess the csv format. This is useful when the input is not strictly a comma separated set of printable data. For exam- ple, a line like %Foo, Bar%:%Wizz: Hey% may be parsed as "Foo, Bar" then "Wizz: Hey". -s c|s|t|p|l --separator c|s|t|p|l Set the given separator as cell separator of the csv format. `c' means a comma (default). `s' means a semicolon. `t' means a tab. `p' means a space. `l' means a colon. -b q|d|n --block q|d|n Set the given block delimiter that embraces the printable data of the csv format. `q' means a simple quote. `d' means a double quote. `n' means no quoting at all (default). -l # --lines # Force to output multiple tabulars, each having a limited number of lines. The given argument must be a POSITIVE INTEGER VALUE. This is useful when the number of input rows is too big to fit into a single papersheet. A good average for a4 paper is about 40 lines (default). 0 means infinity (actualy about 2 Giga lines). -p l|c|r --position l|c|r Set the text position in all cells at once. This simply uses one of the three basic cell formating options of the LaTeX tabular environment. `l' means left-aligned (default). `c' means centered. `r' means right-aligned. -c 0-1 --colorrows 0-1 Alternate white/gray rows on the LaTeX output, having the given graylevel. The given argument must be a REAL NUMBER BETWEEN 0 AND 1. 0 means black while 1 means white. A nice looking value is 0.75 when printed on white paper. This option uses the extra `colortbl' LaTeX package. If you also use --nohead option, do not forget to add the following line into the header of your mas- ter document: "usepackage{colortbl}". -r 1|2|3|4 --reduce 1|2|3|4 Reduce the size of the tabular and the font in the LaTeX output, given a reduction level. The given argument must be one of 1, 2, 3 or 4. The more the level is high, the more the tabular will appear small. This is useful to shrink the table width when the printable data is made of very long text. This option uses the extra `relsize' LaTeX package. If you also use --nohead option, do not forget to add the following line into the header of your master document: "usepackage{relsize}". -z --nohlines Do not output horizontal lines in the table(s). -y --novlines Do not output vertical lines in the table(s). -e --repeatheader Repeat the first row of the first table in every table. This is useful when the output is very long and separated in multiple tables. EXAMPLES
Create a PDF document with small text, alternate gray rows, 80 lines per table, from a guessed csv format of the january stats that my boss created with his super point-and-click spreadsheet program (which could not generate a PDF output!). csv2latex --guess --lines 80 --colorrows 0.75 --reduce 2 january_stats.csv > january_stats.tex && pdflatex january_stats.tex Quickly preview a phonebook from a file formated as "Surname" "Name" "Phone" "Cellular": csv2latex -s p -b d -l 42 phonebook-sorted.txt | latex SEE ALSO
tex (1), latex (1). CSV2LATEX(1)
All times are GMT -4. The time now is 03:37 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy