Sponsored Content
Top Forums UNIX for Advanced & Expert Users cat / sed process weird characters Post 302544441 by methyl on Wednesday 3rd of August 2011 07:23:25 PM
Old 08-03-2011
Untested idea to clean the file by removing any characters except 0-9 A-Z a-z commas and newlines using the unix "tr" command.

Code:
cat ~/Downloads/log.txt | while read old_line
do
         echo "${old_line}"| tr -dc '[0-9][A-Z][a-z],\n'
done

If this does not solve your problem, please post sample data which shows a couple of good lines and a couple of bad lines when displayed by the unix "od" command (which will show exactly what characters are in the file). We don't need the whole file.
Code:
cat ~/Downloads/log.txt | od -xc

This User Gave Thanks to methyl For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to get rid of all the weird characters and color on bash shell

Does anyone of you know how to turn off color and weird characters on bash shell when using the command "script"? Everytime users on my server used that command to record their script, they either couldn't print it because lp kept giving the "unknown format character" messages or the print paper... (1 Reply)
Discussion started by: Micz
1 Replies

2. Shell Programming and Scripting

Weird Ascii characters in file names

Hi. I have files in my OS that has weird file names with not-conventional ascii characters. I would like to run them but I can't refer them. I know the ascii # of the problematic characters. I can't change their name since it belongs to a 3rd party program... but I want to run it. is there... (2 Replies)
Discussion started by: yamsin789
2 Replies

3. Shell Programming and Scripting

weird issue about h, g, x in SED

I have a file called merge2.t: Hi Hello how are you. </Endtag> <New> I am fine.</New> This is a test. freelong how Here is the SED: sed -n ' /<\/Endtag>/ !{ H } /<\/Endtag>/ { x p } (4 Replies)
Discussion started by: freelong
4 Replies

4. Shell Programming and Scripting

long process listing with /usr/ucb/ps weird behaves

hello I am trying to run the following script to get the my-progam pid: #!/bin/ksh tt=`/usr/ucb/ps| grep -i $1| grep -v grep | awk '{print $2}'` echo $tt When I run the script I get the more PIDs $./test.sh my-program 12033 15033 15034 Actually my-program's PID is 12033....I... (6 Replies)
Discussion started by: sreeniatbp
6 Replies

5. Shell Programming and Scripting

cat file_list | [script to print last some characters]

Hello guys, I have a list of files. For example: /disk1/mediator_home/tmp/ntest/TSFILE00.8256.GGG1-U.0908250009.unp.20090824P8.is /disk1/mediator_home/tmp/ntest/TSFILE00.8257.GGG1-U.0908250013.unp.20090825P1.is... (2 Replies)
Discussion started by: mr_bold
2 Replies

6. Shell Programming and Scripting

Extra control characters being added when I create a file using cat command

Hi, I am using Cygwin.I created a new file and type into it using cat > newfile. When I open this using vi editor, it contains loads of extra control characters. Whats happening? (1 Reply)
Discussion started by: erora
1 Replies

7. Shell Programming and Scripting

share a shell script which can replace weird characters in directory or file name

I just finish the shell script . This shell can replace weird characters (such as #$%^@!'"...) in file or directory name by "_" I spent long time on replacing apostrophe in file/directory name added: 2012-03-14 the 124th line (/usr/bin/perl -i -e "s#\'#\\'#g" /tmp/rpdir_level$i.tmp) is... (5 Replies)
Discussion started by: begonia
5 Replies

8. Shell Programming and Scripting

Weird ^M characters is disturbing the paste command

Dear all, I have the files: xaa xab xac and I try to paste them using $paste -d, xaa xab xac I see: output 3e-130 ,6e-78 ,5e-74 6e-124 ,0,007 ,0,026 2e-119 When I type: $ paste -d, xaa xab xac |less I see: output 3e-130^M,6e-78^M,5e-74 6e-124^M,0,007^M,0,026 (2 Replies)
Discussion started by: valente
2 Replies

9. Shell Programming and Scripting

Control characters -weird problem

I am using Korn shell on Linux 2.6x platform , and I am suing the following code to capture the lines which contain CONTROL CHARACTERS in my file : awk '/]/ {print NR}' EROLLMENT_INPUT.txt The problem is that this code shows the file has control characters when the file is in folder A ,... (2 Replies)
Discussion started by: kumarjt
2 Replies

10. Shell Programming and Scripting

How to find out the weird blank characters?

I have a text file downloaded from the web, I want to count the unique words used in the file, and a person's speaking length during conversation by counting the words between the opening and closing quotation marks which differ from the standard ASCII code. Also I found out the file contains some... (2 Replies)
Discussion started by: yifangt
2 Replies
SCRIPT(1)							   User Commands							 SCRIPT(1)

NAME
script - make typescript of terminal session SYNOPSIS
script [options] [file] DESCRIPTION
script makes a typescript of everything displayed on your terminal. It is useful for students who need a hardcopy record of an interactive session as proof of an assignment, as the typescript file can be printed out later with lpr(1). If the argument file is given, script saves the dialogue in this file. If no filename is given, the dialogue is saved in the file type- script. OPTIONS
-a, --append Append the output to file or to typescript, retaining the prior contents. -c, --command command Run the command rather than an interactive shell. This makes it easy for a script to capture the output of a program that behaves differently when its stdout is not a tty. -e, --return Return the exit code of the child process. Uses the same format as bash termination on signal termination exit code is 128+n. -f, --flush Flush output after each write. This is nice for telecooperation: one person does `mkfifo foo; script -f foo', and another can supervise real-time what is being done using `cat foo'. --force Allow the default output destination, i.e. the typescript file, to be a hard or symbolic link. The command will follow a symbolic link. -q, --quiet Be quiet (do not write start and done messages to standard output). -t[file], --timing[=file] Output timing data to standard error, or to file when given. This data contains two fields, separated by a space. The first field indicates how much time elapsed since the previous output. The second field indicates how many characters were output this time. This information can be used to replay typescripts with realistic typing and output delays. -V, --version Display version information and exit. -h, --help Display help text and exit. NOTES
The script ends when the forked shell exits (a control-D for the Bourne shell (sh(1)), and exit, logout or control-d (if ignoreeof is not set) for the C-shell, csh(1)). Certain interactive commands, such as vi(1), create garbage in the typescript file. script works best with commands that do not manipulate the screen, the results are meant to emulate a hardcopy terminal. It is not recommended to run script in non-interactive shells. The inner shell of script is always interactive, and this could lead to unexpected results. If you use script in the shell initialization file, you have to avoid entering an infinite loop. You can use for example the .profile file, which is read by login shells only: if test -t 0 ; then script exit fi You should also avoid use of script in command pipes, as script can read more input than you would expect. ENVIRONMENT
The following environment variable is utilized by script: SHELL If the variable SHELL exists, the shell forked by script will be that shell. If SHELL is not set, the Bourne shell is assumed. (Most shells set this variable automatically). SEE ALSO
csh(1) (for the history mechanism), scriptreplay(1) HISTORY
The script command appeared in 3.0BSD. BUGS
script places everything in the log file, including linefeeds and backspaces. This is not what the naive user expects. script is primarily designed for interactive terminal sessions. When stdin is not a terminal (for example: echo foo | script), then the session can hang, because the interactive shell within the script session misses EOF and script has no clue when to close the session. See the NOTES section for more information. AVAILABILITY
The script command is part of the util-linux package and is available from Linux Kernel Archive <https://www.kernel.org/pub/linux/utils /util-linux/>. util-linux June 2014 SCRIPT(1)
All times are GMT -4. The time now is 08:29 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy