Sponsored Content
Top Forums UNIX for Advanced & Expert Users cat / sed process weird characters Post 302544443 by bencpeters on Wednesday 3rd of August 2011 07:47:42 PM
Old 08-03-2011
Methyl - your idea would work, but when I try it, I get an error: 'tr: illegal byte sequence'

Here is a sample data set:
(in plaintext):

Code:
3B698960,eSWPump,0
3B698C36,sHeatPre,1
3B698C36,ePHPump,1
3B698C36,eSWPump,1
3B698CB4,Ô!˛ˇ√8ÏÔ,1
3B698CB4,eHWRPump,1
3B698CB4,››ù√剃,1
3B698CB4,eDownHRV,1
3B698CB4,eUpHRV,1
3B698E91,sHeatPre,0
3B698E91,ePHPump,0
3B698E91,eSWPump,0

and


Code:
$ cat ~/Downloads/log.txt | od -xc
0000000      4233    3936    3938    3036    652c    5753    7550    706d
           3   B   6   9   8   9   6   0   ,   e   S   W   P   u   m   p
0000020      302c    000a    4233    3936    4338    3633    732c    6548
           ,   0  \n  \0   3   B   6   9   8   C   3   6   ,   s   H   e
0000040      7461    7250    2c65    0a31    4233    3936    4338    3633
           a   t   P   r   e   ,   1  \n   3   B   6   9   8   C   3   6
0000060      652c    4850    7550    706d    312c    330a    3642    3839
           ,   e   P   H   P   u   m   p   ,   1  \n   3   B   6   9   8
0000100      3343    2c36    5365    5057    6d75    2c70    0a31    4233
           C   3   6   ,   e   S   W   P   u   m   p   ,   1  \n   3   B
0000120      3936    4338    3442    ef2c    fe21    c3ff    ec38    2cef
           6   9   8   C   B   4   , 357   ! 376 377 303   8 354 357   ,
0000140      0a31    4233    3936    4338    3442    652c    5748    5052
           1  \n   3   B   6   9   8   C   B   4   ,   e   H   W   R   P
0000160      6d75    2c70    0a31    4233    3936    4338    3442    dd2c
           u   m   p   ,   1  \n   3   B   6   9   8   C   B   4   , 032
0000200      dd1a    c39d    e48c    2cc4    0a31    4233    3936    4338
         032 335 235   Ì  ** 344 304   ,   1  \n   3   B   6   9   8   C
0000220      3442    652c    6f44    6e77    5248    2c56    0a31    4233
           B   4   ,   e   D   o   w   n   H   R   V   ,   1  \n   3   B
0000240      3936    4338    3442    652c    7055    5248    2c56    0a31
           6   9   8   C   B   4   ,   e   U   p   H   R   V   ,   1  \n
0000260      4233    3936    4538    3139    732c    6548    7461    7250
           3   B   6   9   8   E   9   1   ,   s   H   e   a   t   P   r
0000300      2c65    0a30    4233    3936    4538    3139    652c    4850
           e   ,   0  \n   3   B   6   9   8   E   9   1   ,   e   P   H
0000320      7550    706d    302c    330a    3642    3839    3945    2c31
           P   u   m   p   ,   0  \n   3   B   6   9   8   E   9   1   ,
0000340      5365    5057    6d75    2c70    0030                        
           e   S   W   P   u   m   p   ,   0                            
0000351

thanks!

Last edited by bencpeters; 08-03-2011 at 08:49 PM.. Reason: wrong option on od...
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to get rid of all the weird characters and color on bash shell

Does anyone of you know how to turn off color and weird characters on bash shell when using the command "script"? Everytime users on my server used that command to record their script, they either couldn't print it because lp kept giving the "unknown format character" messages or the print paper... (1 Reply)
Discussion started by: Micz
1 Replies

2. Shell Programming and Scripting

Weird Ascii characters in file names

Hi. I have files in my OS that has weird file names with not-conventional ascii characters. I would like to run them but I can't refer them. I know the ascii # of the problematic characters. I can't change their name since it belongs to a 3rd party program... but I want to run it. is there... (2 Replies)
Discussion started by: yamsin789
2 Replies

3. Shell Programming and Scripting

weird issue about h, g, x in SED

I have a file called merge2.t: Hi Hello how are you. </Endtag> <New> I am fine.</New> This is a test. freelong how Here is the SED: sed -n ' /<\/Endtag>/ !{ H } /<\/Endtag>/ { x p } (4 Replies)
Discussion started by: freelong
4 Replies

4. Shell Programming and Scripting

long process listing with /usr/ucb/ps weird behaves

hello I am trying to run the following script to get the my-progam pid: #!/bin/ksh tt=`/usr/ucb/ps| grep -i $1| grep -v grep | awk '{print $2}'` echo $tt When I run the script I get the more PIDs $./test.sh my-program 12033 15033 15034 Actually my-program's PID is 12033....I... (6 Replies)
Discussion started by: sreeniatbp
6 Replies

5. Shell Programming and Scripting

cat file_list | [script to print last some characters]

Hello guys, I have a list of files. For example: /disk1/mediator_home/tmp/ntest/TSFILE00.8256.GGG1-U.0908250009.unp.20090824P8.is /disk1/mediator_home/tmp/ntest/TSFILE00.8257.GGG1-U.0908250013.unp.20090825P1.is... (2 Replies)
Discussion started by: mr_bold
2 Replies

6. Shell Programming and Scripting

Extra control characters being added when I create a file using cat command

Hi, I am using Cygwin.I created a new file and type into it using cat > newfile. When I open this using vi editor, it contains loads of extra control characters. Whats happening? (1 Reply)
Discussion started by: erora
1 Replies

7. Shell Programming and Scripting

share a shell script which can replace weird characters in directory or file name

I just finish the shell script . This shell can replace weird characters (such as #$%^@!'"...) in file or directory name by "_" I spent long time on replacing apostrophe in file/directory name added: 2012-03-14 the 124th line (/usr/bin/perl -i -e "s#\'#\\'#g" /tmp/rpdir_level$i.tmp) is... (5 Replies)
Discussion started by: begonia
5 Replies

8. Shell Programming and Scripting

Weird ^M characters is disturbing the paste command

Dear all, I have the files: xaa xab xac and I try to paste them using $paste -d, xaa xab xac I see: output 3e-130 ,6e-78 ,5e-74 6e-124 ,0,007 ,0,026 2e-119 When I type: $ paste -d, xaa xab xac |less I see: output 3e-130^M,6e-78^M,5e-74 6e-124^M,0,007^M,0,026 (2 Replies)
Discussion started by: valente
2 Replies

9. Shell Programming and Scripting

Control characters -weird problem

I am using Korn shell on Linux 2.6x platform , and I am suing the following code to capture the lines which contain CONTROL CHARACTERS in my file : awk '/]/ {print NR}' EROLLMENT_INPUT.txt The problem is that this code shows the file has control characters when the file is in folder A ,... (2 Replies)
Discussion started by: kumarjt
2 Replies

10. Shell Programming and Scripting

How to find out the weird blank characters?

I have a text file downloaded from the web, I want to count the unique words used in the file, and a person's speaking length during conversation by counting the words between the opening and closing quotation marks which differ from the standard ASCII code. Also I found out the file contains some... (2 Replies)
Discussion started by: yifangt
2 Replies
SCRIPT(1)							   User Commands							 SCRIPT(1)

NAME
script - make typescript of terminal session SYNOPSIS
script [options] [file] DESCRIPTION
script makes a typescript of everything printed on your terminal. It is useful for students who need a hardcopy record of an interactive session as proof of an assignment, as the typescript file can be printed out later with lpr(1). If the argument file is given, script saves all dialogue in file. If no file name is given, the typescript is saved in the file type- script. OPTIONS
-a, --append Append the output to file or typescript, retaining the prior contents. -c, --command command Run the command rather than an interactive shell. This makes it easy for a script to capture the output of a program that behaves differently when its stdout is not a tty. -e, --return Return the exit code of the child process. Uses the same format as bash termination on signal termination exit code is 128+n. -f, --flush Flush output after each write. This is nice for telecooperation: one person does `mkfifo foo; script -f foo', and another can supervise real-time what is being done using `cat foo'. --force Allow the default output destination, i.e. the typescript file, to be a hard or symbolic link. The command will follow a symbolic link. -q, --quiet Be quiet. -t, --timing[=file] Output timing data to standard error, or to file when given. This data contains two fields, separated by a space. The first field indicates how much time elapsed since the previous output. The second field indicates how many characters were output this time. This information can be used to replay typescripts with realistic typing and output delays. -V, --version Output version information and exit. -h, --help Output help and exit. NOTES
The script ends when the forked shell exits (a control-D to exit the Bourne shell (sh(1)), and exit, logout or control-d (if ignoreeof is not set) for the C-shell, csh(1)). Certain interactive commands, such as vi(1), create garbage in the typescript file. Script works best with commands that do not manipulate the screen, the results are meant to emulate a hardcopy terminal. ENVIRONMENT
The following environment variable is utilized by script: SHELL If the variable SHELL exists, the shell forked by script will be that shell. If SHELL is not set, the Bourne shell is assumed. (Most shells set this variable automatically). SEE ALSO
csh(1) (for the history mechanism), scriptreplay(1). HISTORY
The script command appeared in 3.0BSD. BUGS
Script places everything in the log file, including linefeeds and backspaces. This is not what the naive user expects. AVAILABILITY
The script command is part of the util-linux package and is available from Linux Kernel Archive <ftp://ftp.kernel.org/pub/linux/utils/util- linux/>. util-linux September 2011 SCRIPT(1)
All times are GMT -4. The time now is 07:38 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy