10-28-2008
Lines with strange characters and sed...
Dear All:
I Have a bunch of files which I'd like to process with a shell script. The problem is that the files have strange characters in their headers, like
�g�8@L-000-MSG2__-ABCD________-FIRA_____-000001___-200806181330-__
��e�
Data from BLABLABLA, Instrument: BLABLA, Date: 2008/06/18 13:30Z
Row: 1078 Col: 1130 Lat: -22.267 Lon: 22.256 *** Something here ***
For my intents, I only need the information (in this case) from line 3 onwards. Sometimes this strange header occupies 2 lines, others 3...others...I don't know.
I made a very simple test, like
FILE=`find . -type f -name "FILENAME"`
for i in $FILE
do
FNOW=`echo $i`
#Cuts two first lines of the file
sed '1,2d' $FNOW > newfile
sed '/^$/d' -i newfile
HEADER=`head -1 newfile | cut -c1-4`
if [ "$HEADER" != "Data" ]
then
sed '1d' -i newfile
sed '/^$/d' -i newfile
fi
#A simple testing
HEADER2=`head -1 newfile | cut -c1-4`
echo ${HEADER2},${HEADER} >> test.txt
done
The problem is that.....sometimes i don't get to cut all the "strange" headers to obtain "clean" files, as you can see in some lines of test.txt
Data,@H
Data,ۘ
Data,Data
Data,@H
(etc)
So:
Is there any way to fulfill my intentions with sed? Maybe some "delete all the first lines until find the expression «Data»? Honestly, i don't know what else to try.
Thank you very much in advance
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi folks. None of the conventional methods are working for my dilemma:
I have a file in my root directory that has a name comprised of strange characters. When I do an ls, it just hangs at that file until I do a Cntrl-C.
rm ./filename
&
rm \filename
do not work. I am entering the... (4 Replies)
Discussion started by: kristy
4 Replies
2. Shell Programming and Scripting
Hello ,
I'm trying to split a file which contains a single very long line.
My aim is to split this single line each 120 characters.
I tried with the sed command :
`cat ${MYPATH}/${FILE}|sed -e :a -e 's/^.\{1,120\}$/&\n/;ta' >{MYPATH}/${DEST}`
but when I wc -l the destination file it is... (2 Replies)
Discussion started by: jerome_1664
2 Replies
3. Shell Programming and Scripting
Hello all,
I'm new to UNIX and new to this forum, so forgive my lack of knowledge. I'm new with editing in vi so I FTP scripts to a Windows machine and edit the script in notepad (when I need to do something quickly). Sometimes when I FTP the script back to the UNIX box, strange characters... (4 Replies)
Discussion started by: dgower2
4 Replies
4. Shell Programming and Scripting
Hi, I have a bash script and I am looking for a command that will merge specific lines together.
Sample Data:
registration time = 1300890272
Id = 1
setd = 0
tagunt = 26
tagId=6, length=8, value=
tagId=9, length=5, value=
tagId=7, length=2, value=
tagId=16, length=2, value=
tagId=32,... (8 Replies)
Discussion started by: Winsarc
8 Replies
5. Shell Programming and Scripting
I've written a script:
find -depth | awk ‘
{
if ( substr($1,length($0)-2,3) == “/1.” )
{ print $1 }
{ system(“awk -f test1.awk “ $1 ) }
}
‘
The idea is that it trundles through a large directory structure looking for files which are named '1.' and then... (3 Replies)
Discussion started by: nashcom
3 Replies
6. Shell Programming and Scripting
Hello unix users :)
I am trying to grep a string from a file that both the file and the string may have characters in them that are quite... strange, like würzburger.
Well, bash reads this as
W%C3%BCrzburger
For example, if i do
wget W%C3%BCrzburger
the output is:
--2012-01-08... (2 Replies)
Discussion started by: hakermania
2 Replies
7. Red Hat
I am trying to sftp a textfile from windows to linux. The file includes some spanish characters. When I vi the file in LINUX, the special (spanish) characters get converted into some strange characters. anyone know how i can resolve this? for example México gets converted into México on LINUX. (0 Replies)
Discussion started by: mrx1350
0 Replies
8. Hardware
Hello,
I have an x86 server with an ILOM connection that produces strange characters when I perform a start /SP/console, see below:
Oracle(R) Integrated Lights Out Manager
Version 3.0.16.10.a r68533
Copyright (c) 2011, Oracle and/or its affiliates. All rights reserved.
-> start... (9 Replies)
Discussion started by: kerrygold
9 Replies
9. Shell Programming and Scripting
sed -e "s// /g" old.txt > new.txt
While I do know some control characters need to be escaped, can normal characters also be escaped and still work the same way? Basically I do not know all control characters that have a special meaning, for example, ?, ., % have a meaning and have to be escaped... (11 Replies)
Discussion started by: ijustneeda
11 Replies
10. Programming
Hi guys,
After compiling a .f90 code and executing it, i get strange characters in the output file like :
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
Are these windows characters? how can i get rid of this?
Much appreciated.
Paul (1 Reply)
Discussion started by: Paul Moghadam
1 Replies
LEARN ABOUT DEBIAN
test::data
Data(3pm) User Contributed Perl Documentation Data(3pm)
NAME
Test::Data -- test functions for particular variable types
SYNOPSIS
use Test::Data qw(Scalar Array Hash Function);
DESCRIPTION
Test::Data provides utility functions to check properties and values of data and variables.
Functions
Plug-in modules define functions for each data type. See the appropriate module.
How it works
The Test::Data module simply imports functions from Test::Data::* modules. Each module defines a self-contained function, and puts that
function name into @EXPORT. Test::Data defines its own import function, but that does not matter to the plug-in modules.
If you want to write a plug-in module, follow the example of one that already exists. Name the module Test::Data::Foo, where you replace
Foo with the right name. Test::Data should automatically find it.
BUGS
I'm not a very good Windows Perler, so some things don't work as they should on Windows. I recently got a Windows box so I can test things,
but if you run into problems, I can use all the patches or advice you care to send.
SEE ALSO
Test::Data::Scalar, Test::Data::Array, Test::Data::Hash, Test::Data::Function, Test::Builder
SOURCE AVAILABILITY
This source is in Github:
http://github.com/briandfoy/test-data/tree/master
AUTHOR
brian d foy, "<bdfoy@cpan.org>"
COPYRIGHT AND LICENSE
Copyright (c) 2002-2009 brian d foy. All rights reserved.
This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
perl v5.14.2 2009-02-12 Data(3pm)