Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Sed: Delete lines in files that contain other than a-z ,0-9 and "."

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Sed: Delete lines in files that contain other than a-z ,0-9 and "."
# 1  
Old 10-04-2009
CPU & Memory Sed: Delete lines in files that contain other than a-z ,0-9 and "."
Sed: Delete lines in files that contain other than 'a-z' ,'0-9', '.' and '-'

Hello,

I'm looking for a shell command or maybe a small php loop to delete lines in files.txt (in the same directory) that contain character other then 'a-z' ,'0-9', '.' and '-'

All line that have characters like éèÈ etc... will got his line deleted. I don't want to see the output (it's larges files +- 5meg, and +- 100 files)

It's probably a combinasion of Sed and Regex but i'm unable to find the good syntax to do it Smilie

Every help will be appreciated.

Thanks
# 2  
Old 10-04-2009
Hi.

You mentioned "files.txt" and "100 files". Can you be more specific about from which file(s) the text should be deleted?

(assuming all files in directory...)
bash code:
  1. ls | while read FILE; do
  2.   sed -n "/^[a-z0-9.-]\+$/ p" $FILE > FILE.tmp.$$
  3.   cp -f $FILE.tmp.$$ $FILE && rm $FILE.tmp.$$
  4. done

Or
bash code:
  1. ls | while read FILE; do
  2.   grep "^[a-z0-9.-]\+$" $FILE > $FILE.tmp.$$
  3.   cp -f $FILE.tmp.$$ $FILE && rm $FILE.tmp.$$
  4. done

Edit: Highlight=bash bbcode added by neo .....
Last edited by Scott; 10-04-2009 at 04:55 PM.. Reason: Always take a backup before something like this!
# 3  
Old 10-04-2009
Thanks for your reply Scottn

Your code give me the biggest hint for the last 3 days Smilie

My files are in fact sitemap, like sitemap.1.xml, sitemap.2.xml, sitemap.3.xml, ...
and forgot to mention that I also need to include '<', '>', ':', '/'

I tried to use this code but the ':' is not correctly set in this line I think...

(not working correctly)
Code:
sed -n "/^[<>\:a-z0-9.-\/]\{1,\}$/ p" sitemap.1.xml > sitemap.1.xml.tmp;mv sitemap.1.xml.tmp sitemap.1.xml

# 4  
Old 10-04-2009
Hi.

Sed does seem to be somewhat pedantic about where bits go!

bash code:
  1. ls sitemap.*.xml | while read FILE; do
  2.   sed -n "/^[a-z<>/0-9.:-]\+$/ p" $FILE > FILE.tmp.$$
  3.   cp -f $FILE.tmp.$$ $FILE && rm $FILE.tmp.$$
  4. done
# 5  
Old 10-04-2009
Nice !!
Thanks Smilie

Didn't know about the position thing..!

Thanks for your great help
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed not applying /d "delete line" option

So I'm on an AIX machine. And sed is not applying /d "delete line" option when I also include match word options \< and \> examples... echo cat | sed '/\<cat\>/d'will return cat for some reason echo cat | sed "/\<cat\>/d"will also still return cat. Of course i can just run echo cat... (9 Replies)
Discussion started by: escooter87
9 Replies

2. Shell Programming and Scripting

Delete all log files older than 10 day and whose first string of the first line is "MSH" or "<?xml"

Dear Ladies & Gents, I have a requirement to delete all the log files in /var/log/test directory that are older than 10 days and their first line begin with "MSH" or "<?xml" or "FHS". I've put together the following BASH script, but it's erroring out: for filename in $(find /var/log/test... (2 Replies)
Discussion started by: Hiroshi
2 Replies

3. Shell Programming and Scripting

Find lines with "A" then change "E" to "X" same line

I have a bunch of random character lines like ABCEDFG. I want to find all lines with "A" and then change any "E" to "X" in the same line. ALL lines with "A" will have an "X" somewhere in it. I have tried sed awk and vi editor. I get close, not quite there. I know someone has already solved this... (10 Replies)
Discussion started by: nightwatchrenba
10 Replies

4. Shell Programming and Scripting

Delete till ">" is found in all lines in a file

Hi, I have a file which has lines like these : I want to trim everything from the left till ">" such that the file looks like : If you have any ideas how to do this in 1-2 commands please help. Thanks. (3 Replies)
Discussion started by: sinpeak
3 Replies

5. Shell Programming and Scripting

how to use "cut" or "awk" or "sed" to remove a string

logs: "/home/abc/public_html/index.php" "/home/abc/public_html/index.php" "/home/xyz/public_html/index.php" "/home/xyz/public_html/index.php" "/home/xyz/public_html/index.php" how to use "cut" or "awk" or "sed" to get the following result: abc abc xyz xyz xyz (8 Replies)
Discussion started by: timmywong
8 Replies

6. Shell Programming and Scripting

How to print range of lines using sed when pattern has special character "["

Hi, My input has much more lines, but few of them are below pin(IDF) { direction : input; drc_pinsigtype : signal; pin(SELDIV6) { direction : input; drc_pinsigtype : ... (3 Replies)
Discussion started by: nehashine
3 Replies

7. Shell Programming and Scripting

Delete files older than "x" if directory size is greater than "y"

I wrote a script to delete files which are older than "x" days, if the size of the directory is greater than "y" #!/bin/bash du -hs $1 while read SIZE ENTRY do if ; then find $1 -mtime +$2 -exec rm -f {} \; echo "Files older than $2 days deleted" else echo "free Space available"... (4 Replies)
Discussion started by: JamesCarter
4 Replies

8. Shell Programming and Scripting

Unix commands delete all files starting with "X" except "X" itself. HELP!!!!?

im a new student in programming and im stuck on this question so please please HELP ME. thanks. the question is this: enter a command to delete all files that have filenames starting with labtest, except labtest itself (delete all files startign with 'labtest' followed by one or more... (2 Replies)
Discussion started by: soccerball
2 Replies

9. Shell Programming and Scripting

cat $como_file | awk /^~/'{print $1","$2","$3","$4}' | sed -e 's/~//g'

hi All, cat file_name | awk /^~/'{print $1","$2","$3","$4}' | sed -e 's/~//g' Can this be done by using sed or awk alone (4 Replies)
Discussion started by: harshakusam
4 Replies

10. Shell Programming and Scripting

Delete lines ending in "_;" using sed

I could really use some help with this issue. I'm having a lot of trouble getting my sed command to delete only the lines from my file that end with _; I'm also supposed to carry the leading 'c' down to the next line. The commands I've tried either delete everything or nothing at all. Any help... (12 Replies)
Discussion started by: turbulence
12 Replies
Login or Register to Ask a Question