Sed: Delete lines in files that contain other than a-z ,0-9 and "."

devlin · #1 (**permalink**) 10-04-2009

Sed: Delete lines in files that contain other than 'a-z' ,'0-9', '.' and '-'

Hello,

I'm looking for a shell command or maybe a small php loop to delete lines in files.txt (in the same directory) that contain character other then 'a-z' ,'0-9', '.' and '-'

All line that have characters like éèÈ etc... will got his line deleted. I don't want to see the output (it's larges files +- 5meg, and +- 100 files)

It's probably a combinasion of Sed and Regex but i'm unable to find the good syntax to do it

Every help will be appreciated.

Thanks

scottn · #2 (**permalink**) 10-04-2009

Hi.

You mentioned "files.txt" and "100 files". Can you be more specific about from which file(s) the text should be deleted?

(assuming all files in directory...)

bash Code:

ls | while read FILE; do
  sed -n "/^[a-z0-9.-]\+$/ p" $FILE > FILE.tmp.$$
  cp -f $FILE.tmp.$$ $FILE && rm $FILE.tmp.$$
done

Or

bash Code:

ls | while read FILE; do
  grep "^[a-z0-9.-]\+$" $FILE > $FILE.tmp.$$
  cp -f $FILE.tmp.$$ $FILE && rm $FILE.tmp.$$
done

Edit: Highlight=bash bbcode added by neo .....

devlin · #3 (**permalink**) 10-04-2009

Thanks for your reply Scottn

Your code give me the biggest hint for the last 3 days

My files are in fact sitemap, like sitemap.1.xml, sitemap.2.xml, sitemap.3.xml, ...
and forgot to mention that I also need to include '<', '>', ':', '/'

I tried to use this code but the ':' is not correctly set in this line I think...

(not working correctly)

Code:

sed -n "/^[<>\:a-z0-9.-\/]\{1,\}$/ p" sitemap.1.xml > sitemap.1.xml.tmp;mv sitemap.1.xml.tmp sitemap.1.xml

scottn · #4 (**permalink**) 10-04-2009

Hi.

Sed does seem to be somewhat pedantic about where bits go!

bash Code:

ls sitemap.*.xml | while read FILE; do
  sed -n "/^[a-z<>/0-9.:-]\+$/ p" $FILE > FILE.tmp.$$
  cp -f $FILE.tmp.$$ $FILE && rm $FILE.tmp.$$
done

devlin · #5 (**permalink**) 10-04-2009

Nice !!
Thanks

Didn't know about the position thing..!

Thanks for your great help

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
Reading lines from a file, using bash, "at" command	jbsimon000	Shell Programming and Scripting	3	03-17-2009 03:53 PM
"find command" to find the files in the current directories but not in the "subdir"	swamymns	Shell Programming and Scripting	9	07-22-2008 11:23 AM
Delete lines ending in "_;" using sed	turbulence	Shell Programming and Scripting	12	01-17-2008 06:51 PM
Development Releases: Linux Mint 4.0 Beta "Fluxbox", 4.0 Alpha "Debian"	iBot	UNIX and Linux RSS News	0	01-04-2008 03:00 PM
Explain the line "mn_code=`env\|grep "..mn"\|awk -F"=" '{print $2}'`"	Lokesha	UNIX for Dummies Questions & Answers	4	12-20-2007 01:52 AM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Switch to Linear Mode Hybrid Mode Switch to Threaded Mode	Rate This Thread:


	Powered by