Check valid records in really big file with one commend.. - Shell Programming and Scripting

vtischuk@yahoo. · #1 (**permalink**) 05-17-2008

Hi,
I have a 5 gig file, no record terminators, field terminators are newline. The record length is 768 and I would like to check that every 768th byte is a newline and print out the byte position if it isn't. I would like to do this going either forward or backwards with one command if possible. I read that awk has a 3000 byte limit for a record so maybe a complex grep?

Thanks,

Victor

era · #2 (**permalink**) 05-18-2008

Many egreps will probably choke on this, but it's worth a try.

Code:

egrep -v '^.{768}$' file

danmero · #3 (**permalink**) 05-18-2008

Maybe awk will help.

Code:

awk '{len=length($0); if(len < 768) {print NR, len}}' data.file

vtischuk@yahoo. · #4 (**permalink**) 05-18-2008

thanks for the suggestions, I'll try them tomorrow..

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
Splitting a file based on the records in another file	er_ashu	Shell Programming and Scripting	2	05-12-2008 01:34 PM
How to check for a valid numeric input	Vijayakumarpc	Shell Programming and Scripting	1	08-04-2007 05:34 AM
Count No of Records in File without counting Header and Trailer Records	guiguy	Shell Programming and Scripting	2	06-07-2007 09:15 AM
Script to check for a file, check for 2hrs. then quit	mmarsh	UNIX for Dummies Questions & Answers	2	09-16-2005 11:46 AM
renaming files in one commend	fboisvert	UNIX for Dummies Questions & Answers	2	09-12-2001 10:21 AM

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode