How can I do this in VI editor?

Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How can I do this in VI editor?
# 8  
Old 03-09-2013
Originally Posted by hanson44
sed would never choke on this task, no matter how many lines. Even billions of lines. sed just reads one line at a time, so the number of lines is totally not an issue. If I'm wrong about this, I'd like to be educated.
I don't have time at this very moment to consult an implementation's source or run some tests (I might in a few hours), but I would bet that you are wrong about this. The integral type with which line numbers (current input line number and command address line numbers) are implemented will eventually overflow.


---------- Post updated at 10:57 AM ---------- Previous update was at 09:43 AM ----------

Current GNU sed source uses a function named compile_address() to parse command addresses. For numeric addresses (as opposed to regular expression addresses), this function then calls in_integer() to construct the corresponding integer. [1]
/* Read an integer value from the program.  */
static countT in_integer (int ch);
static countT
  int ch;
  countT num = 0;

  while (ISDIGIT(ch))
      num = num * 10 + ch - '0';
      ch = inchar();
  return num;

countT is an unsigned long integral type [2]. On 32-bit platforms where long is 32-bit, overflow is not implausible, but on a 64-bit UNIX platform, where the LP64 data model specifies a 64-bit long, even extremely large files shouldn't be an issue. That's not to say that there is no limit. There is. It's just much higher, on the order of billions of billions, 2^64 (1.8446744 x 10^19).

Since GNU tools are ported to Microsoft Windows, it's noteworthy that unsigned long remains only 32 bits even on 64 bit Windows (which uses the LLP64 data model, instead of the LP64 adopted by UNIX systems [3], [4]).

A demonstration of overflow on a 32-bit Debian Linux machine with GNU sed 4.1.5 (yes, this is an old version, but that code above, which matches the following behavior, is from the current repository):
$ echo hi | sed -n 4294967296p
sed: -e expression #1, char 11: invalid usage of line address 0
$ echo hi | sed -n 4294967297p

A simple overflow check in in_integer() could detect all of these problems, generate a useful error message, abort compilation of the sed script before a single line of data has been read, and return a meaningful exit status.

There's a similar overflow issue with the current line number. It is also a countT integer and it is also not checked for overflow when incremented while reading in the next line of text into the pattern space [5].

This means that that the 4294967297th line in a file is the second instance of a first line. This will cause commands which were only intended for the first line of the file to match a later line. For example, the simple command 1d will delete multiple lines. Specifically, every line for which the following expression is true: (real line number) % 4294967296 == 1

To summarize, in general, the number of lines in a file and the magnitude of the addresses used in the sed script are both restricted by the underlying data type's range. In particular, GNU sed does a terrible job of handling out of range values. Other implementations may not be much better; I did not check.


[1] GNU sed compile.c
[2] GNU sed basicdefs.h
[3] 64-Bit Programming Models: Why LP64?
[4] Why did the Win64 team choose the LLP64 model?
[5] GNU sed execute.c

Last edited by alister; 03-09-2013 at 06:06 PM..
These 3 Users Gave Thanks to alister For This Post:
# 9  
Old 03-09-2013
Originally Posted by kraljic
version info :
vi availabe with RHEL 5.4
I have a text file with 10,000 lines. I want to copy lines from 5000th line to 7000th and redirect to a file. Any idea how I can do this?

The above scenario is just an example. In my actual requirement, the file has 14 million lines and I want to copy 2000 lines after the 12th million line Smilie

BTW.I've just realised that the vi editor we use in RHEL is actually VIM
If you have your file open in Vim, and you want to copy lines from 5000 to 7000 both inclusive to a new file, then use the following editor command:

:5000,7000 w /path/to/new/file

# 10  
Old 03-09-2013
Thank you for educating me. I appreciate the incredibly detailed research you did.

You are right eventually the line count gets too high for that unsigned long, and the data overflows. So I was flat wrong in saying "no matter how many lines", or even saying "billions of lines". When I heard the word "choke", I thought the poster was thinking "running out of memory", and perhaps they didn't understand about sed pattern space. I should have done a simple test before speaking.

I would suggest for the vast majority of real-world situations sed will not "choke" on files with many lines. My guess is very few situations use billions of lines. The previous poster cero was worried about a file with 14 million lines. I hope I'm correct in saying it would at least work with that file. But you are absolutely right that it screws up when that unsigned long overflows. And yes, as you pointed out, sed should abort with an error code before reading a line.

Anyway, I apologize for the totally unrealistic assertion "no matter how many lines".
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Open Source

What editor does everyone use?

I was looking through the topics and I wasn't sure if this was the best place to post this question: I was wondering, out of curiosity, which software everyone was using to code their scripts in. I do mostly sh/ksh and my favorite has always been EditPlus because it is small, fast, yet powerful.... (409 Replies)
Discussion started by: yongho
409 Replies

2. Shell Programming and Scripting

Not able to use @ in VI editor

Hello All, Need one Help for one issue. I am using a French Keyboard, so @ sign is on key 0 and i have to use right Alt + 0 to print it. It is working everywhere but not inside Vi editor. I can type @ in shell, in notepad. But inside Vi editor it is not working, another problem is that if... (2 Replies)
Discussion started by: yadavricky
2 Replies

3. Shell Programming and Scripting

About vi editor

How can ` character be printed on vi editor ? empl_id=`echo $line | awk ' { print $1; } '` (2 Replies)
Discussion started by: senem
2 Replies

4. Solaris

Epic Editor was not able to obtain a license for your use. Feature Epic Editor :Licen

Epic Editor was not able to obtain a license for your use. Feature Epic Editor :License server is down (1 Reply)
Discussion started by: durgaprasadr13
1 Replies

5. Shell Programming and Scripting

set EDITOR=vi -> default editor not setting for cron tab

Hi All, I am running a script , working very fine on cmd prompt. The problem is that when I open do crontab -e even after setting editor to vi by set EDITOR=vi it does not open a vi editor , rather it do as below..... ///////////////////////////////////////////////////// $ set... (6 Replies)
Discussion started by: aarora_98
6 Replies

6. UNIX for Dummies Questions & Answers

Pasting text in VI editor from a different editor

Hi, I knw its a silly question, but am a newbie to 'vi' editor. I'm forced to use this, hence kindly help me with this question. How can i paste a chunk 'copied from' a different editor(gedit) in 'vi editor'? As i see, p & P options does work only within 'vi'. (10 Replies)
Discussion started by: harishmitty
10 Replies

7. HP-UX

vi editor

I am new in hp ux and I want work with vi editor, but in hp ux vi editor the backspaes and del keys doesn't work. how can I enable them. thanks (3 Replies)
Discussion started by: hkoolivand
3 Replies

8. UNIX for Advanced & Expert Users

vi editor

Hi, how can I add at the begining and at the end of all of the lines of my text file in VI editor ? Many thanks before. for exemple if in my file i have line 1 line 2 I want to have : start line 1 end start line 2 end (3 Replies)
Discussion started by: alain123456
3 Replies

9. HP-UX

instead VI editor - which one?

I'd like to find some editor for HP-UX, something like notepad, but not VI editor. Can someone have some ideas which one? thx (6 Replies)
Discussion started by: diamond
6 Replies

10. Shell Programming and Scripting

Vi editor ?

Hello everybody, My question is: how to add /tmp/work at the end of line in vi editor. my file looks like: cp file1 cp file2 cp file3 **** I need to add " /tmp/work" at the end of each line. I tried this :%s/$/" /tmp/work" and this :%s/$/\ /tmp/work\/ but it does not work. (2 Replies)
Discussion started by: billy5
2 Replies
Login or Register to Ask a Question

Featured Tech Videos