Shell Programming and Scripting

That would be a problem if OFS were set to something that's special in a printf format string. For example, if OFS were %, then the output would always be %s.

I suspect it was just an oversight on your part, but for those that don't get it, here's the correct way:

Code:

    printf "%s%s", sep, $i

Regards,
Alister

I remember finding this bug but probably my modification didn't went through.

In the world of awk yes, but there's no way I would do that in C, and somehow doesn't make me want to do it in awk as well. If you're careful about speed you'll naturally not use that method despite appearing to be simpler. One could argue that that could be better but one would not.

Sometimes we think we could simplify things by minimizing our code but sometimes it just gets more bloated. Mine may not have been in its most optimized form but at least there's the balance. Yeah I know speed isn't crucial really but we have our opinions.

Edit: You could actually be correct about appending strings instead of calling multiple printfs since that could possibly cause multiple ioctl calls depending on awk's implementation, but I wouldn't consider print substr(s, length(OFS)+1).

Or

Code:

printf "%s", sep $i

---------- Post updated at 03:15 AM ---------- Previous update was at 02:40 AM ----------

Alister addressed your explicit split(), ignoring the built-in auto-split. That's unnecessary overhead.
The formatting method does not really matter, but why not present an alternative? I was even inspired to present a 3rd method.

Considering the use of FS and OFS I now have this version:

Code:

awk -v m=3 -v n=9 -v FS=, -v OFS=, -- '{
    j = 0
    for (i = 1; i <= NF; ++i) {
        if (i == m || i == n) {
            ++j
            continue
        }
        $(i - j) = $i
    }
    NF -= j
    print
}'

And adding Alister's suggestion this is the best change I could have for my original code:

Code:

awk -v m=3 -v n=9 FS=, OFS=, -- '{
    append = 0
    for (i = 1; i <= NF; ++i) {
        if (i != m && i != n) {
            if (append) {
                s = s OFS $i
            } else {
                s = $i
                append = 1
            }
        }
    }
    print s
    s = ""
}' file

Given the dearth of detail, any optimization efforts would be aimless.

For an average implementation, on average hardware, processing an average text file, under average user expectations, the performance discrepancy between the AWK scripts will be insignificant, and there has been no indication by the OP that this situation is anything but average.

For an extraordinary situation, the details which we do not have (awk implementation? data set characteristics?) are crucial.

Testing with gawk, mawk, and busybox and two types of data, one with modest lines (100 columns, 292 bytes each) and the other with much wider lines (32,765 columns, 185,484 bytes each), yielded highly inconsistent results.

My original suggestion was sometimes the fastest, but only when lines were modestly-sized. As you correctly pointed out, my code does not scale; performance degrades drastically with increasing line length.

Casual testing suggests that you're using gawk, because otherwise the performance of your more recent suggestions regresses greatly compared to your original contribution.

Gawk running the following script was the fastest of all possible implementation/script combinations (that I tested):
However, that very same script under Busybox was also the slowest of all interpreter/script combinations (slower even than any run of my original sloth). This script was also the slowest of all under mawk.

The highlighted statements trigger recomputation of $0 in all three implementations, but only gawk implements an optimization to lazily avoid that overhead until $0 itself (not its fields) is referenced. For the details, follow field0_valid in gawk - field.c

There are a lot of systems out there that do not use gawk by default. Even among Linux installations, most embedded systems and most Debian derivatives (including most Ubuntu and Ubuntu-derivative versions) do not use it. For all of them, this revision is a setback.

In the absence of any specifics, in my judgement, your original solution exhibits the best balance of scalability and predictable performance across implementations. Minus the redundant split, the off-by-one in the loop condition, and the printf format string bugs:

Code:

{
    append = 0
    for (i = 1; i <= NF; ++i) {
        if (i != m && i != n) {
            if (append) {
                printf "%s%s", OFS, $i
            } else {
                printf "%s", $i
                append = 1
            }
        }
    }
    print ""
}

In this specific case, though, since there is no constraint requiring AWK and since any cut implementation would outperform any AWK implementation running any of these scripts ... by a significant margin, the performance debate is academic.

Regards,
Alister