Shell Programming and Scripting

Ok, I've found the bug that was causing the double comma. The only dummy input file that was without an ntp record was the last one, so I wasn't seeing that I had coded that incorrectly.

Have a go with this:

Code:

 awk '
    BEGIN { n = 0; k = 1;      # needed for bug fix
    }

    FILENAME != last {
        fn = FILENAME;
        gsub( ".txt", "", fn );
        printf( "%s%s%s", k ? "" : ",", n ? "\n" : "",  fn );  # bug fix here
        last = FILENAME;
        n = 1;     # signal need for newline before fn (prevent lead newline)
        k = 0;     # when set, signals no comma needed in previous printf
    }

    /ntp server/ || /ntp peer/{
        printf( ",%s", $0 );
        k = 1;
    }

    END {
        printf( "%s\n", k ? "" : "," );
    }'

I'm still perplexed with the newline when there is a second record because I'm not seeing that with my dummied up test data. I'll continue to play with this for a bit, but wanted to see that this should have made the difference with the double commas.

I assumed that the 'ntp peer' record was to be handled the same way as the ntp server record. If not, post how you need it and a sample of the text from the file.

And not to worry about not posting the o/s type. It can help, but in this case I don't think OSX and its tools are to blame.

Me neither! (as if!)

Then it's obviously some important detail that I'm failing to share with you.

You assume correctly!

Glad to hear it!

And I just learned something new about these forums...

I was refreshing the thread each time before I posted an edit to my last post with the idea I would know that you hadn't been back (or at least that you hadn't posted anything new). However, when I reloaded the thread from the forum index, I saw this post from you. Sorry for any confusion that may have introduced!!

---------- Post updated at 10:02 PM ---------- Previous update was at 09:52 PM ----------

OK, what is it about:

awk 'BEGIN (...)

I'm having trouble with??

I guess I should call it a night and start fresh in the morning. I can't thank you enough for what you've done to get me this far...

I didn't include the ls command and the pipe. My test script is still using xargs, so I only cut and pasted the awk portion.

I'm calling it a night; not sure when I can peek in tomorrow -- it will probably be early afternoon. You are most welcome for the help. I'm always surprised at what I end up learning as I figure out, or read other's solutions.

Understood. I have spent countless hundreds (likely thousands) of hours over the years doing likewise for others on the network engineering side (I'm a CCIE if that means anything to you), and I've learned as much if not more from that process as anything.

I have tried a bunch of different things but I always get a ">" after the first line of whatever I paste into my shell. Likely my ignorance is start to seriously show if it hasn't already. Sleep will likely help too!

Good night!

Do you have Perl on your Mac?

Code:

perl -nle 'printf "$ARGV," if $.==1;printf "$_," if /ntp server/;if (eof){$.=0;printf "\n"}' * > out.csv

I can explain the > --

The greather than sign is the 'secondary' prompt (also known as PS2) which is issued when a newline is entered before a terminating single quote. So, pasting the awk snipit would cause this.

The easy solution, or in my opinion the easiest, is to edit a file (test_script.ksh or somesuch) with your favorite editor and paste the code example into that. Save the file, and then from the command line execute the script using ksh or bash (or your preferred shell). Like this:

Code:

ksh test_script.ksh

The other advantage to this is that you can edit the file and easily tweek things if you want, and you have the template for a script when you have something that works.

You can also use Kshell's (or bash's) command line editing features that allow direct invocation of emacs or vi, but that's more difficult to explain if you aren't already familiar with it.

Yeah, I definitely needed a good night's sleep. So I stayed up and smoked a cigar instead!

Anyway, with xargs using both bash and ksh, I was only getting the hostname of the very first device in the folder - nothing else (so no commas, no ntp commands, no nothin'). So I dropped xargs and did *.txt in awk. I get an output file and both 'ntp server' and 'ntp peer' commands are present. However, only my to two core devices have a single-line entry for multiple ntp matches; all others have a newline between first and second entries when there are two matches.

Now this may or may not be an interesting detail: I was unaware that my core devices actually have three 'ntp server' commands. And yes, they're strangely all on one single line in the output file. I don't see any real differences between the core configs and any of the other configs??

---------- Post updated at 10:49 AM ---------- Previous update was at 10:45 AM ----------

Hi bartus11,

Yes, I just installed ActivePerl the other day (and I believe that some stripped down/likely outdated version of Perl ships native to Mac OS X). I'm trying to ensure that I actually understand (as much as possible) what I'm doing, though, so I don't want to jump to Perl just yet - my head is too full of new stuff this weekend as it is! But I promise that once I get this other approach sorted out, I'll turn my attention to your example Perl code. It certainly looks nice and streamlined!

Thanks again to you both!

---------- Post updated at 10:53 AM ---------- Previous update was at 10:49 AM ----------

OK, I admit that I just pasted in your Perl script to see what would happen.

Here's an example of the output:


router1.txt,
router2.txt,ntp server 10.10.250.1
,
switch1.txt,ntp server 10.10.250.2
,ntp server 10.10.250.1 prefer
,

Still suffering from newline issues??