Gawk Script in Windows batch file - Help

 
Thread Tools Search this Thread
Special Forums Windows & DOS: Issues & Discussions Gawk Script in Windows batch file - Help
# 1  
Old 10-04-2010
Gawk Script in Windows batch file - Help

Good morning all. I have been running into a problem running a simple gawk script that selects every third line from an input file and writes it to an output file.

Code:
gawk "NR%3==0" FileIn > FileOut

I am attempting to run this command from a batch file at the command line. I have several hundred files to perform this operation on and it would sure be handy to be able to just work up a quick batch file and let it fly.

The script works fine from the command line. Very slow though since I must do one file at a time and change the file name between each run. Tedious to say the least and I have been up all night now.

The problem that I am having appears to be related to the %3 qualifier in the script. When run from a batch file I do not get an error message. I get an empty file and the screen output at the command line shows the code interpreted as

Code:
gawk "NR==0" FileIn > FileOut

I have tried backslashing the %, enclosing it in double-quotes, enclosing each character in double-quotes, double-quotes and double backslashes, etc. All the combinations that my tired mind could conjure. No joy.

I am beginning to think that gawk will not pass this particular character to the command line from a Windows batch file. I have looked around various forums, read the books that I have, and can find nothing to guide me.

I would be thrilled if someone had a solution to the problem of grabbing every third line from a file and dumping to text output. Perl or gawk preferred. Other options will be considered.

Thanks for looking. 10000Springs (BC)
# 2  
Old 10-04-2010
Quote:
Originally Posted by 10000springs
...a solution to the problem of grabbing every third line from a file and dumping to text output. Perl or gawk preferred. ...
Here's a short Perl program that does that -

Code:
#perl -w
$pattern = "file*";                         # process only those files that match pattern "file*"
while (defined ($in = glob($pattern))) {
  ($out = $in) =~ s/\.in$/.out/;            # read from "xyz.in" and write to "xyz.out"
  open (IN, "<", $in) or die "Can't open $in for reading: $!";
  open (OUT,">>", $out) or die "Can't open $out for writing: $!";
  while (<IN>) {
    print OUT $_ if $.%3 == 0;
  }
  close (IN) or die "Can't close $in: $!";  # good idea to do some housekeeping
  close (OUT) or die "Can't close $out: $!";
}

Testcase -

Code:
C:\>
C:\>
C:\>rem Display the contents of input files "file*.in"
 
C:\>type file1.in
this is line   1 in file1.in ...
this is line   2 in file1.in ...
this is line   3 in file1.in ...
this is line   4 in file1.in ...
this is line   5 in file1.in ...
this is line   6 in file1.in ...
this is line   7 in file1.in ...
this is line   8 in file1.in ...
this is line   9 in file1.in ...
this is line  10 in file1.in ...
 
C:\>type file2.in
this is line   1 in file2.in ...
this is line   2 in file2.in ...
this is line   3 in file2.in ...
this is line   4 in file2.in ...
this is line   5 in file2.in ...
this is line   6 in file2.in ...
this is line   7 in file2.in ...
this is line   8 in file2.in ...
this is line   9 in file2.in ...
this is line  10 in file2.in ...
 
C:\>type file3.in
this is line   1 in file3.in ...
this is line   2 in file3.in ...
this is line   3 in file3.in ...
this is line   4 in file3.in ...
this is line   5 in file3.in ...
this is line   6 in file3.in ...
this is line   7 in file3.in ...
this is line   8 in file3.in ...
this is line   9 in file3.in ...
this is line  10 in file3.in ...
 
C:\>rem Now display the content of the Perl program
 
C:\>type process_files.pl
#perl -w
$pattern = "file*";                         # process only those files that match pattern "file*"
while (defined ($in = glob($pattern))) {
  ($out = $in) =~ s/\.in$/.out/;            # read from "xyz.in" and write to "xyz.out"
  open (IN, "<", $in) or die "Can't open $in for reading: $!";
  open (OUT,">>", $out) or die "Can't open $out for writing: $!";
  while (<IN>) {
    print OUT $_ if $.%3 == 0;
  }
  close (IN) or die "Can't close $in: $!";  # good idea to do some housekeeping
  close (OUT) or die "Can't close $out: $!";
}
 
C:\>
C:\>rem Run the Perl program
 
C:\>perl process_files.pl
 
C:\>
C:\>rem Now check the output files
 
C:\>type file1.out
this is line   3 in file1.in ...
this is line   6 in file1.in ...
this is line   9 in file1.in ...
 
C:\>type file2.out
this is line   3 in file2.in ...
this is line   6 in file2.in ...
this is line   9 in file2.in ...
 
C:\>type file3.out
this is line   3 in file3.in ...
this is line   6 in file3.in ...
this is line   9 in file3.in ...
 
C:\>
C:\>

HTH,
tyler_durden
This User Gave Thanks to durden_tyler For This Post:
# 3  
Old 10-04-2010
Quote:
Originally Posted by 10000springs
Good morning all. I have been running into a problem running a simple gawk script that selects every third line from an input file and writes it to an output file.

Code:
gawk "NR%3==0" FileIn > FileOut

I am attempting to run this command from a batch file at the command line. I have several hundred files to perform this operation on and it would sure be handy to be able to just work up a quick batch file and let it fly.

The script works fine from the command line. Very slow though since I must do one file at a time and change the file name between each run. Tedious to say the least and I have been up all night now.

The problem that I am having appears to be related to the %3 qualifier in the script. When run from a batch file I do not get an error message. I get an empty file and the screen output at the command line shows the code interpreted as

Code:
gawk "NR==0" FileIn > FileOut

I have tried backslashing the %, enclosing it in double-quotes, enclosing each character in double-quotes, double-quotes and double backslashes, etc. All the combinations that my tired mind could conjure. No joy.

I am beginning to think that gawk will not pass this particular character to the command line from a Windows batch file. I have looked around various forums, read the books that I have, and can find nothing to guide me.

I would be thrilled if someone had a solution to the problem of grabbing every third line from a file and dumping to text output. Perl or gawk preferred. Other options will be considered.

Thanks for looking. 10000Springs (BC)
You could try to escape the "%":
Code:
gawk "NR\%3==0"

This User Gave Thanks to Franklin52 For This Post:
# 4  
Old 10-04-2010
Isn't the Escape in MSDOS Batch %% ?

Have you tried:

Code:
gawk "NR%%3==0" FileIn > FileOut

This User Gave Thanks to methyl For This Post:
# 5  
Old 10-04-2010
I noticed that Cygwin's gawk works as expected on my system -

Code:
C:\>
C:\>type file1.in
this is line   1 in file1.in ...
this is line   2 in file1.in ...
this is line   3 in file1.in ...
this is line   4 in file1.in ...
this is line   5 in file1.in ...
this is line   6 in file1.in ...
this is line   7 in file1.in ...
this is line   8 in file1.in ...
this is line   9 in file1.in ...
this is line  10 in file1.in ...
 
C:\>
C:\>c:\cygwin\bin\gawk "NR%3==0" file1.in
this is line   3 in file1.in ...
this is line   6 in file1.in ...
this is line   9 in file1.in ...
 
C:\>
C:\>c:\cygwin\bin\gawk 'NR%3==0' file1.in
this is line   3 in file1.in ...
this is line   6 in file1.in ...
this is line   9 in file1.in ...
 
C:\>
C:\>

tyler_durden
This User Gave Thanks to durden_tyler For This Post:
# 6  
Old 10-04-2010
My problem is solved in more ways than one. Thanks to all who viewed the thread and especially those who posted their useful solutions. I learned a bit in the process and have improved my turn-around on the processing of these files considerably.

As Methyl and Franklin52 both pointed out and as I suspected, I was not escaping the % correctly in the batch flow. I also probably contributed to confusion by using the term "backslashing" in place of "escaping" in my original post. The actual term escaped me since I have been up for 26 hours now and I am trying to get back in tune with programming after a long dry spell.

As I noted in the first post the script Would run from the command line in Windows (XP).

I can also run the same script with no problems in cygwin, as tyler_durden pointed out.

It was only when I tried to use it in a batch file to process multiple files that I had a problem. I could not remember how to escape the % character in a batch file. The gawk solution put forth by Methyl worked perfectly in my batch file. I appreciate that immensely. My batch file has been modified as in the example posted by Methyl with the %% in front of the decimation interval.

I appreciate too the code example (perl) posted by tyler_durden above. Using that example, I was able to modify a short perl program that I put together a few weeks ago (with assistance from some of the same people on this site) so that I can perform this file reformat and decimation in one step instead of two.

I'm almost ready to stop and smell the roses.

Thanks for everything. 10000Springs (BC)
# 7  
Old 01-12-2011
Code:
@echo off
for /f "tokens=2 delims=:" %%_ in (' 
	^(
		echo 3^,3l
		echo e
	^) ^| edlin /b FILEIN.TXT
') do (
	echo %%_ > OUT.TXT
	)
exit /b 0

Code:
C:\>type FILEIN.TXT
Dan
Brown
El
Simbolo
Perdido

C:\>code.bat

C:\>type OUT.TXT
 El

C:\>

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Windows batch script to Shell script

Hi everyone, I've below windows batch script which is used to filter the file contents line by line and assign the matched values to the variables in for loop. for /F "tokens=1,3 delims=:" %%A in (%LOG_DIR%\PM_IS_workflow_status.log) do ( set "ATTR_NAME=%%A" if /i "!ATTR_NAME!" EQU "Folder"... (1 Reply)
Discussion started by: Kathraji
1 Replies

2. Shell Programming and Scripting

To run a shell script in remote server from windows batch file

Hi all, i need to run a shell script on remote server. I have created file .bat file in windows server with following code, c:\Users\Desktop\putty.exe -ssh -pw password user@server ./script.sh i need to run the script.sh in my remote server Above command is not working, any... (4 Replies)
Discussion started by: rammm
4 Replies

3. Shell Programming and Scripting

Windows Batch script for Unix commands

I wish to create a folder on a unix server B from my windows box using windows batch script. Below is my windows batch script. @ ECHO OFF ::Enter your Directory name: echo Enter your Directory name: set /p mydir= plink user1@ServerA mkdir %mydir% At plink command i get logged... (7 Replies)
Discussion started by: mohtashims
7 Replies

4. Windows & DOS: Issues & Discussions

How to start a vbs from a windows batch file?

Morning, I'm trying to execute a vbs from a .bat file. Can someone tell me what the difference is between these statements: start c:\lib\runit.vbc c:\lib\runit.vbs When I run the batch with the 'start' parameter it doesn't seem to do anything. (1 Reply)
Discussion started by: Grueben
1 Replies

5. Windows & DOS: Issues & Discussions

Check the file size using Batch script in windows

Hi, I need to check the file size using a batch script. Pls advise. (0 Replies)
Discussion started by: krackjack
0 Replies

6. Shell Programming and Scripting

Change the Windows Batch script to UNIX shell script.

Hi, When I run the below script in UNIX it's throwing syntax errors. Actually it's a windows batch script. Could anyone change the below Windows Batch script to UNIX shell script... Script: REM :: File Name : Refresh_OTL.bat REM :: Parameters : %1 - Region REM :: : %2 - Cube Type REM ::... (5 Replies)
Discussion started by: tomailraj
5 Replies

7. Shell Programming and Scripting

Executing Windows batch file from UNIX

Hi everyone, let me get straight to the points. My manager wants to execute a remote batch file (on a Windows server) from a UNIX Machine, does anyone know if this is possible and what packages would be needed? Thanks p.s. Sorry i cant give OS specifics, we use most UNIX's; AIX, Solaris,... (5 Replies)
Discussion started by: flip387
5 Replies

8. UNIX for Dummies Questions & Answers

backup through batch file in windows

Hi, I would like to backup particular files from unix to windows for every day through ftp to my desktop. For that anyone tell me syntax for create batch file in windows. Regards, Arulkumar (0 Replies)
Discussion started by: arulkumar
0 Replies

9. Shell Programming and Scripting

how to convert unix .ksh script to windows .batch script

I am using awk in my .ksh script but when I am trying to run in windows its not recognising awk part of the ksh script , even when I changed it to gawk it does not work, this is how my .ksh and .bat files look like. thanx. #!/bin/ksh egrep -v "Rpt 038|PM$|Parameters:|Begin |Date: |End... (1 Reply)
Discussion started by: 2.5lt V8
1 Replies

10. UNIX for Dummies Questions & Answers

How can I run scripts in my unix account from batch file in Windows?

Hi all, I'm working on Windows, connecting to my Unix account by different ways: by FTP opening files in UltraEdit32, by mapping drive to browse, by Exceed or Telnet to compile at Unix account. Actually, that is what I would like to change: I'd like to make a batch file which would connect to... (7 Replies)
Discussion started by: olgafb
7 Replies
Login or Register to Ask a Question