Awk, Sed and Grep


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Awk, Sed and Grep
# 1  
Old 07-07-2008
Question Awk, Sed and Grep

Hello. I am an older newbie trying to learn Unix. I have a task to perform and it entails counting lines of code. Currently, I am pointing to the directory where the files are contained and performing a 'find' on the file extensions (cpp, c, html, java, etc.) and piping that info with a 'wc -l'. But we all know that 'wc' counts every line and I only want to count lines of actual code.

What I would like to know is of the awk, sed, grep/egrep tools, which would be easier and accurate to truncate the files by removing comments, blank spaces and going through the entire line (some lines will have code three tabs in and such)? Once I get that, I want to do wc -l. Any help would be appreciated.
# 2  
Old 07-08-2008
The tools are as accurate as one defines the regular expressions to identify code and parse it correctly. They are working usually flawless, which means they are always right. If something isn't working and I start to get grey hairs about a problem to solve, I can be sure, that sed, awk or grep is always right in the end Smilie
Easier... hm, maybe awk is looking a bit more structured and has commands like other languages have. sed is like ed with only 1 letter for a function and can look somewhat messed up if complex regexp's are written.
If you start with small examples, you will get used to both, I bet, usually it takes it's time though like with all things Smilie
You can do a lot of tasks with sed as you can do with awk. The easy thing with awk is, that it is by default designed to parse fields and has therefore variables like NF, FS, OFS, .. and some more and is very readable compared to sed.

So here a short list, when what to use (other people might use the tools with other priorities) basically:

grep --- search every line of a file for some content
egrep/grep -E --- for using regular expressions
sed --- substituting, filtering, line and multiline parsing, ...
awk --- parsing fields, lines and multiline parsing, substitution, ...

For your wc -l problem, maybe try something like this for examle (asuming your commented lines start with a hash (#):

Code:
root@isau02:/data/tmp/testfeld> cat infile
line of code 1
line of code 2
# comment 1
line of code 3
line of code 4
# comment 2
line of code 5
root@isau02:/data/tmp/testfeld> for file in `find . -type f -name "infile*" -print`; do echo $file `grep -cv ^# $file`; done
./infile 5


Edit: Ie. you have to go into the use of other regular expressions for comments in c/c++ like those /* ... */ and don't forget to escape them with usually backslashes.

Last edited by zaxxon; 07-08-2008 at 03:25 AM..
# 3  
Old 07-08-2008
Okay. Now here is my issue. I am trying to count lines of code with many different files. The previous programmer created an script that creates an AWK. The programmer has two lines of code like this:

Code:
 find <directory> -name <file types> | awk '{print \wc -l \"$1}

Is there an equivalent to this using egrep or grep or something else? Also, how do the arguments help with storage? Thank you very much.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Grep/awk/sed help

got a file as y.txt 1 abc,def,ghj 2 defj,abc.kdm,ijk 3 lmn,cbk,mno 4 tmp,tmop,abc,pkl 5 pri,chk,cbk,lmo 6 def,cbk.pro,abc.kdm i want to search in the above file the key word like abc looking for two outcomes by passing the parameter value as abc into function and the two outocmes are... (6 Replies)
Discussion started by: silgun
6 Replies

2. Shell Programming and Scripting

grep? awk? sed? I don't know

Hi everyone! I have a file like this And I would like to find the Medium label when the value "last write" is "Jan 14" (it's could be another value like "jan 6") I really don't know what way to use to solve this problem... Thanks! (5 Replies)
Discussion started by: Castelior
5 Replies

3. UNIX for Dummies Questions & Answers

grep/awk/sed?

Thread1 { x = 2 y = 10485 } Thread2 { x = 16 y = 1048 } Thread3 { x = 1 y = 1049 } Thread4 { x = 4 y = 1047 z = 500 } Suppose the above is a piece of code. I need to automate and verify that the value of x under Thread1's 2. There are several... (3 Replies)
Discussion started by: foxtron
3 Replies

4. Shell Programming and Scripting

help using sed/awk/grep

thanks for your reply. but i'm not quite sure what your code is doing. i may be using it wrong but i'm not getting what i'm supposed to get. could you please elaborate? thanks again, (6 Replies)
Discussion started by: kratos.
6 Replies

5. Shell Programming and Scripting

grep or awk or sed not sure which to use here

Hi All, I have a huge file, I need to two things from this file. I need to know the IP address or the hostname and second thing is the date&time. The file looks like this and I need to get my data from this... Trying... Connected to 204.109.172.117. Escape character is '^]'. Fri... (4 Replies)
Discussion started by: samnyc
4 Replies

6. UNIX for Dummies Questions & Answers

awk grep sed or something better

Hello all, Can anyone help with the following? :) I have file1 with 150,000 words in a list and file2 with 148,000 words in a list - all of which are in file1. I want to create a new file with the words that DO NOT match (i.e of 2000 words). I have done this very simple command , which is... (1 Reply)
Discussion started by: dr_sabz
1 Replies

7. UNIX for Dummies Questions & Answers

How could i get this by sed or grep or awk ????

------------------------------------------------------------------ Ex of Warning messgae,(Many similar lines occure for Both Test and Test1) -WARNING:Below Field not implemented in file File name: /home/test/ new/file1, msg buffer is: :Test:000948 ... (1 Reply)
Discussion started by: prsam
1 Replies

8. UNIX for Dummies Questions & Answers

Grep Sed or Awk?

I have two .txt files one called good.txt and the other one is called bad.txt. Both contain email addresses in the following format: john@john.com bob@bob.com sarah@sarah.com Basically, I want to scrub good.txt against bad.txt and save the resulting output in scrubbed.txt meaning that if... (2 Replies)
Discussion started by: holyearth
2 Replies

9. Shell Programming and Scripting

Sed | Awk | Grep

Can someone help me in understanding when to use SED, AWK and GREP (3 Replies)
Discussion started by: kn.naresh
3 Replies

10. Shell Programming and Scripting

need help!!!awk,grep,sed

hi all by using cat /etc/passwd I've got these output. ajh1ect:x:839:501:Anthony:/home/ajh1ect:/bin/bash mjb1ect:x:840:501:Michael:/home/mjb1ect:/bin/bash mv3ect:x:841:501:Marian:/home/mv3ect:/bin/bash now I want to see just the user ID and group ID. so what is the code will be with... (2 Replies)
Discussion started by: nokia1100
2 Replies
Login or Register to Ask a Question