Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Checking for duplicate code
Top Forums Shell Programming and Scripting Checking for duplicate code Post 302650179 by figaro on Friday 1st of June 2012 06:39:24 PM
Old 06-01-2012
Checking for duplicate code
I have a short line of code that checks very rudimentary for duplicate code:
Code:
sort myfile.cpp | uniq -c | grep -v "^.*1 " | grep -v "}"

It sorts the file, counts occurrences of each line, removes single occurrences and removes the ubiquitous closing brace. The language is C++, but is easily extensible to other programming languages.

I would like to make this a bit more advanced. A few examples:

1- Allow for spaces, so that the following lines of output are considered identical:
Code:
   2     for (i = 0; i < N; i++) {
   2        for (i = 0; i < N; i++) {

2- Allow for spaces within the code, so that the following lines of output are considered identical:
Code:
   2     for (i = 0; i < N; i++) {
   2     for ( i = 0; i < N; i++ ) {

If there are easy ways to fix this, I like to hear from you.

I am deliberately not excluding lines of comment, such as those containing "/*" or "*/" or "//", as this would reduce the case to tell developers to document their code better.

Any other one-liner ideas to check for duplicate code are also welcome.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Code checking for all values in the same if statement.

I am trying to set up a variable based on the name of the file. function script_name { if then job_name='MONITOR' return job_name; elsif then job_name='VERSION' return job_name fi } for i in `ls *log` do script_name $i done. (4 Replies)
Discussion started by: oracle8
4 Replies

2. Shell Programming and Scripting

Error code checking

I'm trying to create a directory from my Perl script. Only if the there was an error I want to let the user know about it. So if the folder exists is ok. This is what I think should work: `mkdir log 2>/dev/null`; if($? == 0 || $? == errorCodeForFileExists) { everyting is fine } else {... (3 Replies)
Discussion started by: jepombar
3 Replies

3. Shell Programming and Scripting

checking the return code

hi i have a file, i am reading line by line and checking a line contains a string , `grep "Change state" $LINE` if then echo "The line contains---" else echo "The line does not contains---" i need to check the return code , but i am getting an error ... (4 Replies)
Discussion started by: Satyak
4 Replies

4. Shell Programming and Scripting

checking duplicate entry in file

Hi i have a file like 110.10 120.10 -1120 110.10 and the lines are having more than 10k. do we have anycommand to check the duplicate entries in the file. I applied the while loop by greping each line with whole file, but it is taking huge amount of time as the file size is large. ... (5 Replies)
Discussion started by: saluja.deepak
5 Replies

5. Shell Programming and Scripting

code checking

i was just wondering how would you check , beside the lock method, if an instance of another code is already running and if it is then output a message to the user saying the program is already running and exit!! the code is in BOURNE SHELLL!!! thanks in advance!! (3 Replies)
Discussion started by: bshell_1214
3 Replies

6. UNIX for Dummies Questions & Answers

Nested for loops for checking duplicate files

I am very new to bash scripting and this is my first script. I am trying to write a script that takes an argument d as the directory. It looks through the files to find duplicates and delete them. Here's some sorta-pseudocode but am unsure how to implement it: #! /bin/bash #get... (1 Reply)
Discussion started by: shubham92
1 Replies

7. Shell Programming and Scripting

awk remove duplicate code

Hi, In a previous, now closed thread, I found the following awk script: awk '{t=$5" "$6" "$7}END{for (i in t){print i,t}}' This code does a great job of removing duplicates by the the first four fields from a 7-field set of columns. I would very very much like to understand how this code... (3 Replies)
Discussion started by: pawelrc
3 Replies

8. Shell Programming and Scripting

REMOVE DUPLICATE IN a ROW AFTER CHECKING THE FIRST SIMILAR NAME

Hi all I have a big file like this in rows and columns from 2 column onwards the next column is desciption of previous column means 3rd columns is description of 2 columns and 5 column is description of 4 column. All cloumns are separated by comma ... (1 Reply)
Discussion started by: manigrover
1 Replies

9. UNIX for Beginners Questions & Answers

Code for checking if certain no of files exists

Hi, I am writing the shell script in ksh to check certain no of files exists,In my case there are 7 files exist like below Sales1_timstamp.csv Sales2_timstamp.csv Sales3_timstamp.csv Sales4_timstamp.csv Sales5_timstamp.csv Sales7_timstamp.csv Sales7_timstamp.csv Once all the files... (4 Replies)
Discussion started by: SRPR
4 Replies

10. UNIX for Beginners Questions & Answers

Iterate through a list - checking for a duplicate then report it ot

I have a job that produces a file of barcodes that gets added to every time the job runs I want to check the list to see if the barcode is already in the list and report it out if it is. (3 Replies)
Discussion started by: worky
3 Replies

LEARN ABOUT DEBIAN

grid-mapfile-check-consistency

GRID-MAPFILE-CHECK(8)						  Globus Commands					     GRID-MAPFILE-CHECK(8)

NAME

       grid-mapfile-check-consistency - Add an entry to a grid map file

SYNOPSIS

       grid-mapfile-check-consistency [-h] [-help] [-usage] [-version]

       grid-mapfile-check-consistency [-mapfile MAPFILE] | [-f MAPFILE]

DESCRIPTION

       The grid-mapfile-check-consistency program performs basic checks for validity of a gridmap file. These checks include checks for existence,
       duplication of entries, and valid local user names. If the gridmap file is valid, grid-mapfile-check-consistency exits with a zero exit
       code, otherwise it exits with a non-zero exit code. In either case, it displays information about its progress as it parses and validates
       the gridmap file.

       By default, grid-mapfile-check-consistency will check the gridmap file named by the GRIDMAP environment variable if present. If that
       variable is not set, it will check the file $HOME/.gridmap for non-root users if present. If that doesn't exist or
       grid-mapfile-check-consistency is run as root, it will then check /etc/grid-security/grid-mapfile. This can be changed by the use of the
       -mapfile or -f command-line options.

       The full set of command-line options to grid-mapfile-check-consistency are:

       -help, -h, -usage
	   Display the command-line options to grid-mapfile-check-consistency.

       -version
	   Display the version number of the grid-mapfile-check-consistency command.

       -mapfile MAPFILE, -f MAPFILE
	   Check the gridmap file named by MAPFILE instead of the default.

EXAMPLES

       Check that the gridmap file in /etc/grid-security is valid:

	   % grid-mapfile-check-consistency -f /etc/grid-security/grid-mapfile
	   Checking /etc/grid-security/grid-mapfile
	   Verifying grid mapfile existence...OK
	   Checking for duplicate entries...OK
	   Checking for valid user names...OK

       Check a gridmap file that has an invalid local user name:

	   % grid-mapfile-check-consistency -f /etc/grid-security/grid-mapfile
	   Checking /etc/grid-security/grid-mapfile
	   Verifying grid mapfile existence...OK
	   Checking for duplicate entries...OK
	   ERROR: baduser is not a valid local username
	   ERROR: Found 1 invalid username(s)

ENVIRONMENT VARIABLES

       The following environment variables affect the execution of grid-mapfile-check-consistency:

       GRIDMAP
	   Path to the default gridmap to check.

FILES

       The following files affect the execution of grid-mapfile-check-consistency:

       $HOME/.gridmap
	   Path to the default gridmap to check if the GRIDMAP environment variable is not set for non-root users.

       /etc/grid-security/grid-mapfile
	   Path to the default gridmap to check if GRIDMAP environment variable is not set and the above file does not exist.

SEE ALSO

       grid-mapfile-add-entry(8), grid-mapfile-delete-entry(8)

AUTHOR

       University of Chicago

Globus Toolkit 5.0.1						    03/19/2010						     GRID-MAPFILE-CHECK(8)
All times are GMT -4. The time now is 11:31 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy