Visit Our UNIX and Linux User Community

Multiple scanners on same input file using lex/flex

Thread Tools Search this Thread
Top Forums Programming Multiple scanners on same input file using lex/flex
# 1  
Old 10-25-2006
Question Multiple scanners on same input file using lex/flex

Hi all,

I'm working with flex (version 2.5.4a) on GNU/linux. I used it to develop 4 scanner C files for matching different patterns within an input file. But the problem now on my hand is that I need to conditionally combine these. That is in main (placed in a separate C file other than the 4 scanner files) I'll have check the options provided by the user and accordingly call the scanner functions from the scanner C files.
For ex. if main is in file app.c and options are:
-f - for scanner function of 1st scanner file
-s - for scanner function of 2nd scanner file
-t - for scanner function of 3rd scanner file
-r - for scanner function of 4th scanner file
so something like './app -f -s -r filename' would mean that the scanners from 1st, 2nd and 4th file are called while 3rd's is not called.

The problem with such a situation is that when lex is invoked it generates C file with scanner function yylex() (and supporting functions for it). Now this function has same signature in all 4 scanners and hence the inclusion of all 4 files causes name conflict ("error: yylex() redifned here". Even -P option does not help as it just #define s the original function).

Now I hope everybody must be understanding my difficulty. Is there any way out (without changing to C++)?
While I'll like a functional decomposition as stated above but even if someone can come up with a solution where I need to combine all lex files to one and still be able to control what pattern are to be matched according to the switch provided by user in 'main()' will be a great help.

Thanks in advance!
# 2  
Old 10-25-2006
What's wrong with the -P option? The preprocessor can rename it as much as anything else can.
# 3  
Old 10-26-2006
No Corona688, that wont work because -Pprefix just #define's the original function (like -Pfirst would do '#define yylex firstlex') which would be expanded by preprocessor before compilation itself so while compiling the complier will complain that the function (yylex) has been redefined (because now there are 4 scanner functions from our 4 files expanded to yylex). And in fact not just yylex there are many other supporting functions and global variables that are #define'd this way & get expanded before compilation genrating a lot of errors.
Now can someone tell me way out?
# 4  
Old 10-26-2006
You have four lex-generated files - go intothose files and mark all of the functions in them static - this makes their scope local to the C file they live in. Next create a wrapper function that is NOT static.
in file1.c create scanner1() which calls the yylex function or whatever entrypoint you have defined. Create scanner2(), scanner3() and scanner4() in each of ther other files. In the main block, have for extern declarations for scanner1() scanner2() etc.

ld will hide the fact that each of those four modules has functions all named the same:
cc main.c file1.c file2.c file3.c file4.c -o scanmain

Assuming I understood your problem.... I didn't get your first problem very clearly either.
# 5  
Old 10-26-2006
Originally Posted by Rakesh Ranjan
No Corona688, that wont work because -Pprefix just #define's the original function (like -Pfirst would do '#define yylex firstlex') which would be expanded by preprocessor before compilation itself so while compiling the complier will complain that the function (yylex) has been redefined (because now there are 4 scanner functions from our 4 files expanded to yylex).
Could you post the errors from this? They might be solvable. The #define would NOT expand firstlex to yylex, quite the opposite -- the preprocessor would do a literal search and replace of yylex to firstlex. The -P method ought to work, that's what it's there for.

Jim's method would also work, but has the disadvantage of needing to hand-edit the .c files every time they're regenerated.
# 6  
Old 12-08-2006
Thanks Corona688 and thanks jim. Smilie Though late but I must thank you both.
I really interpreted the errors wrongly (a real silly but grave mistake). The errors were there for I was linking by #including files which I should not have and that confused me. Any way I got the things right after getting the mistakes.

Thanks again for your help.

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing multiple lines from input file, if multiple lines match a pattern.

GM, I have an issue at work, which requires a simple solution. But, after multiple attempts, I have not been able to hit on the code needed. I am assuming that sed, awk or even perl could do what I need. I have an application that adds extra blank page feeds, for multiple reports, when... (7 Replies)
Discussion started by: jxfish2
7 Replies

2. Shell Programming and Scripting

Avoid overwriting backup file when multiple entries need to replace in one file input from another

Hello, I have been working on script in which search and replace the multiple pattern. 1. read the multiple pattern from input file ParamMapping.txt(old_entry|New_entry) and passing this values one by one to 2. read... (0 Replies)
Discussion started by: ketanraut
0 Replies

3. Shell Programming and Scripting

Copying a file to multiple other files using a text file as input

Hello, I have a file called COMPLIST as follows that contains 4 digit numbers.0002 0003 0010 0013 0015 0016 0022 0023 0024 0025 0027 0030 0031 0032 0033 0035 0038 0041 (3 Replies)
Discussion started by: sph90457
3 Replies

4. Shell Programming and Scripting

Multiple file input

Once a time, I did see a way to multiply a file input in bash, but have forgotten how it was done. Eks. awk 'FNR==NR {a++;next} ($2 in a) {print $2}' file file Here you need file two times. I seen it some like this awk 'FNR==NR {a++;next} ($2 in a) {print $2}' 2&file (16 Replies)
Discussion started by: Jotne
16 Replies

5. Homework & Coursework Questions

Lex: analyzing a C file and printing out identifiers and line numbers they're found on

Florida State University, Tallahassee, FL USA, Dr. Whalley, COP4342 1. The problem statement, all variables and given/known data: Create a lex specification file that reads a C source program that ignores keywords and collects all identifiers (regular variable names) and also displays the line... (3 Replies)
Discussion started by: D2K
3 Replies

6. Shell Programming and Scripting

How to ftp multiple files by taking the file name from a input file.

Hi, I'm working on a script which has to copy multiple files from one server to another server. The list of files that are to be copied is present in a file say input.txt. vi input.txt abc.c welcome.c new.c welcome1.c for ftp'ing a single file say 'new.c' the following code... (2 Replies)
Discussion started by: i.srini89
2 Replies

7. Shell Programming and Scripting

Split a file into multiple files based on the input pattern

I have a file with lines something like. ...... 123_start ...... ....... 123_end .... ..... 456_start ...... ..... 456_end .... ..... 789_start .... .... 789_end (6 Replies)
Discussion started by: abinash
6 Replies

8. Shell Programming and Scripting

building output file from multiple input files

Hi there, I am trying to figure out a way to combine multiple sources with different data on a single file, and I am trying to find the best way to do it. I have multiple files, let's say A, B, C and D. A has a field in common with B, B has a field in common with C, and C has a field in... (2 Replies)
Discussion started by: ppucci
2 Replies

9. Shell Programming and Scripting

isql input file with multiple sql statements

I've got: isql -U $USERID -S $SERVER -D $DATABASE -i inputfile.sql -o outputfile.txt in inputfile I have: go sql#1 go sql#2 go sql#3 go I also tried without "go" and with";" instead which did not work SQL statements will work if I paste them directly into the script and use EOF ... (0 Replies)
Discussion started by: Cailet
0 Replies

10. Shell Programming and Scripting

Can Xargs execute multiple commands of evry input file

Hello , I am trying to print the footer of evry file in the given directory with xargs command like follows ls -1 | xargs -I {} gzcat {} | tail -1 now problem with this is only last file foooter is getting printed as " | tail -1 " is getting executed for the last file. I know this can... (4 Replies)
Discussion started by: nilesrex
4 Replies

Featured Tech Videos