Lexer Project
Compiler Project
COP-5621
Table of Contents
1. Overview
Implement a lexer for the While language as described in Program Analysis Section 2.1
The class will go over using ANTLR and the python runtime for its implementation, but you are free to hand-write a lexer or use other generators with instructor approval.
Create example inputs (both valid and invalid) to test your lexer.
Submit via git.
Consult the syllabus for deadlines and grading.
2. Project setup
2.1. Log in to eustis
2.2. Create the project directory
mkdir ~/cop5621spring25/compiler
2.3. Go to the project directory
cd ~/cop5621spring25/compiler
2.4. Download the grammar files
wget https://www.cs.ucf.edu/~gazzillo/teaching/cop5621spring25/files/lex.tar
2.5. Untar the files
tar -xvf lex.tar
2.6. Generate the lexer/parser
make -C grammar
2.7. Setup your environment
Set your PYTHONPATH to be the root of your compiler project, by adding this to your ~/.bashrc to avoid having to type it every time, i.e.,
echo 'export PYTHONPATH=${HOME}/cop5621spring25/compiler:${PYTHONPATH}' >> ~/.bashrc
Process the .bashrc file without having to log back in:
source ~/.bashrc
3. Implementing the project
3.1. Create a new file in a subdirectory called compiler/Lex.py
mkdir compiler touch compiler/Lex.py
Be sure the the file is in a new subdirectory called copmiler. This will be the same name as your project directory. Confirm its path with
readlink -f compiler/Lex.py
You should see
/home/net/NID/cop5621spring25/compiler/compiler/Lex.py
Note that you have the subdirectory compiler/compiler
3.2. Edit compiler/Lex.py and add the following code.
3.3. Run your project
python3 compiler/Lex.py < examples/lex.while
You should see the following output:
x := 1
3.4. Creating tests
Create at least five new test programs in a new tests/ folder that exercise as many of the tokens of the While language as you can. Run them as before.
4. Configure git
You only need do this once for the entire semester.
4.1. Set user name
Replace "John Doe" with your full name.
git config --global user.name "John Doe"
There will be no output. If you get any output, double-check the command and re-enter it.
Validation
git config --global user.name
The output should be the full name you put.
4.2. Set your email address
Replace "johndoe@example.com" with your ucf email address.
git config --global user.email johndoe@example.com
Validation
git config --global user.email
The output should be the email address you put.
4.3. Set your editor to emacs
git config --global core.editor emacs
Validation
git config --global core.editor
The output should be emacs.
4.4. Explicitly set the default branch name to master
This is the current default and what our grading tools will expect when pulling your projects.
git config --global init.defaultBranch master
Validation
git config --global init.defaultBranch
The output should be master.
5. Submitting your project
5.1. Confirm your location
Confirm that you are within your top-level compiler project directory with pwd
pwd
You should see
/home/net/NID/cop5621spring25/compiler/
Not compiler/compiler
5.2. Create the repo
git init
5.3. Add the remote
Add a remote called submission to the following URL, replacing NID with your own UCF NID.
git remote add submission gitolite3@eustis3.eecs.ucf.edu:cop5621/NID/compiler
5.4. Add the files
Use git add to add your Lex.py and other program files
git add compiler/__init__.py git add compiler/Lex.py git add grammar/__init__.py git add grammar/Makefile git add grammar/While.g4
and all of your test files, e.g.,
git add tests/test1.while
for all five (or more) tests.
5.5. Commit
git commit
Write a message in your editor
5.6. Push
Push the repo (first time)
git push --set-upstream submission master
Future pushes just do
git push