P13541: Eye Tracker
/public/

Planning & Execution

Table of Contents

Project Summary

The main goal of our project is to create a tablet-based eye tracker. The reason behind this is that there are two main categories of eye trackers that currently exist. The first is the remote eye tracker, which attaches to a desktop computer and thus is not portable. The second category is a pair of eyeglasses with integrated cameras. The eyeglass solution is more portable, but requires the user to wear a laptop backpack.

The goal for this project is to create a tablet-mounted device that houses at least one camera to track the user’s eye. It will be used in conjunction with the tablet’s built-in rear camera in order to determine what the user is looking at.

Intellectual Property Considerations

Any IP generated by the students during the course of this project will belong to the students.

Customer Needs

Customer needs are based on a 1,3,9 scale with 9 being the most important.
Customer Need # Importance Description
CN1 3 Uses at least two cameras: at least one to capture user's eye, and one to face outward from user
CN2 9 Feeds the captured images to the tablet.
CN3 9 Provides visual feedback – plots the user’s gaze points overlaid on the image on the tablet’s display
CN4 9 Provides an eye image, accessible on the tablet display, for debugging purposes
CN5 9 Operates successfully when a user is hand-holding the tablet
CN6 3 Operates successfully in both portrait and landscape orientations
CN7 9 Provides calibration routine
CN8 9 Portable
CN9 9 Operates successfully real-time
CN10 3 Easy to use
CN11 1 Easy to train user
CN12 9 Safe to use
CN13 3 Parts cost <$700
CN14 1 Uses tablet battery as power source
CN15 3 Operates successfully for users wearing glasses
CN16 9 Operates successfully indoors
CN17 3 Operates successfully outside
CN18 1 Attractive

Specifications

Customer Needs vs. Specifications comparison chart:
S# Source Function Specification (metric) Unit of Measure Marginal Value Ideal Value Comments/Status
S1 CN8,10,12 System Size inches <= 4x2x1 Total of added components
S2 CN8,10,12,14 System Weight lbs <= 0.25 Total of added components
S3 CN1,13 System System cost $$ < 700
S4 CN14 System Device is untethered (not plugged into the wall) Yes
S5 CN5,6 System Allowable tilt degrees <=5
S6 CN2,9 System Operating rate frames/sec 10 30
S7 CN1,4 System Size of the eye in captured image pixels 320x240
S8 CN3,5,6,7,9,15,16,17 System Accuracy of gaze detection Degrees of visual angle <=5 <=2
S9 CN14 System Uses tablet battery as power source External 7.4V Lithium ion battery Yes
S10 CN12,18 System No exposed circuitry Yes
S11 CN16,17 System Operating conditions: temperature deg C 10-30 Ambient indoor
S12 CN16,17 System Operating conditions: relative humidity % 10-80 Ambient indoor
S13 CN7,10,11 System Total Calibration time min <10 <5
S14 CN11,12 System Training time (1st time) min < 30
S15 CN3,4,5,6,7,9,15,16,17 System Reliability TBD
S16 CN12 System Internal (enclosure) temperature deg F < 120
S17 CN8,14 System Time Operations hr >1/2

Team Norms & Values

Code of Ethics

What are your expectations for team behavior?

Our team is expected to inform other teammates if a problem arises in a team or individual task. Our team is also expected to maintain progress and consistently be updated on project activities at all times.

How will you ensure balanced distribution of work?

We will attempt to work together on problems that arise. Also we will distribute work based on skill without overloading any single member.

How will you determine roles and responsibilities?

We will determine the strengths and weaknesses of each member and determine which responsibilities correspond with each member's strengths

How will you honor the strengths of the individuals?

We will make sure the individual has work in their area of expertise.

How will you interact with your Guide?

We will meet with him at scheduled times and make sure he is aware of any problems ahead of time.

How will you communicate with each other?

We will communicate through phone, email, google docs, and texts.

How will you make decisions?

For broad issues, our decisions will be decided by majority. For technical issues, we will defer to the expert.

What are the expectations for integrity or responsibility with assigned tasks?

We expect each member to be honest and not lie about problems that they are facing. Also we expect to set realistic goals that should be accomplished by the end of senior design II.

How will you manage conflict or when these expectations are Not met?

We will defer the conflict to the team leader. If the problem continues will we bring the issue to the team or guide.

Project Plans & Schedules

MSD I & II: WBS, Gantt charts, schedule, etc

Gantt Chart

MSD I Gantt Chart
public/Photo Gallery/Gantt Chart.png

MSD II Gantt Chart

Green - Completed

Yellow - In Progress

Red- Not Started

Meeting Minutes, Notes, & Actions

MSDII Week 2 Meeting Notes

Completed

Meeting with Susan to Review Usability; Remove IR Filter from Camera; Prototyping Circuit for IR;

In Progress

Camera Recognized by Tablet- still trying to get the camera to take pictures, also not using the logitech camera purchased by the team; Experiment with EMGU CV- Matt discovered a major issue with the software, may need to use Open CV instead; Testing IR- some testing has been complete; Ordering PCB - Delayed due to uncertainty about the amount of PCB's needed; Set up EMGU CV on tablet- Brad said he is currently working on it and it should be completed with in the upcoming week; Testing Usability- should be completed by the end of week three;

Not Started

Setting up developing Environment- not started yet but should not require a lot of time; Meeting with Susan to Measure IR- Mike plans on meeting with Susan today at 10; Optimize Camera Focus- Delayed due to wrong film type- need to use developed color film; Design Mount- Delayed due to new issue with software, may need to redesign mount to add another; PCB, placement of PCB is uncertain as well;

Matt, Brad, Mike and Hemsley plan on meeting with Susan today at 10 to discuss issues with software, Measuring IR, and camera issues.

MSDII Week 3 Meeting Notes

Completed

Meeting with Susan to Review Usability; Remove IR Filter from Camera; Prototyping Circuit for IR; Setting up developing Environment; Optimize Camera Focus- Delayed due to wrong film type- needed to use develop color film; Ordering PCB - Delayed due to uncertainty about the amount of PCB's needed- should be completed today; Testing Usability- had 15 people run four calibration routines and collected the data- just need to run a statistical analysis to determine the best calibration method; Program Both Camera's Working at Same Time- a similar logitech camera and the tablet camera are running at the same time;

In Progress

Camera Recognized by Tablet- a similar logitech camera is working with the camera and both cameras are able to run at the same time- needs to get our camera running on the camera but has not had access to our camera to do so; Experiment with CV- Matt discovered a major issue with the software, may need to use Open CV instead - still currently experimenting and may take a few weeks to complete; Testing IR- some testing has been complete- the video is working with the IR;

Not Started

Meeting with Susan to Measure IR- Mike plans on meeting with Susan next week on Tuesday; Set up Open CV; Design Mount- delayed because the PCB's have not been ordered; Fill PCB- PCBs have not been ordered; Integration of all Components- Can't complete because the PCBs have not been ordered; Test PCB- same issue;

Optimize Camera Focus- Delayed due to wrong film type- need to use developed color film; Design Mount- Delayed due to new issue with software, may need to redesign mount to add another PCB, placement of PCB is uncertain as well;

MSDII Week 4 Meeting Notes

Completed

Meeting with Susan to Review Usability; Remove IR Filter from Camera; Prototyping Circuit for IR; Setting up developing Environment; Optimize Camera Focus- Delayed due to wrong film type- needed to use develop color film; Ordering PCB - Delayed due to uncertainty about the amount of PCB's needed- should be completed today; Testing Usability- had 15 people run four calibration routines and collected the data- just need to run a statistical analysis to determine the best calibration method; Camera Recognized by Tablet- the logitech camera is working with the camera and both cameras are able to run at the same time- there is a little bit of latency on the webcam; Meeting with Susan to Measure IR- Mike says it's should be safe to use;

In Progress

Experiment with CV- Matt discovered a major issue with the software, may need to use Open CV instead - still currently experimenting and may take a few weeks to complete; Testing IR- some testing has been complete- the video is working with the IR- having trouble quantifying the difference between top and bottom. The closer they are together the harder it is to recognize two glints; Fill PCB- just need to fix a slight problem, should be a quick fix; Learning how to Make User Interface - in progress, Hemsley will be helping with this;

Not Started

Set up Open CV; Design Mount- delayed because the PCB's have not been ordered; Integration of all Components- We should be able to do this soon; Test PCB- They should work but not tested yet; Design Final mount - Haven't been able to work on this but I should have something next week; Design User Interface- will be working on it soon after the mount is designed; Paper- need to start; Power PCB boards;

MSDII Week 5 Meeting Notes

Completed

Meeting with Susan to Review Usability; Remove IR Filter from Camera; Prototyping Circuit for IR; Setting up developing Environment; Optimize Camera Focus- Delayed due to wrong film type- needed to use develop color film; Ordering PCB - Delayed due to uncertainty about the amount of PCB's needed- should be completed today; Set up Open CV; Testing Usability- had 15 people run four calibration routines and collected the data- just need to run a statistical analysis to determine the best calibration method; Camera Recognized by Tablet- the logitech camera is working with the camera and both cameras are able to run at the same time- there is a little bit of latency on the webcam; Meeting with Susan to Measure IR- Mike says it's should be safe to use; Test PCB- they work; Power PCB boards;

In Progress

Experiment with Open CV- Matt was able to find the pupil with the software. Matt displayed several pictures showing glints; Testing IR- some testing has been complete- the video is working with the IR- having trouble quantifying the difference between top and bottom. The closer they are together the harder it is to recognize two glints.; Design Mount- temporary mount was made but the final mount is still in progress; Fill PCB- just need to fix a slight problem, should be a quick fix; Learning how to Make User Interface - in progress, Hemsley will be helping with this;

Not Started

Integration of all Components- We should be able to do this soon; Design Final mount - Haven't been able to work on this but I should have something next week; Design User Interface- will be working on it soon after the mount is designed; Paper- need to start; Design Test Fixture;

MSDII Week 6 Meeting Notes

Completed

Meeting with Susan to Review Usability; Remove IR Filter from Camera ; Prototyping Circuit for IR; Setting up developing Environment; Optimize Camera Focus- Delayed due to wrong film type- needed to use develop color film; Ordering PCB - Delayed due to uncertainty about the amount of PCB's needed- should be completed today; Set up Open CV; Testing Usability- had 15 people run four calibration routines and collected the data- just need to run a statistical analysis to determine the best calibration method; Camera Recognized by Tablet- the logitech camera is working with the camera and both cameras are able to run at the same time- there is a little bit of latency on the webcam; Meeting with Susan to Measure IR- Mike says it's should be safe to use; Test PCB- they work; Power PCB boards; Integration of all Components- we have combined all of the parts; however, not everything is working together yet.; Experiment with Open CV; Fill PCB- completed with spares; Testing IR- Completed;

In Progress

Design Mount- temporary mount was made but the final mount is still in progress; Learning how to Make User Interface - in progress, program crashed but Mike has made some good progress; Working on Calibration- should allow us to move towards gaze tracking; Design Final mount - talking to people in the machine shop to help design and manufacture the mount;

Not Started

Design User Interface- will be working on it soon after the mount is designed; Paper- need to start - Outline is in progress; Design Test Fixture; Poster Board- needs to be complete by week 9;

MSDII Week 7 Meeting Notes

Completed

Meeting with Susan to Review Usability; Remove IR Filter from Camera; Prototyping Circuit for IR; Setting up developing Environment; Optimize Camera Focus- Delayed due to wrong film type- needed to use develop color film; Ordering PCB - Delayed due to uncertainty about the amount of PCB's needed- should be completed today; Set up Open CV; Testing Usability- had 15 people run four calibration routines and collected the data- just need to run a statistical analysis to determine the best calibration method; Camera Recognized by Tablet- the logitech camera is working with the camera and both cameras are able to run at the same time- there is a little bit of latency on the webcam; Meeting with Susan to Measure IR- Mike says it's should be safe to use; Test PCB- they work; Power PCB boards; Integration of all Components- we have combined all of the parts; however, not everything is working together yet.; Experiment with Open CV; Fill PCB- completed with spares; Testing IR- Completed; Design Mount- Mount has been designed; just waiting on the plastic to start manufacturing;

In Progress

Learning how to Make User Interface - in progress, Has developed the menu and the instructions for calibration. Working on Debug and adding the calibration routine into the code; Working on Calibration- Detect the eye region, the pupil, and the glints- still can't eye track; Design Final mount - talking to people in the machine shop to help design and manufacture the mount- just waiting on the plastic; Detect the eye region, the pupil, and the glints- still can't eye track; Paper- need to start - Outline is in progress- some info on drive, but we need significant progress in the next week;

Not Started

Design User Interface- will be working on it soon after the mount is designed; Design Test Fixture; Poster Board- needs to be complete by week 9;

MSDII Week 8 Meeting Notes

Completed

1.Meeting with Susan to Review Usability; 2. Remove IR Filter from Camera; 3. Prototyping Circuit for IR; 4. Setting up developing Environment; 5. Optimize Camera Focus- Delayed due to wrong film type- needed to use develop color film; 6. Ordering PCB - Delayed due to uncertainty about the amount of PCB's needed- should be completed today; 7. Set up Open CV; 8. Testing Usability- had 15 people run four calibration routines and collected the data- just need to run a statistical analysis to determine the best calibration method; 9. Camera Recognized by Tablet- the logitech camera is working with the camera and both cameras are able to run at the same time- there is a little bit of latency on the webcam; 10. Meeting with Susan to Measure IR- Mike says it's should be safe to use; 11. Test PCB- they work; 12. Power PCB boards; 13. Integration of all Components- we have combined all of the parts; however, not everything is working together yet.; 14. Experiment with Open CV; 15. Fill PCB- completed with spares; 16. Testing IR- Completed; 17. Design Mount- Mount has been designed; just waiting on the plastic to start manufacturing;

In Progress

1. Design Final mount - the case and covers are complete. the mount system is in progress.; 2.Design User Interface- in progress, will test the interface and determine any problems; 3. Learning how to Make User Interface - in progress, Has developed the menu and the instructions for calibration. Just needs to integrate it into the program in order to finish the system.; 4. Working on Calibration- Detect the eye region, the pupil, and the glints- still can't eye track- much more accurate than last week; 5. Detect the eye region, the pupil, and the glints- still can't eye track; 6. Paper- Made significant progress this week. Roughly 6 pages complete, with the bulk of the material done. Just need more pics and conclusions; 7. Poster Board- we have general information completed, just need to fill in the detailed sections. needs to be complete by week 9;

Not Started 1.Test movable Eye-Tracker; 2. Improve Eye Tracking;

Home | Planning & Execution | Systems Design | Detailed Design | Build, Test, Document | Photo Gallery | Project Review | Technical Paper | Poster | Final Results and Handoff