Developing Input Elements for Eye-Controlled Application
Mark Laane
This project is linked to my final thesis, where an interactive, fully eye-controlled browser application is developed. The application needs input elements (buttons, dropdown menus, checkboxes) that can be used solely with eyes. In this project I will only focus on developing input elements, separate from any application. As a result, existing
Milestone 1 (01.03)
Main goal of this milestone is to create the base of the demo/development application and tranfer at least one eye-controlled input element to it. For this a new JavaScript project is created and currently developed input elements are extracted from an existing eye-controlled application and placed in to the new application. As the main goal of this aplication is to speed up development process, following features are critical: Hot Module Reloading (HMR) for fast development feedback, linting for enforcing good code style, and source maps for debugging.
progress
- Application was created with ES6, HMR, linting and source-map support.
- Eye-controllable buttons and dropdowns were transferred to the new application
- I realised how over-engineered dropdown code was. Rewrote it.
- Moved from CSS to SCSS and started using a color palette.
- Redesigned buttons - user focus is displayed with text color change.
- Created "primary", "warning" and "disabled" variations of the button
Milestone 2 (15.03)
Rest of the elements from application are added to demo
- Text input
- Keyboard for enterin textual data.
Two new input elements are created:
- a button that displays "confirm" button next to itself, when user looks at it.
- a button that displays 2/3 dots, when user looks at it. the button is "pressed" when user activates all the dots by looking at them.
progress
- Added Text input
- Added Keyboard
- Created a button that displays 2 dots on its sides when it is looked at. The user first needs to "unlock" the button by glancing both dots. After unlocking the user can look at the button itself to activate it.
Milestone 3 (29.03)
- Investigate how smooth animations are done in React.
- Implementation of "shrinking letter" on keyboard.
- Investigate visual glitches:
- buttons with confirmations stay "activated"
- keyboard buttons do not "flash" when used with mouse
Progress
- Added smooth animation of opening/closing keyboard using ReactCSSTransitionGroup
- Added shrinking letter - while it does provide feedback, it is also very distracting and causes a lot of visual noise. delayed shrink helps a bit.
- Keyboard now gives visual feedback properly regardless of whether it is being used by eyes or keyboard. (Did not flash when used with mouse)
- Separated "activated" and "pushed" state
Milestone 4
- Realization of keyboard Cancel/Enter functionality: Keyboard should hold the editable value internally not write it directly to the input field, so Cancel/Enter functionality could be realised
- Keyboard should have backspace next input text, not as one of the buttons. While deleting, user wants to check the result frequently.
- Keyboard should also have "delete by word" and "Clear" buttons.
- Have another go at fixing the Confirmable button
Progress
- Keyboard has backspace next to input, "Delete Word" button and also a "Clear" button
- Confirmable button - fixed - works as expected
Milestone 5
Next week I am doing some user testing, so i want to make adjustments based on feedback collected
- Make adjustments according to feedback collected.
Now that Delete buttons are really accessible and destroy more than one character, undo/redo functionality is really needed, so:
- Implement the cancel/enter or undo/redo functionality for keyboard from Milestone 4
Currently, I am managing two code-bases, one for thesis-work and one for this project. Whenever i get something ready in this project, I manually transfer the code/functionality.
- Investigate how to make the components reusable through NPM (privately), so the package could be just imported to reuse the functionality.
Progress
Adjustments based on feedback:
- Updated the form fields. It now has a placeholder that eyes can fixate better than a blank space. Now it is more consistent with the overall design - having no borders and it highlihts on fixation.
- Eye-selectable objects make an auditory "click" sound in addition to highlighting, to give more feedback. Testing revealed that users were frequently confused wether they had clicked something or not - this could aid them.
- Keyboard now displays a label of the field being edited, and delete buttons are not so terrible any more.
- Testing revealed, that users did not notice the "EyePositionDisplay" i have in my project. So i extracted it from my thesis an worked on it, in order to make it more noticeable and understandable. Now it grows, if errors last for longer than a second. It also suggests a solution, "move closer" etc.
- Investigated the possibility of having a private repo, but I am not yet sharing components.
- undo/redo: not done
Milestone 6
- Package
- Documentation
- plugin download