Skip to content

Commit

Permalink
Draft #8.1
Browse files Browse the repository at this point in the history
More spell-checking
  • Loading branch information
DavidVuescu committed Jun 29, 2023
1 parent 94584be commit a72ec51
Show file tree
Hide file tree
Showing 7 changed files with 22 additions and 22 deletions.
4 changes: 2 additions & 2 deletions Chapter 0 - Abstract.tex
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ \chapter*{Abstract}\label{cap:abstract_ro}

\chapter*{Abstract}\label{cap:abstract_en}
%%%%%%%%%%%% Section: Abstract %%%%%%%%%%%%
This bachelor work consists in realising a platform that works in conjunction with a Brain-Computer Interface(BCI), specifically the Unicorn Hybrid Black BCI\cite{Unicorn_Technology}, to facilitate access to applications compatible with the aforementioned platform. The platform aims to improve accessibility for a disabled person that is using a BCI as follows: normally when changing between applications the disabled subject must wait for an able-bodied helper to change applications and recalibrate the hardware; by creating a platform that houses applications the user may need and that allows the user to change between them seamlessly using an already existing solution\cite{Unicorn_Speller} the need for the said helper is removed, thus giving the end-user more freedom in interacting with his/her favourite programs.
This bachelor's work consists in realising a platform that works in conjunction with a Brain-Computer Interface(BCI), specifically the Unicorn Hybrid Black BCI\cite{Unicorn_Technology}, to facilitate access to applications compatible with the aforementioned platform. The platform aims to improve accessibility for a disabled person that is using a BCI as follows: normally when changing between applications the disabled subject must wait for an able-bodied helper to change applications and recalibrate the hardware; by creating a platform that houses applications the user may need and that allows the user to change between them seamlessly using an already existing solution\cite{Unicorn_Speller} the need for the said helper is removed, thus giving the end-user more freedom in interacting with his/her favourite programs.
\vspace{\baselineskip}\newline
Due to the fact that the application is targeted at impaired individuals, the need for an able-bodied helper will arise when it comes to adding new applications to the platform and/or removing existing ones should that need to arise. An installer tool was created to aid the helper in installing the BCI applications via a graphical user interface. The apps used to demonstrate the platform will be taken (with permission) from UVT's submissions in the br41n.io hackathons, in which students competed with ideas for applications that integrate Brain Computer interfacing.
Due to the fact that the application is targeted at impaired individuals, the need for an able-bodied helper will arise when it comes to adding new applications to the platform and/or removing existing ones should that need to arise. An installer tool was created to aid the helper in installing the BCI applications via a graphical user interface. The apps used to demonstrate the platform will be taken (with permission) from UVT's submissions in the br41n.io hackathons, in which students competed with ideas for applications that integrate Brain-Computer interfacing.
\vspace{\baselineskip}\newline
Accordingly, this paper consists of two main parts, the platform solution itself and the installer solution. In creating the platform, the Unity engine was employed for creating an easy-to-understand user experience and for interfacing with Unicorn's proprietary BCI speller\cite{Unicorn_Speller} using the C\# programming language. The installer tool is an MFC-type dialogue application written in the C++ programming language that uses batch files to download and manipulate the application files inside of the Windows file system. While both programs can be used separately, they create an easier user experience for both user groups if used together: the impaired and the helpers, consequently creating a complete solution.
2 changes: 1 addition & 1 deletion Chapter 1 - Introduction.tex
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ \section{Objectives}\label{sect:objectives}
\vspace{\baselineskip}\newline
All objectives hinge on the experience of the end-user and all features of this project are catered towards a disabled individual. This solution tries to prioritize accessibility and ease of use over other aspects. That being said, no testers from the target demographic have been employed in developing this solution, so it may or may not work as intended on the targeted user base. That being said, the implementation method may also work on other problems that require a platform controlled by the user.
\vspace{\baselineskip}\newline
Therefore we can conclude on 3 primary objectives: creating an easily understandable and navigable user experience, ensuring the stability of the proposed solution, and developing a way to integrate apps within the platform without the need to modify the apps themselves to fit in.
Therefore we can conclude with 3 primary objectives: creating an easily understandable and navigable user experience, ensuring the stability of the proposed solution, and developing a way to integrate apps within the platform without the need to modify the apps themselves to fit in.



Expand Down
12 changes: 6 additions & 6 deletions Chapter 2 - Theoretical Basis.tex
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ \subsection{Brain-Computer Interfacing}
A brain-computer interface(BCI) or brain-machine interface(BMI) is a computer-based system that acquires brain signals, analyses them and translates them into usable data or commands which are then exported to other devices to be used to other extents. By definition, a BCI uses some form or another of data processing and as such an EEG alone cannot be called a BCI\cite{Shih_2012}. BCI implementations vary based on the invasiveness of their data acquisition methods. As such, there are non-invasive BCIs (using electroencephalography, magnetoencephalography, electrooculography or magnetic resonance imaging), semi-invasive BCIs (using electrocorticography and endovascular methods) and invasive BCIs (using microelectrode arrays).

\subsection{Unicorn Hybrid Black BCI}
The BCI that will be used in the implementation of this paper is the Unicorn Hybrid Black. This is a non-invasive BCI using EEG technology. The 8 electroencephalogram electrodes are placed at the following fixed positions on the provided cap: Fz, C3, Cz, C4, Pz, PO7, Oz and PO8[see fig 2.3], in accordance with the 10/20 international system. Two extra electrodes marked L and R are to be attatched on the mastoid bones behind the user's ears and act as a "ground" for the rest of the electrodes due to their placement on an area with virtually no electrical activity.
The BCI that will be used in the implementation of this paper is the Unicorn Hybrid Black. This is a non-invasive BCI using EEG technology. The 8 electroencephalogram electrodes are placed at the following fixed positions on the provided cap: Fz, C3, Cz, C4, Pz, PO7, Oz and PO8[see fig 2.3], in accordance with the 10/20 international system. Two extra electrodes marked L and R are to be attached to the mastoid bones behind the user's ears and act as a "ground" for the rest of the electrodes due to their placement on an area with virtually no electrical activity.

\begin{figure}[H]
\centering
Expand All @@ -51,13 +51,13 @@ \subsection{Unicorn Hybrid Black BCI}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Technologies Used}
\subsection{Unicorn Suite}
The Unicorn BCI always comes alongside the Unicorn Suite, a software environment touched upon in the similar solutions section of this paper. In this implementation, the suite is used for its development tools and for the Unicorn Speller contained within. Other useful tools include the Unicorn recorder, the .NET framework which uses the C\# programming language and other tools that allow the creation of different software solutions. Another function of the Suite is to keep the licenses required to use the software, as most tools require licenses purchased directly from g.tec\cite{Unicorn_Shop}, including the Speller used in this implementation.
The Unicorn BCI always comes alongside the Unicorn Suite, a software environment touched upon in the similar solutions section of this paper. In this implementation, the suite is used for its development tools and the Unicorn Speller contained within. Other useful tools include the Unicorn recorder, the .NET framework which uses the C\# programming language and other tools that allow the creation of different software solutions. Another function of the Suite is to keep the licenses required to use the software, as most tools require licenses purchased directly from g.tec\cite{Unicorn_Shop}, including the Speller used in this implementation.


\subsection{Unicorn Speller}
The Unicorn Speller is a spelling system that uses the P300 complex\cite{UnicornSuite_Manual}. The speller's interface consists of a QWERTY key layout that sits on the screen along a number row, an auto-complete row and a special functions row[see Fig 2.4]. The numbers and letters are used for spelling while the special keys are used for other functions such as text-to-speech, printing, etc. Each key is considered an item, and the layout is called a board. The user can choose the board he/she wants to use, thus choosing the layout of the items on the screen. The speller also allows for creation of user-made boards (using the \textit{.ibc} file extension), the use of which will also be employed in this work.
The Unicorn Speller is a spelling system that uses the P300 complex\cite{UnicornSuite_Manual}. The speller's interface consists of a QWERTY key layout that sits on the screen along a number row, an auto-complete row and a special functions row[see Fig 2.4]. The numbers and letters are used for spelling while the special keys are used for other functions such as text-to-speech, printing, etc. Each key is considered an item, and the layout is called a board. The user can choose the board he/she wants to use, thus choosing the layout of the items on the screen. The speller also allows for the creation of user-made boards (using the \textit{.ibc} file extension), the use of which will also be employed in this work.
\vspace{\baselineskip}\newline
The actual mode of operation of the speller is through "flashing"\cite{UnicornSuite_Manual}, where the items on the board are flashing a different image for a brief moment (eg. the letter S is replaced with the face of Chuck Norris for a brief second before returning to its normal appearance). The images that flash on the letters are usually the faces of famous people. This is due to discoveries regarding P300 spellers, where researchers found out that superimposing familiar faces on letters helps accuracy and improves the performance of the ERP\cite{Li_2015}\cite{Kaufmann_2011}. In order to select an item, the user must focus on said item and count each time the item he/she wants to select flashes and ignore all other flashes\cite{UnicornSuite_Manual}.
The actual mode of operation of the speller is through "flashing"\cite{UnicornSuite_Manual}, where the items on the board are flashing a different image for a brief moment (eg. the letter S is replaced with the face of Chuck Norris for a brief second before returning to its normal appearance). The images that flash on the letters are usually the faces of famous people. This is due to discoveries regarding P300 spellers, where researchers found out that superimposing familiar faces on letters helps accuracy and improves the performance of the ERP\cite{Li_2015}\cite{Kaufmann_2011}. To select an item, the user must focus on said item and count each time the item he/she wants to select flashes and ignore all other flashes\cite{UnicornSuite_Manual}.

\begin{figure}[H]
\centering
Expand All @@ -67,7 +67,7 @@ \subsection{Unicorn Speller}

The speller flashing occurs in flash cycles, a group of flashes such as that each item on the board has flashed exactly once. There are different modes of flashing that the user can choose from as follows: row/column, single character and randomised patterns. To create the solution proposed by this paper, I will be using the randomised patterns mode, a randomised selection of items distributed across the whole board that flash at the same time, with patterns flashing consecutively\cite{UnicornSuite_Manual}.
\vspace{\baselineskip}\newline
Furthermore, to use the speller, a calibration sequence needs to be completed. In order to calibrate, we must input a word or different letters that the user has to look at while the calibration sequence runs[see Fig 2.4]. During calibration, the speller learns the difference between the user's typical EEG signals and afterwards, it creates a calibration file, storing the user's calibration data to be reused. In this way, the speller adapts to the user, each user having different calibrations depending on different variables such as the user's environment. Therefore, it is best to calibrate in a quiet place, away from distractions and disturbances that might introduce rubbish data and noise in the calibration file.
Furthermore, to use the speller, a calibration sequence needs to be completed. To calibrate, we must input a word or different letters that the user has to look at while the calibration sequence runs[see Fig 2.4]. During calibration, the speller learns the difference between the user's typical EEG signals and afterwards, it creates a calibration file, storing the user's calibration data to be reused. In this way, the speller adapts to the user, each user having different calibrations depending on different variables such as the user's environment. Therefore, it is best to calibrate in a quiet place, away from distractions and disturbances that might introduce rubbish data and noise in the calibration file.

\begin{figure}[H]
\centering
Expand All @@ -85,7 +85,7 @@ \subsection{Unity Game Engine}
\subsection{Visual Studio 2022 IDE}
Visual Studio 2022 is an Integrated Development Environment (IDE) used for editing, debugging, building code and publishing different applications. Visual Studio includes compilers, code completion tools, graphical designers and other features to enhance software development processes\cite{VisualStudio}. Visual Studio also provides plugins and different integration with external tools, such as the Unity Game Engine. In the making of this bachelor's work, the main Development Environment used is the 2022 version of this IDE, both as the default script editor for Unity and in other miscellaneous tasks as a code editor.
\vspace{\baselineskip}\newline
Due to the comprehensive support for Microsoft Foundation Class (MFC) applications, the IDE was also used to create the platform's installer, which is an MFC type application. Visual Studio has extensive tools for designing the MFC user interface and the logic behind it and was crucial in developing, building, running and deploying the platform's installer.
Due to the comprehensive support for Microsoft Foundation Class (MFC) applications, the IDE was also used to create the platform's installer, which is an MFC-type application. Visual Studio has extensive tools for designing the MFC user interface and the logic behind it and was crucial in developing, building, running and deploying the platform's installer.
\vspace{\baselineskip}\newline
For trivial and smaller code editing tasks, the simpler Visual Studio Code was used as a quicker alternative to the vastness of its bigger and more complex counterpart.
\vspace{\baselineskip}\newline
Expand Down
4 changes: 2 additions & 2 deletions Chapter 3 - Creating a BCI Application Platform.tex
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\chapter{Creating a BCI Application Platform}\label{cap:development}
In this bachelor's thesis, the main objective is creating a BCI App Platform, meaning the creation of a program that is able to open other applications while interfacing with the Unicorn Hybrid Black, a Brain-Computer Interface, to receive input from a disabled user via a P300 speller. The user can navigate through his favourite applications which are displayed on the platform's graphical user interface using the Unicorn Speller, g.tec's implementation of the P300 speller. Using the speller, he can then select an application to launch and use for as long as he desires, after which he can once again close it through the speller's board by looking at its elements. The results show a stable application that can be used to navigate through compatible apps and open them with ease.
In this bachelor's thesis, the main objective is creating a BCI App Platform, meaning the creation of a program that can open other applications while interfacing with the Unicorn Hybrid Black, a Brain-Computer Interface, to receive input from a disabled user via a P300 speller. The user can navigate through his favourite applications which are displayed on the platform's graphical user interface using the Unicorn Speller, g.tec's implementation of the P300 speller. Using the speller, he can then select an application to launch and use for as long as he desires, after which he can once again close it through the speller's board by looking at its elements. The results show a stable application that can be used to navigate through compatible apps and open them with ease.

\section{Platform Architecture}
The application to be implemented makes use of a BCI Speller, namely the Unicorn Speller which is part of the Unicorn Suite Hybrid Black. This speller can only be used with the Unicorn Hybrid Black Brain-Computer Interface. The speller is used as the primary and only input method for a disabled user to interact with the application. The BCI headset sends its output (the item the user selected from the board) through a UDP server to a specific IP address and port combo specified in the speller settings. The application implements a UDP listener that runs as a background process of the application, waiting for input from the user. When the aforementioned input arrives, the listener sends it to the application which then interprets it accordingly.
Expand Down Expand Up @@ -89,7 +89,7 @@ \subsection{Speller Board}
\end{figure}


The Platform speller consists of a full QWERTY-style keyboard and a navigation pad on the right-hand side of the keyboard[see Fig 3.3]. This may seem a bit excessive for a simple platform and that is because it is. The applications inside of the platform use the speller for input as well, all of them implementing their receivers. This would be a non-issue if the board file could be changed during run-time, but it cannot as the speller settings cannot be changed while the speller itself is running. Scripts could be created to automate this, but scripts can fail and in the event of such failure, an able-bodied person would have to step in, which is unsatisfactory. This creates an issue in the implementation where either all apps have to be designed to work with the same style of board (which would mean app integration takes longer, as each individual app's receiver code needs to be changed) or an umbrella board that covers all bases needs to be created. In this implementation, I have opted to create such a board that covers all needs.
The Platform speller consists of a full QWERTY-style keyboard and a navigation pad on the right-hand side of the keyboard[see Fig 3.3]. This may seem a bit excessive for a simple platform and that is because it is. The applications inside of the platform use the speller for input as well, all of them implementing their receivers. This would be a non-issue if the board file could be changed during run-time, but it cannot as the speller settings cannot be changed while the speller itself is running. Scripts could be created to automate this, but scripts can fail and in the event of such failure, an able-bodied person would have to step in, which is unsatisfactory. This creates an issue in the implementation where either all apps have to be designed to work with the same style of board (which would mean app integration takes longer, as each app's receiver code needs to be changed) or an umbrella board that covers all bases needs to be created. In this implementation, I have opted to create such a board that covers all needs.


\subsection{Speller Receiver}
Expand Down
Loading

0 comments on commit a72ec51

Please sign in to comment.