Building Signal2Speech: answering questions with an EEG headset at Hack the Nest 2026 (Update 1)

Note: This is only the first of many updates. We will also include a final post which will summarize the whole event. We are Order of the Golden Eagles, a team of four at Hack the Nest 2026 (Eastern Michigan University), and we are building Signal2Speech. It is an accessibility prototype for people who have difficulty speaking or typing. A conversation partner asks a question, the app offers a few short replies, and the person wearing an EEG headset (a cap that reads electrical activity from the brain) picks, previews, and confirms the words they want spoken. Our main rule is that the app suggests what to say, but the wearer decides what actually gets said. We are about five and a half hours in. This post covers where the project stands and a few things we ran into along the way. How it works The wearer sees two choice tiles that flash. To pick one, we use a P300 chooser. The P300 is a brain response that shows up about 300 milliseconds after the thing you’re watching flashes, so the app averages several flashes and classifies them with Linear Discriminant Analysis (LDA). The chosen reply then shows up on a “Say this?” preview, and nothing is spoken yet. To confirm, the wearer closes their eyes for about two seconds. Closing your eyes raises the brain’s alpha rhythm, which our alpha switch turns into exactly one Confirm. Only then does the app speak. The same controls run two features. AI Answering suggests replies to a typed question using Claude, and it falls back to saved phrases like “Yes.” and “No.” if the AI is slow or offline. OpenOrbit lets the wearer explore Earth data. Our demo question is which Michigan county’s Air Quality Index (AQI) rose more when wildfire smoke arrived in June 2023. The answer comes from NASA satellite images and the EPA’s daily county AQI: Washtenaw rose 186 points, and Wayne rose 168. The pivot We started out planning to build in Unity with g.tec’s starter. About an hour and a half in, we switched the whole project to Python, g.Pype (g.tec’s Python toolkit for EEG), and PySide6. The main reason was that g.Pype lets judges actually see the signal pipeline, including OSCAR artifact removal (cleaning out noise from blinks and muscle movement). The selection rules we had already written carried over with no changes because they were written as a shared scenario file that every input has to pass, not built into one engine. What works right now The full choose, preview, and confirm flow runs end to end on keyboard input and on g.Pype’s simulated brain. OpenOrbit has its real data bundled in the repo, so the demo works with Wi-Fi off. We also added a NASA FIRMS map showing where the smoke came from. One thing we learned from that data is that southeast Michigan had no wildfire detections on either day. The fires were actually in Quebec and Ontario, and only the smoke reached Michigan. If you want to try it without a headset, it runs on the keyboard: git clone https://github.com/Order-of-the-Golden-Eagles/Signal2Speech.git cd Signal2Speech pip install -r requirements-lock.txt python -m signal2speech.app --input keyboard Press 1 or 2 to pick a tile, 1 to confirm, and 2 or Esc to cancel. A strip along the bottom always shows whether the input is Keyboard, Simulated, Recorded, or Live EEG, so simulated input is never mistaken for the headset. A bug worth sharing The P300 chooser and the alpha switch each made their own live EEG adapter, and both numbered their events e1, e2, and so on. Because of this, when the alpha switch sent its Confirm, the app saw the same ID as the P300 preview and rejected it as a repeat. Rejecting repeats is on purpose, since a repeated signal should never make the app speak twice, but here it meant the live flow could never actually confirm. The fix was one counter shared by every adapter, so IDs are unique for the whole run. A new test now has two live adapters preview and then confirm. Keeping it safe Since the app speaks for someone, the worst failure would be a late or repeated signal saying something the wearer never chose. Only one place in the code can speak or change the map, a decision left over from an old round is thrown away, and an unanswered preview cancels itself after 15 seconds. The preview now shows that countdown, so the wearer and the people around them can see it. This follows the fail-safe defaults principle: when something is unclear, the app does nothing instead of guessing. On the privacy side, only the typed question is ever sent to the AI, never EEG data. Shared git hooks block API keys and raw EEG recordings from being committed, and every pull request has to pass CI on Windows, macOS, and Linux before it can merge. EEG is personal health data, so there is absolutely no reason for it to end up in a public repo. What’s next The P300 chooser and the alpha switch both work on their own, but they aren’t wired into the app with the real headset yet. So far, everything has been tested on simulated or synthetic brain data. Next, we will connect the g.tec Unicorn Hybrid Black, calibrate it on an actual wearer, and run the full flow live. After that, we will record the demo video. Questions Please leave any questions you have in the comments! We would love to discuss our project! Repo https://github.com/Order-of-the-Golden-Eagles/Signal2Speech