How far can you go in building a “psychic” app without having to wait for Elon’s Neuralink?
After seeing recently announced breakthroughs Elon Musk had with Neuralink, I got very excited and decided that I couldn’t wait to see what it can do! No really, I couldn’t wait and decided to take a risky dive into what I called a poor man’s Neuralink.
There are some significant restrains on this project, hence the name. Namely, it must be cheap, widely available in terms of hardware, and user friendly as much as possible (no brain surgery required!).
Let’s consider the next use case
I want the app to be able to predict my thoughts and say them out loud (using Google’s text-to-speech) while I’m trying to visualize them sitting comfortably in my chair. I need to be able to train the app to learn more “words” and expend it’s vocabulary as I record more patterns. Brain wave signals are to be picked from some EEG device and sent to the app via Bluetooth. App will record, and store locally the EEG patterns as sessions in csv. The app should not require an internet connection to work.
Shortly, make the app read my mind and guess what I’m thinking of (kinda)!
Hardware and software
Consumer-level EEG devices are not a scarce market. For years only one stuck to my mind and it turns out it’s the cheapest option – MindWave Mobile.
For programing the app I almost succumbed and went with Java to process Bluetooth signals, but after some careful research, I decided that an amazing tool from MIT like AppInventor could do the job just as likely.
Following KISS (keep-it-simple-stupid) guidelines I had to roll my sleeves up, simplify the whole thing and do some visual programming if I ever hope this to be read and used by other human beings.
Now, that we have it laid out in clear terms lets create our mind reader app!
Amazing UI, right?
Wait, but what data does this device provide with only one passive sensor on your forehead and at what speed?
MindWave technical specs
- Outputs 12 bit Raw-Brainwaves (3 – 100Hz) with sampling rate at 512Hz.
- Uses the TGAM1 module.
- Outputs EEG power spectrums (Alpha, Beta, etc.) – this is what we need raw data!
- Outputs NeuroSky proprietary eSense meter such as Attention, Meditation, and other future meters (we won’t be using this).
- EEG/ECG signal quality analysis (can be used to detect poor contact and whether the device is off the head).
Signal sampling in AppInventor
Building block of a function called Read_MindWave that reads data and sends it to parsing.
After some hours of point-and-click programing here is the complete app:
Training the algorithm
- Be calm and don’t move while training the app
- Calm yourself before clicking train
- Decide on a word to train the algorithm (object, command or whatever you want, but make it one word only eg. triangle, blue, up, down etc…)
- Say the word after pressing train
- Concentrate on the subject for at least 5 seconds (can repeat it later, the more data the better)
- Drop the threshold as vocabulary expands to tweak the similarity/difference trigger.
Predicting
Train at least 3 different terms before trying prediction. The algorithm has to guess which specific word you are thinking of out of it’s vocabulary. Concentrate and think of a thing you already trained the algorithm to recognize and text-to-speech will say the word every time the similarity threshold is reached between trained and/or new state of EEG patterns.
Here is the complete .aia project file so you can load it into your AppInventor account and not only learn about how it’s built, but also export it as a regular Android app and use it (if you happen to own one of these simple EEG devices).
If you are still curious about this, you can take a peak into my own mind waves as I think about different fruit like apple, banana etc… in further analysis with different algorithms and bigger dataset in this repo.
That’s it! I hope you enjoy it as much as I did.