Meta’s Brain2Qwerty v2: Mind-Reading AI Without Surgery

Meta AI has introduced its latest innovation, Brain2Qwerty v2, which is a non-invasive brain-to-text BCI capable of translating typed sentences from brain signals without the need for any surgical implants. The development of this system was done by Meta’s FAIR research division in collaboration with Basque Center on Cognition, Brain and Language (BCBL).

What Is Brain2Qwerty v2?

Brain2Qwerty v2 is a machine learning algorithm designed to translate natural sentences through MEG, which is a non-invasive way to measure the brain’s magnetic field generated by neurons. Brain2Qwerty v2 differs from the previous version, Brain2Qwerty v1, in that it can decode whole sentences without needing precise timing of keystrokes and generate them in real-time by combining letter-level, word level and sentence-level decoding in a hierarchical manner.

Brain2Qwerty v2’s architecture consists of three parts: CNN which extracts features from raw MEG signals, transformer which models longer brain signal ranges and character level language model which helps to make the results closer to human language.

Meta researchers also fine-tuned large language models using neural data and AI agents which were used to iteratively tune the decoding pipeline. Engineers manually selected the training pipeline that worked the best.

How the AI Was Trained

Brain2Qwerty v2 was trained using 22,000 sentences provided by nine volunteer participants who wore MEG devices for 10 hours of typing. Compared to the first version, this amount of data is 10 times bigger per participant.

It should be noted that this system decodes brain activity which is associated with physically typing. Therefore, this system is not passively decoding arbitrary silent thoughts.

Accuracy: How Good Is It, Really?

According to Meta, Brain2Qwerty v2 has 61% word accuracy, a tremendous improvement over the 8% word accuracy of the older version of the system. If we look at the single best-performing participant of the study, then the accuracy reaches 78% in this case and more than half of all sentences contain no more than one mistake. Overall, this corresponds to a 39% average word error rate.

Meta states that decoding accuracy grows log linearly as more training data is collected, with no evidence of plateau detected. Such behavior indicates that the gap between non-invasive and invasive (implant-based) BCIs like stereoelectroencephalogram and electrocorticogram may get narrower with increased amounts of training data.

Why Meta Built This

According to Meta, the purpose of this system is to help millions of people who suffer from brain lesions or neurological problems preventing them from normal speech or typing. While invasive brain computer interfaces such as Elon Musk‘s Neuralink have already proved that AI-based implants are able to provide the ability to communicate again, surgery remains a huge obstacle preventing such technologies from widespread implementation. Non-invasive alternatives, even with poor accuracy, may have much bigger potential.

To facilitate outside research, Meta decided to open-source training code of Brain2Qwerty v1 and v2 and its research partner, BCBL, released the dataset of v1 publicly.

The Big Limitation: The Scanner Itself

The most refreshing thing in this research paper is that Meta is honestly discussing the main limitations that prevent this technology from reaching real patients. First of all, the accuracy of translation is currently insufficient for practical applications since there are too many word and character level errors.

Secondly, the system requires a huge and expensive room-based MEG scanner which is out of reach for most patients. However, Meta sees some hope here because the direction in which the MEG technology is evolving is towards portable scanners.

TheTweaks Analysis

One point that most people forget when discussing this system is that Brain2Qwerty v2 is not a “mind reading” breakthrough, it’s a data scaling breakthrough. As indicated by Meta’s research, accuracy increased log linearly simply due to the collection of more hours of data per participant. Therefore, this is actually the most interesting part of this research. Architecture wise (CNN+transformer+language model) Brain2Qwerty v2 is quite conventional for modern AI research, but increase in data and language model smoothing are the most significant improvements between v1 and v2.

The main limitation of the system discussed by Meta is its scanner. In fact, software can improve itself with additional data, but a bulky and room based MEG scanner is the actual bottleneck for turning this impressive lab result into a real communication device that can help patients. Without portable MEG hardware which is affordable, Brain2Qwerty v2 remains in the research phase and all viral claims about “thought-reading” by Meta get well ahead of the science. The technology that needs to be carefully watched in future is the wearable MEG hardware.