NeuroDesk: How Little Input Does a Computer Actually Need?

Imagine you are standing in a queue with a coffee in one hand and a bag in the other. A long-running coding agent has been working on a failing test for the last twenty minutes. You glance at a small notification, hold your gaze for a moment, make a small deliberate movement, and a short proposal appears: investigate the failing test in the payments module and report back. You look at it, confirm it, and put your phone away. You have not typed anything. You have not said anything out loud. Nobody around you noticed.

That scene is not a demonstration. NeuroDesk is currently an early concept and architecture project — there is documentation and a standalone project configurator, but no implemented runtime, no sensor pipeline, no AI gateway, and no hardware prototype. The scene describes the experience we want to test, and the rest of this post is about why that experience seems worth investigating and what has to be true for it to work.

The research question underneath it is deliberately narrow: how little reliable human input is needed to direct a capable AI agent?

The problem with the devices we have

Keyboards, mice, and voice assistants are excellent, and none of them is going anywhere. But each assumes conditions that are not always available.

A keyboard assumes free hands and a surface. A mouse assumes both, plus fine motor control. Voice assumes you are somewhere you can speak, that the people around you don’t mind, and that you don’t mind them hearing. Every one of those assumptions fails regularly: walking, cooking, carrying something, sitting in a shared office, riding public transport, being in a meeting, being in a quiet home at night with someone asleep in the next room.

There is also a group of people for whom conventional input is difficult or exhausting rather than merely inconvenient. Their needs are not an afterthought here, but they are also not a solved problem, and I will come back to why.

Something else has changed alongside this. When computers only did what you told them step by step, input bandwidth was the bottleneck — you needed a keyboard because you had a lot of characters to transmit. As AI systems become able to carry out longer, more open-ended tasks, the balance shifts. Increasingly the valuable human contributions are specifying a goal and reviewing a result, not entering each individual instruction. Those two acts might not need a keyboard’s worth of bandwidth.

What the experience could feel like

The interaction NeuroDesk is exploring has a consistent shape, whatever the task.

Something presents itself as a possible target — a notification, an item in a short list, a device in the room. Your attention lands on it. Then you make a small, deliberate signal to say that one. Contextual AI turns that minimal selection into a concrete proposal for work. You see the proposal. You approve it, adjust it, or cancel. If you approve it, something happens, and you get told what happened.

Three examples of the same loop:

Checking on a coding agent. You are away from your desk. An agent has stalled. You select the alert and confirm a proposed next step: look into why this is failing and summarise it. You review the summary later, at a keyboard, where reviewing is easy.
A discreet personal assistant. In a meeting, you want to note a follow-up, or ask a private assistant something you would not say out loud in the room. A silent selection is more appropriate than speech here — though silence is a matter of social discretion, not a privacy guarantee.
A familiar smart-home device. Hands full, you look at a lamp or a speaker and select the one action you almost always want from it. The vocabulary is tiny and the context does the rest.

Coding agents are one application. The broader objective is interacting with a computer without traditional input devices at all.

How it works, conceptually

Several kinds of signal can feed the same intent, and the AI layer sits between that intent and anything that actually runs.

flowchart LR
    A["Eye movement"] --> D["Deliberate user intent"]
    B["Subtle gestures"] --> D
    C["Experimental neuro-signals"] --> D
    D --> E["Contextual AI"]
    E --> F["Proposed action"]
    F --> G["User review and authorization"]
    G --> H["Computer or connected device"]
    H --> I["Feedback to the user"]

The diagram’s key point is that no single input method carries the whole burden, and nothing reaches a device or a piece of software without passing through a step where the person reviews and authorises it.

The complementary signals, in plain language:

Eye movement is good for navigation — moving among candidates, indicating what you are currently considering.
Subtle muscle activity or small gestures are good for selection — a discrete, intentional act that you either did or did not perform.
Experimental brain-signal input may be useful for constrained choices, where the system only needs to distinguish between a small number of options.

Not every interaction needs all three. A given task might use one. The point of having several is that each is asked to do the thing it is plausibly good at.

The most important distinction in the whole design is between looking and selecting. Attention must never, on its own, become permission to act. If merely resting your gaze on something could trigger it, the system becomes a minefield — you would have to be careful about where you looked, which is an absurd thing to ask of a person. Looking is a proposal for consideration. Selecting is an act.

What the AI is and is not doing

The AI’s job is to take a small, contextual selection and expand it into a richer, well-specified task, using context you have authorised it to see — what you are working on, what is on screen, what device you are near, what happened a minute ago.

It is not decoding thoughts. It is not inferring intentions you never expressed. When the selection is ambiguous, the correct behaviour is to ask, not to guess and act.

The loop, end to end

flowchart TD
    A["User identifies a target"] --> B["Deliberately selects a request"]
    B --> C["AI interprets the available context"]
    C --> D["User reviews the proposed action"]
    D --> E{"Approve?"}
    E -->|Yes| F["Authorized action runs"]
    E -->|Revise| B
    E -->|Cancel| G["No action"]
    F --> H["Result returned to the user"]

Two things matter in that flow. The first is that revision loops back rather than dead-ends: getting it slightly wrong should cost one more small interaction, not a restart. The second is that cancelling is a first-class outcome. A system built around very small inputs will sometimes misread them, so an easy, obvious way to say no, stop is part of the design rather than an error path.

Who this might be for

People looking for accessible input alternatives, where a keyboard and mouse are difficult, tiring, or unavailable.
People whose hands are occupied — in a workshop, a kitchen, a lab, a vehicle, or simply carrying things.
People in shared or quiet environments where speaking to a computer is intrusive or awkward.
People supervising AI work away from their desk, who need to check in, redirect, and approve rather than author.

On accessibility specifically: I want to be careful. Accessibility here is a design opportunity that demands user research and individual adaptation. It is not a proven clinical benefit, and it is certainly not a universal solution — the range of individual difference in this space is large, and anything that works will work because it was adapted to a person, not because it was shipped to everyone.

Why the approach seems worth trying

These are reasons to investigate, not results.

Communicating a goal can take fewer inputs than performing every step. “Find out why this is failing” is a smaller message than the sequence of clicks and keystrokes that would investigate it manually.
Context reduces what has to be specified. If the system already knows what you are looking at and what you were doing, the part you must supply gets much smaller.
Complementary signals avoid overloading any one channel. Asking eye tracking alone to handle both navigation and confirmation is where a lot of gaze-based interaction historically becomes uncomfortable.
Control comes from feedback, correction, cancellation, and deliberate confirmation. Small inputs are only safe when the person can always see what is about to happen and stop it.
Private or locally controlled AI is a design goal, because this interaction model depends on the system seeing context that is genuinely personal.
A phone is a realistic first display. It is already in your pocket and already good at short glanceable feedback. Wearable displays are interesting later, not a prerequisite.

The honest uncertainties

This is not arbitrary thought reading, and it will not be. Silent, free-form text — composing a paragraph without moving or speaking — remains genuinely hard. Constrained selections and contextual requests are a far more realistic starting point, and that constraint is doing real work in the design rather than being a temporary limitation to be engineered away.

Open questions that need actual testing:

Signal reliability. Small signals are noisy, and noise near a confirmation step is a serious problem.
Accidental activation. How often does the system act when you did not mean it to, and how bad is it when that happens?
Individual variation. Signals differ enormously between people, and probably between days for the same person.
Comfort. Anything worn or held has to be tolerable for hours, not minutes.
Privacy. Contextual AI requires access to context; being silent in a room is not the same as being private in a system.

And two claims I will not make: silence does not guarantee privacy, and conventional input devices are not going to become obsolete. Keyboards will remain the best tool for writing a lot of text for a long time.

What we are actually chasing

The possibility that motivates NeuroDesk is a simple one. If a person can express what they want and stay genuinely in control of the resulting work through a handful of silent, deliberate interactions — chosen on purpose, reviewed before anything happens, and reversible when they are wrong — then a large amount of useful computer work stops requiring a desk, a keyboard, or a voice.

Whether that holds up is exactly what there is to find out.