Can Oscillators Alone Produce Depth?

This June, I was invited to Tepoztlán for the QRI research retreat taking place there. I had limitations in my schedule so I was only able to attend the last week, but I’m glad I did. It was a great week of conversations, vibecoding explorative demos, and marinating in an incredible environment of like-minded people focused on the frontier of consciousness research. While there, a number of conversations I had with Andrés Gómez Emilsson were about depth perception, how it works, and what the involvement of coupled oscillators may be.

Most of the time when navigating the world, our depth perception is greatly informed by stereoscopic vision. However, stereoscopic vision only provides precision close up. Our ability to sense depth at a distance is essentially an ability to determine depth from 2D input alone. This sort of depth perception is at work in the photo above; even though the photo is printed on flat paper, you are able to understand an approximate model of the depth within.

The model is certainly approximate as it is easy enough to show limitations compared to full stereo depth. Hang a ring in front of yourself and attempt to put a pencil through it while covering one eye with a hand — it’s difficult. Uncover your eye and try again, and there is no challenge whatsoever. The target for something extracting depth from 2D alone isn’t a perfect model at all depths but something “close enough” to understand the scene in general. When you look out at mountains, the brain’s 2D-to-depth conversion is allowing you to understand them as distant mountains, not stereo vision.

Kuramoto systems are implemented by having each “cell” of the system oscillate and modify its oscillations according to its neighbors. These systems can easily emerge a variety of patterns and behavior like synchrony, and traveling waves which are also seen in exotic experiences. Andrés and I talked several times about the possibility of Kuramoto oscillators extracting depth or otherwise play some role in processing visual input into a world model. Andrés tasked me with exploring if any configuration of Kuramoto oscillators and depth from a scene as input could replicate unique patterns. I made a small demo with a raymarcher of a simple scene (a plane with a ball) and an oscillator and a few different modes. One mode takes the scene’s depth as input, with a wide variety of ways it can influence the field: amplitude, frequency, how strongly pixels couple with their neighbors, or a sine wave whose frequency the depth modulates. This depth-as-input mode fit Andrés’s task for a replication explorer. Another mode has the same freedom of input exploration, but the input is a render of the scene with basic lighting. This mode was more for exploring the other way around, seeing if any setup appeared to go even slightly in the direction of modeling depth. One configuration was able to produce a distinct pattern for each layer of the scene - the ground plane, the ball, the background. That configuration involved sine waves as basic inputs which I believe helped the system, more on that below. I think in that case there was enough difference in shading between the elements to act as boundaries, but while somewhat trivial this is still a good result. It shows that at least basic features can be extracted this way and hints toward semantic segmentation as well.

Andrés Silva, another attendee, brought a different kind of generator: stacked sine waves of varying frequency and orientation to produce replications, which he showed in his skillshare with a tool he built for it. Much of our exploration was focused on emergent waves, but Silva’s approach suggested sine waves as standing input. Andrés made a demo based on this where the frequency, amplitude, orientation, and translation speed of two sine waves could be modified, producing a wide and interesting state space of behavior. Not that it is radically different from existing Kuramoto behavior, but the sine waves provide regularity, so patterns happen on a cycle instead of by chance. It was this approach that I modified to try extracting depth — I figured the sine waves might resonate with gradients formed by lighting, which even if it didn’t give depth would be a start at extracting information from the scene. The demos made for this and other similar experimentation were all kept in a folder called PDP — Parametrized Dynamic Phenomenology, each demo featuring an array of sliders to tune. All to allow easily getting the “vibe” of a system, as well as ability to find exact parameters that make a system produce a pattern that near exactly resembles some exotic experiential element.

Exploring what patterns would result from depth as input was interesting as it automatically can replicate some general psychedelic qualities. For one, you get effective texturing of surfaces, texturing with rainbows oscillating! Depending on configuration, the oscillators may form into lattice lines or even Truchet-like patterning, mimicking Shipibo patterns. Even more, 5-MeO-DMT states for many people often include visuals that appear directly to be Kuramoto oscillators. So there is a reasonable case for these oscillators being a significant part of generating visual experience.

A common report in experiences either psychedelic or meditative as well as common in psychedelic art, is patterns that resemble Hilbert curves. The Hilbert curve is a space-filling curve which enables encoding two-dimensional space as one-dimensional, with a key feature that this encoding approximately preserves spatial relationships. Things close in 2D space stay close (mostly) on the 1D Hilbert curve line. Since this provides a way of “hopping dimensions”, it makes sense that the brain may use it in visual processing. Functions which do that, jump from one dimension to another, seem uniquely interesting. At least they are to me, since if we find these functions useful and efficient to process data digitally, it makes sense the same functions show up in natural systems like the human brain.

If there is a way this works, it may be a specific implementation with exact parameters — so for now the sensible goal is to find an implementation that hypothetically might work. An implementation that either shows unique replication of experience (patterns which are not accurately replicated by simpler systems) or manages to produce even very low-precision depth information from an input, would be valuable supporting evidence.

While exploring this idea of depth from oscillators, Andrés made several vibecode demos and testing grounds for various hypotheses. On one night we spent several hours exploring ideas about Hilbert curves in this process, as well as seeing if systems combining them with coupled oscillators may produce accurate replications. Andrés was curious to test this and encouraged me to see what may result by setting up a shader with a standard Kuramoto oscillator as well as one in which the neighbors are based on the curve instead of the 2D Euclidean distance. Our explorations on that night attempting to make a demo didn’t show any strong confirmation of this idea, but that wasn’t really expected anyway. The goal was more to see if this even seems coherent, as well as looking out for unique visuals that could emerge.

What if this doesn’t make sense, though? It is possible that this whole approach just falls flat on its face if the actual depth computation in the brain is far more black-box. By black-box, I mean that the way it works is much more like a compressed lookup-table which is filled by training data, more commonly referred to as life experience. Right now, there are neural network depth models you can submit an image to and get a depth map back, but how these systems work internally is not clear. There may be some interpretable system within, but it also may be very much like the inverse square root line in Quake.

If the bulk of work being done is irreducible computation like this, then discovering exactly what is going on with the brain and neural processing will likely be much more difficult. That said, it doesn’t seem likely that is the case, if only because the wide yet specific variety of geometric patterning seen in exotic conscious experience supports the idea that there are simple generators in the brain which compose into complex behavior. If the brain were more black-box then I’d expect psychedelics to behave far closer to their popular culture representation of purely chaotic randomized content.

If there is a way to extract depth using Kuramoto oscillators alone, I think it can be experimentally shown. The exact right setup is not obvious to reason about, but I believe there is one. For now, I had Claude run an automated researcher on a single-layer setup and it found nothing promising, but this was to be expected from only one layer. That setup was more of a proof of concept which I can now think about how to extend, and what sort of multi-layer setup makes sense.