360° field of vision?

[Greywander]

I realize that there's a lot more pressing things being worked on at the moment, but I just stumbled across this and though it would be hella cool.

https://www.youtube.com/watch?v=f9v_XN7Wxh8

https://github.com/shaunlebron/blinky

As one of the commenters points out, human vision is pretty close to 180°, so a Panini projection giving 170° would give pretty close to normal peripheral vision. Complete 360° vision would also be pretty cool; maybe it could be a magical effect.

[Chris]

Projections like that aren't natural for viewing on a flat monitor. But I could certainly see the technique being useful for having correct per-pixel view projection (vs per-vertex) for curved displays. Possibly also for recording spherical video. Would certainly be more GPU intensive though, considering it has to render 6 times per frame, then apply complex math to warp them for display.

[afritz1]

I've played around with Blinky before. It's pretty neat! I think the Panini projection is a nice departure from the typical method.

I've also experimented with different projections, but I used 3D ray casting instead of rasterization. I was able to get a projection that was like a wide-lens camera, where straight lines were not preserved (that is, vertical walls appeared curved). Here's an example from early experiments in my Arena re-implementation when it still used OpenCL. The higher the field of view, the more everything would bend around the center of the screen like a circle (I think that image uses a vertical FOV of 90 degrees, so it's not that distorted). I imagine the Brigade path tracer uses something like that for its lens effect.

While it might take several renders with rasterization, similar results could be achieved with a compute shader in a single pass. All it takes is the right offsets to initial ray origins and directions. It probably won't be as fast, but I suppose it would be viable if your graphics pipeline is primarily based on ray casting (like a path tracer).

[ezze]

Flat monitors?

I always thought that this 360° field of vision craze was about using virtual reality headset. So you play normally, but if you rotate your head you could see around...

[SquireNed]

It's about extending your mind, maaaaan.

But on a more serious note, I've always done flat-screen gaming and high-FOV where it's possible. When I used to play Minecraft I had the FOV all the way up to the max, which was the most generous game regarding FOV that I think I ever played (Quake had high FOV too, but I never played that in any competitive form).

OpenMW and its predecessor both have awful high-FOV handling, because the projection of the character model is iffy and it really distorts stuff. I wonder if some of these projection methods would improve that (though it could also be improved by reworking the method of drawing the character in high-FOV situations).

[Chris]

Flat monitors?

I always thought that this 360° field of vision craze was about using virtual reality headset.

Yes and no. The actual FOV for VR is fixed to what's appropriate for the headset's physical screen size compared to its distance from the eye (less than 180 degrees per eye). When the headset rotates, it sends a signal back to the system telling it how the head moved. Then for a game, it uses that movement information to alter the point-of-view of the virtual camera, where it renders a new stereographic image with the new coordinates. As long as it's all fast enough, it fools the brain into thinking the view moved because your head moved, giving a better sensation of being in a "live" 3D space. It's not really a 360-degree FOV.

For non-real-time content, such as compressed video, you need a way to encode the full spherical field into a 2D image. It does this by projecting a 3D image onto a 2D plane using a known spherical projection method. A video player can then manipulate (rotate/zoom) the image and "unproject" it back for viewing on the user's display. You can use a VR headset's rotation information to calculate the appropriate rotation, or you can use any device capable of motion detection, like smart phones. Or you can take user input, such as from a mouse or joystick.

[Greywander]

Projections like that aren't natural for viewing on a flat monitor.

But is it an improvement over what we're currently using? That's the question. As long as things toward the center of the screen are relatively undistorted I think it would still work well.

Plus I'd kind of like to see a game with a race or spell effect that gives 360° vision, even if it is distorted. Hey, if we can do it, then why not implement it as an option? You know, once the more important stuff has been taken care of.

(Also, the Vive has an FOV of about 110°. Good, but it doesn't capture the entirety of human peripheral vision. I've heard some YouTubers say the current VR headsets are sort of like looking through a periscope or something. Sounds like we can actually do better than them with just a flat screen monitor in terms of peripheral vision.)

[Chris]

But is it an improvement over what we're currently using?

Depends what you consider an "improvement". It's a less physically correct view, but it allows seeing more with less severe distortion giving you a more tactical advantage. So if your measure for an improvement is a more correct view, then it's worse. Or if you consider a more advantageous view an improvement regardless of how incorrect it is, then it's better.

(Also, the Vive has an FOV of about 110°. Good, but it doesn't capture the entirety of human peripheral vision. I've heard some YouTubers say the current VR headsets are sort of like looking through a periscope or something. Sounds like we can actually do better than them with just a flat screen monitor in terms of peripheral vision.)

Likely a limitation of the screen size and resolution. The screen is pretty small since it has to be lightweight and fit into the headset (while also being cost-effective), and it needs to be split into two, with each eye receiving one half of the available pixels. Each half needs to fill as much of its eye's peripheral vision as it can without becoming too pixellated or blurry. You'd have to be really close to a flat screen monitor to get a >90 degree FOV, and you'd lose out on the stereoscopic vision (the two eyes overlap in what they can see, while having a slightly different POV; it's a subtle but extremely important effect for depth perception), and you don't have head tracking (another important feature for a perception of all-encompassing space, your view correctly shifting from both subtle and not-so-subtle head movements).