The end goal is to produce a stable, usable desktop environment compatible with the Oculus Rift that makes you feel like you're living in a mid-90s virtual reality movie.
Due to the low level nature of the project, I cannot use any off the shelf 3D engines.
Here's a backlog of in-progress videos
-
4/23/2013 - After working on it for two weeks, I finally managed to grab textures out of the Windows;
DWM.
The original idea is inspired by
dwmaxx2;
however I could never get it working on my machines and ended up going with a different technique.
This version is very manual - I'm manually peeking into window handles, process ids, and texture sizes, but it's the first working proof of concept.
Performance is very good however; FRAPS was showing 6000+ fps on the Chrome cube.
-
4/25/2013 - Automatic texture grabbing from the DWM based off process. In this version I only have to type a process ID to get the application texture, instead of the previous manual process.
-
4/28/2013 - Detecting and displaying all active processes. Child windows are not detected properly, nor is there any way to actually switch focus. Performance is still good; 500+ fps still.
-
4/29/2013 - Added detection of currently focused window. Active window will be zoomed in much closer so things are readable.
-
4/30/2013 - Added stereo rendering and head tracking support for Oculus Rift. No barrel distortion shader yet.
-
5/5/2013 - Added crappy mouse rendering/tracking. A 3d mouse cursor appears that allows you to interact with the focused window. Lots of mouse state is not properly tracked in this version.
-
5/11/2013 - Added barrel distortion shader for Oculus Rift.
-
5/15/2013 - Improved child window detection and mouse tracking. Windows treats everything as a 'window' - tool tips, context menus, etc. This causes problems when trying to get a tool tip to be associated with the right window, because the parent-child relationships are often unclear.
-
5/19/2013 - Basic picking code. You can now switch tasks inside the application by looking at the window and hitting a hot key.
-
5/21/2013 - Added cursor state tracking. Current mouse cursor icon is detected as well as its visibility. There are no other cursor models yet, but it should be possible to render the right cursor now.
- 5/23/2013 - Begin adding Kinect support.
-
5/26/2013 - Basic Kinect tracking and skeleton rendering. This the Windows version of Kinect, which includes a 'near mode', but even that isn't good enough to be usable at desktop distances. I'm going to get a Leapmotion controller at some point and use that instead. I have a Razer Hydra, but I decided it would be annoying to switch between keyboard and mouse and the Hydra constantly
-
5/30/2013 - Add a simple directional lighting model.
-
6/1/2013 - Add a point lighting model.
-
6/3/2013 - Begin curving windows. Because of the resolution of the Rift, windows must be rendered quite large to be legible. This causes a problem where the edges of the window aren't readable due to the extreme angle they're viewed at. The solution is to curve the windows such that every point is equidistant from the viewer.
-
6/4/2013 - Better curved windows. Improved window geometry, fixed surface normals. Background windows are also laid out in a sphere now. 2D window coordinates are now remapped to the curvature of the window, so the mouse cursor stays floating above the window at the right height.
-
6/5/2013 - Window picking was broken by the curved layout. Partially fixed again.
-
6/6/2013 - Added a basic HUD with some text rendering.
-
6/15/2013 - Added the preliminary ability to load Alias Wavefront OBJ files. No material support yet
Could you tell me more about how you are getting the texture data?
ReplyDeleteHow are you grabbing the texture? I'm thinking of making something similar and would appreciate the advice.
ReplyDelete