The Product Formerly Known as Surface

I haven't been updating VROffice recently because I've been working on building a coffee table.

It's a touch screen coffee table, like the Product Formerly Known as Microsoft Surface (PixelSense).




I have very little experience working with wood, so I picked the simplest design I could think of. The table is made from a series of 1x8 boards, a piece of 3/4" thick plywood, a bunch of 4" right angle brackets, and some 1/4" carriage bolts.


The TV is an old 40" HDTV I had lying around not being used. It's a few years old, so by modern standards it's very thick and heavy (a little over 5" deep, close to 50 pounds in weight).

The touch screen overlay is from Peau Productions. All of the touch detection is done through the frame, so the glass can be used as a plain table surface without worry.


This is all hooked up to an old desktop PC running Windows 8.

It will be mainly used to control an HTPC hooked up to a projector.

I just finished the build and haven't had a lot of time to play with it, but a few things I'm looking forward to trying out 

• Two player air hockey
• Virtual Pinball
• Running an XBox 360 on the projector while using the table for SmartGlass
• Battlescreen for Battlefield 4
• Civ V
• Kerbal Space Program
• OneNote using a stylus

I might end up writing some media player software for this particular set up too.

Build

1. Some of the starting materials
   
2. Cutting the plywood
   
3. Cutting the boards
   

4. Testing the assembly of a corner using the brackets and carriage bolts on scraps
   
5. Joining a leg to a short side
   
6. First corner joined!
   
7. Making the rest of the legs
   
8. Test assembly of the frame
   
9. Adding in the support slats
   
10. Testing if the overlay still fits
    
11. Disassembled and prepped for staining
    
12. Staining, two coats
    
13. Test reassembly after staining
    
14. The touch screen overlay is 46"; the TV is 40". To cover the gap, I used some craft paper cut to size
    
15. Final assembly
    

Fixed the Sky Box

I'm rendering a proper sky box now. Previously I had a series of individual bitmaps and it wasn't sampling properly, giving rise to seams.
I also increased the size of the ground so it forms a proper horizon now. The shader I accidentally broke was also fixed.

Since there's a ground now, the background apps can be placed underneath now. I'll have to fix that later.

Adding a SkyBox

I implemented a primitive skybox. There's quite a few problems with the current implementation. In particular, seams are visible and the texture sampling isn't spherical. I also broke the shaders for the mouse cursors.

The skybox was generated using SpaceScape The ground is a model file. The model reader still needs to be improved; there are some lighting issues with the models as well.

Adding Cursor Models

Now that a basic model loader has been added, I can finally wire up the cursor state management code to show the correct model for the current mouse state. I made a bunch of untextured placeholder cursors, but it's just a matter of replacing them with better looking ones in the future.

There's a bit of an issue with the cursor clipping through the curvature of the window that needs to be fixed - either I need to warp the geometry of the entire cursor to conform or rotate it such that the x-y plane of the model is tangential to the curvature of the window. The latter will likely look better but won't completely fix the clipping issue.

There's at least one cursor type that seems to be undocumented - the reversed arrow that appears in Visual Studio 2012 when you move your mouse over to the column where the breakpoint dots are placed on code. That shouldn't be hard to figure out though.
The video shows proper handling of the loading, resizing, move, i-beam, and hand icons.

Adding MTL support to the OBJ reader

I added MTL support to the OBJ reader. However, it only loads diffuse texture maps right now. Currently every object in the app is generated programmatically. Adding MTL and OBJ parsing will allow the use of real modellers. The first thing that this allows is support for rendering mouse cursors in 3D properly.

In this video, Thane has two separate diffuse textures - the head and the body are two groups, each using a different material described in an MTL file.

Catching up on the backlog

VROffice is a 3D Desktop Environment for Windows 8.
The end goal is to produce a stable, usable desktop environment compatible with the Oculus Rift that makes you feel like you're living in a mid-90s virtual reality movie.
Due to the low level nature of the project, I cannot use any off the shelf 3D engines.
Here's a backlog of in-progress videos

• 4/23/2013 - After working on it for two weeks, I finally managed to grab textures out of the Windows; DWM. The original idea is inspired by  dwmaxx2; however I could never get it working on my machines and ended up going with a different technique.
  
  This version is very manual - I'm manually peeking into window handles, process ids, and texture sizes, but it's the first working proof of concept.
  
  Performance is very good however; FRAPS was showing 6000+ fps on the Chrome cube.
  
  
  
  
  
• 4/25/2013 - Automatic texture grabbing from the DWM based off process. In this version I only have to type a process ID to get the application texture, instead of the previous manual process.
  
  
• 4/28/2013 - Detecting and displaying all active processes. Child windows are not detected properly, nor is there any way to actually switch focus. Performance is still good; 500+ fps still.
  
  
• 4/29/2013 - Added detection of currently focused window. Active window will be zoomed in much closer so things are readable.
  
  
  
  
• 4/30/2013 - Added stereo rendering and head tracking support for Oculus Rift. No barrel distortion shader yet.
  
  
  
  
• 5/5/2013 - Added crappy mouse rendering/tracking. A 3d mouse cursor appears that allows you to interact with the focused window. Lots of mouse state is not properly tracked in this version.
  
  
  
  
• 5/11/2013 - Added barrel distortion shader for Oculus Rift.
  
  
• 5/15/2013 - Improved child window detection and mouse tracking. Windows treats everything as a 'window' - tool tips, context menus, etc. This causes problems when trying to get a tool tip to be associated with the right window, because the parent-child relationships are often unclear.
  
  
• 5/19/2013 - Basic picking code. You can now switch tasks inside the application by looking at the window and hitting a hot key.
  
  
• 5/21/2013 - Added cursor state tracking. Current mouse cursor icon is detected as well as its visibility. There are no other cursor models yet, but it should be possible to render the right cursor now.
  
  
• 5/23/2013 - Begin adding Kinect support.
• 5/26/2013 - Basic Kinect tracking and skeleton rendering. This the Windows version of Kinect, which includes a 'near mode', but even that isn't good enough to be usable at desktop distances. I'm going to get a Leapmotion controller at some point and use that instead. I have a Razer Hydra, but I decided it would be annoying to switch between keyboard and mouse and the Hydra constantly
  
  
• 5/30/2013 - Add a simple directional lighting model.
  
  
• 6/1/2013 - Add a point lighting model.
  
  
• 6/3/2013 - Begin curving windows. Because of the resolution of the Rift, windows must be rendered quite large to be legible. This causes a problem where the edges of the window aren't readable due to the extreme angle they're viewed at. The solution is to curve the windows such that every point is equidistant from the viewer.
  
  
• 6/4/2013 - Better curved windows. Improved window geometry, fixed surface normals. Background windows are also laid out in a sphere now. 2D window coordinates are now remapped to the curvature of the window, so the mouse cursor stays floating above the window at the right height.
  
  
• 6/5/2013 - Window picking was broken by the curved layout. Partially fixed again.
  
  
• 6/6/2013 - Added a basic HUD with some text rendering.
  
  
• 6/15/2013 - Added the preliminary ability to load Alias Wavefront OBJ files. No material support yet