About blendFX
blendFX is a small studio for 3D, VR, AR and VFX based in Leipzig/Germany.
We produce virtual and augmented reality applications, architectural visualizations, 3d reconstructions and animations & visual effects for TV and cinema, with a focus on high quality content for mobile VR.
Rendering of Renaissance Period
How we came to VR
We started working with virtual reality in 2014. During our first experiments it became clear quite quickly that we wanted to concentrate on mobile VR, simply because it’s more accessible and affordable. For the time being, however, smartphones are simply not capable of rendering high quality content with reflections, transparencies and nice antialiased edges. That’s why we began focusing on pre-rendered content, using stereoscopic panoramas. We developed a workflow with Blender and Unity, where we integrate interactive elements and 3d animations into stereo panoramas. The resulting apps perform very well on recent smartphones like the Samsung S6 and S7.
Rochlitz VR
“Rochlitz VR” is a virtual reality app which we created for Schlösserland Sachsen. It’s a virtual reconstruction of one of the chambers from the old castle “Schloss Rochlitz” in Saxony, Germany. Today you can see the remains of 5 different periods from the 800 years long history of the “captain’s chamber” (“Hauptmannsstube”) in that room. There are doors and traces of paint from the gothic period, the backwall of a fireplace and romanesque windows, a painted ceiling from the renaissance and on top of that plumbing, drillholes and wallpaper from the GDR (German Democratic Republic) period in Eastern Germany. It’s a really weird and crappy looking room.
The room as it looks like today
Usually rooms like this are being restored to one particular time period. But then of course you lose the chance to see what the room would have looked like in other periods. In this case the castle museum wanted to make it clear to the visitors that the castle has been in constant change throughout the ages. Traces of all these periods can be found today. So the museum needed a way to make this room accessible and understandable for visitors without destroying its current peculiar state with all the various items from different time periods. Virtual Reality turned out to be the perfect medium for this!
Together with conservators and museologists we reconstructed the captain’s chamber how it might have looked like in those 5 different periods. The Rochlitz Virtual Reality Experience will be shown on 4 Samsung Gear VR devices inside the captain’s chamber starting in spring 2017.
How
We used Blender for the entire process of modeling, texturing and rendering.
The captain’s chamber changed a lot throughout the 800 years, and so we had to heavily adjust the model for each time period.
The 3d modeling turned out to be a crucial part of the scientific reconstruction. The conservators and museologists would provide us with data, measurements and theories of how the room should have looked like back in the days, but it was only during the actual modeling process where we found that some of these theories would have been impractical or even impossible in real life. So the 3D model of the room was also a tool to test a scientific hypothesis.
Screenshot of the blendfile of the gothic period
For the gothic period Nora Pietrowski, the museum’s conservator, gave us the flowery wall paintings, which we mapped onto the four sides of the room. The painted ceiling of the renaissance period was also created by Nora. Then with Blender’s 3d painting tools we added subtle dirt and weather effects to the walls and the floor.
One of the most interesting periods was the one of the second half of the 20th century. The museum wanted to stay as scientifically accurate as possible with the reconstruction. That’s why most of the rooms we reconstructed have no furniture, because there is no detailed information of what kind of furniture actually was in the room. However, there are two photos from the GDR period that do show some the furnishing. That was a good reference for us to rebuild the room with a bit more detail.
Rendering of the GDR Period. In the back you can see the reference photo on the wall.
Rendering
For rendering we used Blender’s path tracing engine Cycles and its stereoscopic panorama rendering mode, which Dalai Felinto had added in 2015. However, we found that equirectangular rendering is wasting a lot of processing power for the poles of the spherical images, which made rendering a lot slower than it needed to be. So we hired Dalai to code a script to render cube maps, which not only render much faster but also even look better on the poles than the usual equirectangular panoramas. Rendering a cube map panorama creates 6 different images, one for each side of the cube. Each side has a resolution of 1280×1280 pixels. And because we are rendering stereo we end up with 12 images, which add up to a resolution of 15360×1280 pixels for each stereo panorama. Luckily Cycles can take advantage of GPU rendering, which made the renders a bit less painful.
Cubemap Stripe
Another thing we needed was pole merging: It’s a common problem of stereoscopic panoramas that top and bottom (zenith and nadir) look a bit weird because of left and right eye perspective overlap. One way to fix that is simply to let the parallax converge to zero right on the poles of the panorama, also known as pole merging. And since we are not Blender coders ourselves but know some very skilled programmers at the Blender Institute we simply drove to Amsterdam in spring 2016 and talked to Sergey Sharybin, one of the amazing core developers. Within hours we had a working prototype and not much later pole merging was implemented to Blender. Amazing!
For testing the VR renderings internally we used VRAIS, a VR viewer which we developed ourselves together with our colleagues from Mikavaa. To get feedback from conservators and museologists on the test renderings and panoramas we used ReViz, a VR viewer with comment and revisioning system, developed by Mikavaa.
The high resolution stereoscopic panoramas where then taken to the game engine Unity, where we put interactive elements in to the scenes in order to create an intuitive and immersive VR experience, which the visitors of the castle will be able to enjoy with a GearVR headset. The user can travel to the different time periods simply by focusing the interactive overlays on the walls or by using the time line.
Screenshot of some UI elements in the app
The Installation
Four turning chairs have been customized for this exhibition. The 4 headsets need to be continuously attached to a power supply in order to prevent discharging during constant usage. For that the museum’s constructor designed a custom solution with wiper contacts so that the USB cable that charges the GearVR headset can be attached to the turning chair without any cable clutter.
The goal is to be able to have four visitors experience RochlitzVR simultaneously without the need for museum attendants. So the GearVR itself needs to be protected against theft. Therefore a USB-C cable mount was designed that can be screwed onto the GearVR body. The cable itself is quite strong by itself. The museum figured that anyone who would bring the tools to cut those cables would also be able to bring gear to cut stronger metal cables, so the USB cable as theft protection should do it. Further, the phone itself is hidden behind the plastic GearVR cover, which is held in place by 4 screws.
The installation in the captain’s chamber
VR is still a relatively young medium (even though last year’s hype has already faded a bit). But especially for cases like this we think that it can be a brilliant addition for museums, exhibitions and education. Of course it would be even more exiting to be able to walk around in VR, like you can for example with the HTC Vive, but for exhibitions that’s not really practical: it is way more heavy, more expensive and you definitely need someone to operate it and help the users, which makes it even more expensive. It just adds too much maintenance costs for a small museum.
This project took over a year from start to finish, with some breaks in between. We think it’s quite remarkable that an old castle is open and forward thinking enough to embrace a fresh technology like VR, where there is still a lot of development and change happening all the time. Therefore it was an absolute pleasure to work with the castle’s museologist Frank Schmidt, who also initiated the whole thing.
Being able to use Blender for this project was a great experience. Over the course of the last year several features have been implemented that helped us a lot in our workflow. Cycles was improved a lot and is now even faster than when we started. So I think it’s safe to say that Blender really is a perfect tool for producing high quality VR content, and we can’t wait to see what the future brings!