How to live stream 360 video

“Can you live stream Jizz?” I never thought I’d ever be asked this question, but a couple of weeks ago the phone rang, and it happened. After hanging up, I nervously typed Jizz, Brighton into Firefox, and breathed a sigh of relief (no pun intended) to confirm that it is actually an innocent (ish) evening of poetry, music and dance.

Jizz is the brainchild of acclaimed English poet John Davies, and based around his book of the same name. Having made its debut at a theatre near Glastonbury, the second performance was to be held at the historic St George’s Church in Kemptown.

At Tilt, we love to push the boundaries of creative technology. Our answer to John’s peculiar question was a resounding “yes… and how about we do the live stream in 360”. I had first been keen to try a 360 live stream after seeing the Expedia Dream Adventures Film from 2016, where severely unwell children are transported through live VR to their dream world locations. I’m not quite sure how I feel about the ethics of using sick kids to indirectly sell holidays, but the idea of transporting people to anywhere in the world in real-time is truly inspirational. The potential of bringing in live virtual audiences from all over the planet is clear, and recently bands like Coldplay have started to get in on the action.

The camera

So our first step was to decide what to shoot it on. We’ve done a lot of 360 filming involving arrays of GoPros, with footage stitched laboriously in post, but never anything that is stitched real-time and pumped out live over social media. It turns out that it is technically possible to live stream with our existing Freedom 360 GoPro setup , however experience has taught us to be cautious. Having filmed events in 360 for superclubs like Fabric and Ministry of Sound, we’d learned the hard way that GoPros aren’t particularly good in low light, and are prone to overheating.

In the best cases, overheating has produced excessive noise in the image that can thankfully be cleaned up in post with plugins like Neat Video. In the worst cases however, overheating has locked up one of the cameras, and one bad camera in the array will ruin the entire take.

So we decided to see what else was out there on the market that we could use for the job. The first decision was whether to hire or to buy. The usual suspects for production hire – like VMI and Promotion Hire – were surprisingly lacking in 360 camera tech. There are a few specialist hire companies out there, and two cameras seemed to be leading the way in the pro market:

Firstly there’s the mid-range Insta 360 Pro which retails at around £3,800. Insta 360 create a selection of cheaper consumer 360 products, but the 360 Pro is a, 6 lens, 8k beast that can shoot RAW 3d footage, and be monitored with a headset in real-time as you live stream up to 4k. The image quality can’t be denied, but there have been some negative reports regarding slow startup, and more importantly for us, noisy cooling fans – kind of a deal breaker for live streaming.

Next on the wish list was the Nokia Ozo. This is an incredible 8 lens, 8 microphone, stereoscopic 360 camera that seems to be setting the bar for pro 360 film at the moment. But with a retail price of around £60,000 and hire costs upwards of £1500 per day, this seemed like overkill for a low budget production.

We drew a line under hiring a camera and looked at what we could buy for under £500 that produced a decent image, and could stream in 4K. After some research, we drilled down to the Samsung Gear 360 2017 , and the newly released Ricoh Theta V – an upgrade from the Theta S which can only stream HD footage. Our local Jessops had a Samsung in stock, and Park Cameras had one of the first Theta Vs to hit the stores, so we went to test out both.

The Samsung seemed to have better stabilization, worked well with our existing Samsung Gear VR setup (phone and headset), and was very reasonably priced at £220. The more expensive Theta V on the other hand had superior Pentax optics giving a better image quality and showed much less digital noise in low light situations – and this was the clincher for us. For the money, the Theta V is a very impressive 360 camera, and we departed Park Cameras £400 lighter.

YouTube, Facebook or Vimeo

Now we had the camera, we had to decide on a host. YouTube were first to offer 360 live streaming last year, and Facebook followed in July 2017. Then there’s Vimeo Live which was launched this month, but unfortunately live streaming of 360 video is not yet available. When I pushed Vimeo on this, they responded with the following:

“…360° live streaming is definitely on our radar, but we’ve got a few more live features we’re focused on rolling out in the coming months…”

With YouTube being the first to integrate 360 live streaming we decided that this would be the safest bet for hosting our stream.

Encoders

Next was the streaming software. We have previously streamed live (non-360) performances for Citroen, where a Citroen C1, rigged with cameras and microphones is used as a popup performance ‘stage’ in city centres. For this we used Wirecast and it seemed to do the job very well. Looking at the blogs however, everything was pointing towards using the free OBS app (Open Broadcaster Software) for live 360, and with a lot of info out there about OBS and VR, we opted for this instead. Check out a summary of the event here.

Network

 For small budget live streaming of events, we tend to rely on two 4G beacons from different network providers – in this case Three and EE. We bring two beacons along so that we can find the strongest network in our specific location, and have a backup provider should the signal drop out for any reason. At indoor locations that already have good wifi, this can be the most reliable option, but we still bring along the beacons topped up with data, just in case.

Teething problems

 As you can see, this is quite a long section! This was certainly a project that required a lot of patience to get off the ground. Theoretically, you should just plug everything in, hook OBS into YouTube with a streaming key, connect to wifi, and off you go – 4K 360 broadcast to the world. In practice however this was not the case.

Relying on a camera that has only just hit the shops is never a great idea, particularly with fledgling technology. The Theta V worked fantastically in normal circumstances with a really impressive image and sound – it’s got 4 good built in mics for spacial sound, and you can pimp it with the bolt on TA-1 3d microphone. It connected well with our Macs to transfer footage, and performed perfectly with the smartphone app.

With regard to live streaming however, it seems the product had perhaps been brought to market too soon. The first hitch was a plugin issue. The Theta S is provided with the Live-Streaming app (which includes a plugin), however this doesn’t support the Theta V, and there’s no provision on the site for the new camera. Consulting the online manual, it vaguely states “…Start an app that can play back live camera images. (“Media Player Classic – Home Cinema” (MPC-HC) is used as an example in the following descriptions.)…”. This was not very helpful to an Apple user.

Plugged into our iMacs and Macbook Pros and set into Live Stream mode, we could receive audio from the Theta V in OBS, Wirecast and Google Hangouts, but nothing for video – just a blank screen. The Netherlands-based telephone support tried their best to be helpful, but did not seem to offer any solutions other than to try it on a Windows10 PC. So we tested it as specified in MPC-HC (and in OBS and Hangouts) on a cheap PC laptop to no avail, then on the beast that powers our Occulus Rift setup – again, the screen was black.

A ticket from the Ricoh Europe support team mentioned trying to update the firmware. Being that this was the first week that the camera had been available, we hadn’t expected a system update, but low and behold there was one. With Version 1.00.2 freshly installed, we eagerly booted up the Theta expecting all our sorrows to be over…will it work?…will it? …will it?…No. Black screen.

With the date of the Jizz performance looming we had to make a decision. Do we accept that the Theta V is not fit for purpose, take it back to the shop and settle for the inferior image quality of the Samsung Gear 360, or do we hold fire. The support team assured us that we should do the latter, and that continuous updates were in production – just keep checking. This didn’t instill me with too much hope.

Being that this was very early days for the Theta V, there was very little chatter online around this. It was clear that the few people who had received their camera and were trying to live stream were experiencing problems, but no solutions seemed to present themselves. Then out of the blue, whilst looking at the ‘Unofficial guides for RICOH THETA’ forum we found a thread where one member detailed how he had found a solution by contacting the Japanese Customer Support and acquired a BETA Theta V UVC 4K driver for Windows10. We got in touch with him, and he kindly forwarded the driver to us within an hour.

This time it did work – well to a degree anyway. After installing the driver, the live 360 image was finally there for us to show to the world. Unfortunately, we have yet to get the on-board sound working when doing a live stream, and so for the Jizz event, we ran in a non-3d audio line from the mixing desk. The upside of this was that we were streaming the best quality audio – albeit non-spacial. On the downside, it meant that there was some latency, throwing the video and audio stream very slightly out of sync.

We were now free to hook OBS into YouTube which was a straightforward process that is widely documented. We took everything to test on-location prior to the performance of Jizz, and the final hurdle was trying to secure an internet connection capable of streaming 4K 360 film.

In short, this was disappointingly not possible with our internet setup. For 4K steaming at 30fps you need between 13,000 and 34,000 Kbps upload speed – ideally being closer to 34,000 Kbps if you want to keep it steady. Our 4G beacons tried their best, but delivered very little. They were only achieving a miserable 2000 – 2500 Kbs upload speed which couldn’t even maintain the 3000 – 6000 Kbs required for HD. Thankfully the location wifi was giving us 5000 Kbs which could not achieve 4K, but was fine for HD, and the client was happy to go ahead with this.

You can see one of our location test recordings with the Theta V here

The Event

With every crease seemingly ironed flat, we rigged up at the event full of hope and excitement. We were also tasked with filming Jizz in a traditional sense. We did this with our Canon C300 mkII shooting wide on sticks, a dolly mounted Panasonic GH5 (with the incredible v2.0 firmware update) shooting close up stage right, and a fluid head mounted GH4 recording 4:2:2 ProRes to an Atomos recorder to stage left.

The Ricoh Theta V was positioned centrally (but discreetly because of the small form factor) between stage and audience in this beautiful symmetrical environment. This positioning would capture the majority of the show from the front, but allow for the dancers to pass in front of and circle the camera – which looked fantastic in a VR headset. A steady preview stream was maintained throughout dress rehearsal during the day, and as night fell, and the physical / virtual audience arrived / tuned in, we were all set to go live with renewed confidence.

But then the unthinkable happened. With two minutes to go, just as the presenter was about to step out on stage, for the first time all day we received the dreaded message “BAD stream quality”. Whether this was the physical audience using the same wifi signal to tune in to the live stream, or just bad luck, we’ll never know. All we could do though was sit tight with everything crossed, hoping that good luck would once more prevail – and to our relief it did. We watched the status turn from Bad, to Poor, to Good, just in time for the start of the show, after which a good signal was maintained throughout. The crew sighed a collective sigh of relief.

There was a great turn out for the event, and the numbers were doubled by virtual attendees from across the globe, with the stream living on in Youtube for audiences in the future. Although this project was quite a challenge, we had a fantastic time doing it, and learned a great deal from the thrill ride. The client was really pleased with how things went – here’s what they had to say:

“Tilt were recommended to me, and when I approached them about covering an unusual theatrical event including poetry, music and dance, the team responded enthusiastically and creatively, suggesting the use of 360 VR for the live stream as well as recording the event in 4K with a three camera set up.

Their commitment and technical capability is superb. They worked tirelessly to resolve any issues with the live stream and provided excellent high quality coverage of the whole event.

With over forty years experience in film and video I can honestly say this is one of the most professional outfits I’ve ever worked with.”

You can view the entire HD live stream here.