Skip to content

Commit

Permalink
Adding vfx renders and arxiv paper
Browse files Browse the repository at this point in the history
Adding blender vfx renders and link to arxiv submission
  • Loading branch information
cvachha committed Feb 10, 2024
1 parent e662bcf commit e1a668e
Show file tree
Hide file tree
Showing 11 changed files with 51 additions and 5 deletions.
Binary file added assets/pdfs/Cyrus Vachha-Resume Spring 2024.pdf
Binary file not shown.
Binary file added assets/videos/ns_vfx_breakdown.mp4
Binary file not shown.
Binary file added assets/videos/ns_vfx_elevator_composites.mp4
Binary file not shown.
Binary file added assets/videos/ns_vfx_lego.mp4
Binary file not shown.
Binary file added assets/videos/ns_vfx_portal_composites.mp4
Binary file not shown.
Binary file added assets/videos/ns_vfx_trailer_short.mp4
Binary file not shown.
9 changes: 6 additions & 3 deletions nerfstudio_contributions.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ Since Jan 2023 I have made contributions to the Nerfstudio API system including

## Nerfstudio Blender VFX Add-on

<ul class="actions">
<li><a href="nerfstudio_vfx_reel.html" class="button">View NeRF VFX Renders</a></li>
</ul>

<iframe width="560" height="315" src="https://www.youtube.com/embed/A7La8tWp_0I?si=uChvOIFJ7WniBMTY" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>

I created a Blender add-on that allows NeRFs to be used in visual effects. This enables a pipeline for integrating NeRFs into traditional compositing VFX pipelines using Nerfstudio. This approach leverages Blender, a widely used open-source 3D creation software, to align camera paths and composite NeRF renders with meshes and other NeRFs, allowing for seamless integration of NeRFs into traditional VFX pipelines. It allows for more controlled camera trajectories of photorealistic scenes, compositing meshes and other environmental effects with NeRFs, and compositing multiple NeRFs in a single scene. This approach of generating NeRF aligned camera paths can be adapted to other 3D tool sets and workflows, enabling a more seamless integration of NeRFs into visual effects and film production. This also supports Nerfstudio gaussian splatting as well.
Expand All @@ -24,16 +28,15 @@ The exported mesh or point cloud representation is imported into Blender and a r
I created documentation for it [here](https://docs.nerf.studio/extensions/blender_addon.html) and a tutorial video demonstrating basic examples using the add-on as well as a breakdown of other effects that can be done with it.

<iframe width="560" height="315" src="https://www.youtube.com/embed/vDhj6j7kfWM?si=zmlFcZoxZipyTEqs" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
<br>
I have also written a technical write-up on Arxiv [here](https://arxiv.org/abs/2401.08633).

<!--
I also have a blog post style walkthrough of making it.
<ul class="actions">
<li><a href="nerfstudio_vfx_blender.html" class="button small">Read More</a></li>
</ul>
-->

<br>

## 🥽 VR Video Rendering

I implemented VR180 and VR360 (Omnidirectional stereo) render cameras to support VR video rendering. This allows users to render stereo equirectangular videos to view on VR headsets or post on YouTube. Documentation is [here](https://docs.nerf.studio/quickstart/custom_dataset.html#render-vr-video).
Expand Down
1 change: 1 addition & 0 deletions nerfstudio_vfx_blender.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ I also have a blog post style walkthrough of making it.
<li><a href="nerfstudio_vfx_blender.html" class="button small">Read More</a></li>
</ul>

Some of my renders were shown in an educational video on NeRFs [here](https://www.youtube.com/watch?v=1uoaRGvy3Rc).


<ul class="actions">
Expand Down
42 changes: 42 additions & 0 deletions nerfstudio_vfx_reel.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
---
layout: post
title: Nerfstudio VFX Reel

show_tile: false
---

<ul class="actions">
<li><a href="research.html" class="button small">Go back and see more research projects</a></li>
</ul>

## NeRF VFX Renders
<center>
<video id="v0" width=700px controls>
<source src="assets/videos/ns_vfx_trailer_short.mp4" type="video/mp4" />
</video>
</center>

<center>
<video id="v0" width=700px controls>
<source src="assets/videos/ns_vfx_lego.mp4" type="video/mp4" />
</video>
</center>

### Compositing with footage
<center>
<video id="v0" width=700px controls>
<source src="assets/videos/ns_vfx_elevator_composites.mp4" type="video/mp4" />
</video>
<center>

### Breakdown
<center>
<video id="v0" width=700px controls>
<source src="assets/videos/ns_vfx_breakdown.mp4" type="video/mp4" />
</video>
<center>


<ul class="actions">
<li><a href="research.html" class="button small">Go back and see more research projects</a></li>
</ul>
2 changes: 1 addition & 1 deletion research.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ menu-show: true
<br>
- Shown in CVPR 2023 Art Gallery
<br>
- Arxiv submission in progress</p>
- Abstract on Arxiv <a href="https://arxiv.org/abs/2401.08633">here</a></p>
<ul class="actions">
<li><a href="https://docs.nerf.studio/extensions/blender_addon.html" class="button">Documentation</a></li>
<li><a href="https://www.youtube.com/watch?v=A7La8tWp_0I" class="button">Video</a></li>
Expand Down
2 changes: 1 addition & 1 deletion resume.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,5 @@ nav-menu: true
---


<iframe src="assets/pdfs/Cyrus Vachha-Resume Fall 2023.pdf" width="100%" height="500px">
<iframe src="assets/pdfs/Cyrus Vachha-Resume Spring 2024.pdf" width="100%" height="500px">
</iframe>

0 comments on commit e1a668e

Please sign in to comment.