aframe-multi-camera

multi-camera.mp4

Overview

This repository contains

components that can be used to create multi-camera scenes in A-Frame. Cameras can render to the canvas, or a to plane surface within the scene.
components to create a viewpoint selector widget for a A-Frame scene being displayed in 2D. This widget provides an intuitive interface that allows a user to jump to any viewing angle of an object, in 45 degree increments. This is one example of how the multi-camera function can be used.

Examples

We have the following examples:

Multiple Cameras

Multiple cameras and canvases: https://diarmidmackenzie.github.io/aframe-multi-camera/examples/multi-screen.html

Cursor use across multiple secondary cameras: https://diarmidmackenzie.github.io/aframe-multi-camera/examples/multi-screen-with-cursor.html

Multiple camera views embedded in an HTML page without any background full-screen A-Frame scene: https://diarmidmackenzie.github.io/aframe-multi-camera/examples/embedded-views.html

Secondary cameras rendering to planes in the scene, to create a CCTV monitor effect: https://diarmidmackenzie.github.io/aframe-multi-camera/examples/camera-texture.html

Primary camera rendering to a plane in the scene, to create a CCTV monitor effect: https://diarmidmackenzie.github.io/aframe-multi-camera/examples/monitor-user-view.html

Mirrors rendered to planes in the scene: https://diarmidmackenzie.github.io/aframe-multi-camera/examples/mirror-example.html

Viewpoint Selector

A basic example of a scene where the central objects can be manipulated using the viewpoint-selector: https://diarmidmackenzie.github.io/aframe-multi-camera/examples/viewpoint-selector-basic.html

A second example, demonstrating various different configuration options for the viewpoint selector.https://diarmidmackenzie.github.io/aframe-multi-camera/examples/viewpoint-selector-alternate.html

Components

The components in this repository are as follows:

add-render-call - This is a generic component that supports extensions to the main A-Frame render cycle, allowing for rendering from multiple cameras, and to multiple canvases.
secondary-camera - This is a generic component that uses add-render-call to support one or more secondary cameras, optionally also using a cursor.
layers - This allows configuration of layers on objects. Layers control whether an object can be seen by a camera, whether it is intersected with a raycaster etc. They are used to allow the viewpoint selector to exist in the main A-Frame scene, without interfering with it.
viewpoint-selector - This implements the cube-shaped selector itself.
viewpoint-selector-renderer - This configures the viewpoint-selector, and the accompanying cursor, camera, lighting and renderer

The configuration options for each of these components are detailed below.

The following components are used to support examples.

mouse-rotate-controls - A very simple utility component that allows an entity in a scene to be rotated by dragging the mouse (no configuration parameters).
hover. A very simple hove effect, used to demonstrate cursor controls in some of the examples. (no configuration parameters)

Installation

There are 4 modules that you may need.

To use secondary cameras at all, you need the multi-camera.js module from the src directory.
To use mouse cursor on a secondary camera, you will also need the cursor.js module from the src directory (see note below).
To use the viewpoint selector, you will need both of the above, and also the viewpoint-selector.js module from the src directory.
To use mirrors, you just need the mirror.js module from the src directory

You can either download them and include them like this:

Or via JSDelivr CDN (check the releases in the repo for the best version number to use)

Theses components are not yet available in npm. If that would be useful to you, please raise an issue against the repo and I'll work on it...

A note on cursor.js

Using the mouse cursor on a canvas or camera that is not the main A-Frame canvas/camera depends upon a fix that is not yet available in any main A-Frame release (see this PR for latest status on getting this into A-Frame.)

Therefore you will need to include the patched version of A-Frame cursor.js from the aframe directory of this project. This module can be included in an HTML file after including A-Frame to patch cursor.js.

Limitations

Here are some current limitations with these components.

Various components have limited support for updates (i.e. changes to properties after initial creation). I'm happy to fix issues like this up on a case by case basis - please raise an issue on the repository. PRs for these fixes also welcome too!.
Rendering in-scene (i.e. mirror and secondary cameras that are not rendering to screen) currently only render to planes. Rendering to other flat shapes (circle, triangle, ring) should be straightforward - if someone raises an issue or PR I'll happily take a look. Not sure about rendering to 3D surfaces - getting some texture in place is probably not too hard. Getting physically accurate mirror reflections is probably very difficult!
For rendering in-scene, there is no frustrum culling. Textures are rendered whether or not they are actually in view of camera.
Rendering to screen is intended for desktop only, and is not intended to be used in VR. When in VR, rendering is skipped for secondary-cameras that render to screen (for rendering to a VR HUD, you can use rendering in-scene to a plane fixed directly in front of the camera).
In VR (tested on Oculus Quest 2) , mirror textures seem to be extremely pixelated, regardless of "quality" settings. I haven't yet understood why this is.
Cursor controls cannot be used on mirror textures, or secondary cameras rendered to a plane - only to secondary cameras rendered to screen. If you have specific use cases that need cursor on either of these, raise an issue and I will take a look at how hard this is.

Components

add-render-call

This is a generic base component that supports extensions to the main A-Frame render cycle, allowing for rendering from multiple cameras, and to multiple canvases.

If you use secondary-camera or viewpoint-selector-renderer, they will set this up for you, and you don't need to worry about this component.

This could be added anywhere in the scene and should have the same effect, but typically it is added to the scene. Multiple instances of this component can be added, in which case they will compound with each other, in the order that they are initialized in.

NOTE: for reasons unknown, it appears that this doesn't work unless it has an identifier - so add specify as e.g. add-render-call___1 rather than add-render-call

Property	Default value	Description
entity	none	A DOM selector for an entity that has a component configured on it with a suitable render extension function.
componentName	none	The name of the component on the entity that has a suitable render extension function.
sequence	'after'	'before', 'after' or 'replace'. Determines whether the camera view is rendered before or after the main scene is rendered, or whether this camera replaces the main camera completely. If 'replace' is set on any one secondary camera, this will mean that the main camera is not rendered.

This component expects the specified entity / component to provide a suitable render extension function.

This takes the following form:

It is available on the component as this.render(renderer, renderFunction)
It is written to be called after the main A-Frame rendering is completed, to perform additional rendering steps. It can and should assume that renderer.autoClear has been set to false.
It expects to be passed the following parameters:
- renderer: The THREE.WebGLRenderer object being used to render the scene
- renderFunction: The original render() function for this renderer, which can be used to perform any additional rendering required. This should be used in place of renderer.render(), which has been co-opted by this component (and calling it would lead to infinite recursion)

For an example of a suitable render extension function, see render() in viewpoint-selector-renderer.

WARNING: removal of this component is currently bugged: see "Limitations".

secondary-camera

This is a generic component that uses add-render-call to support one or more secondary cameras, optionally also using a cursor.

Property	Default value	Description
output	'screen'	'screen' or 'plane'. Determines whether the camera output goes to screen, or to a plane within the scene.
outputElement	none	The DOM element that camera output should be rendered to. For output = 'screen', this is typically a HTML div outside the A-Frame scene, with suitable styling to set its position. For output='plane' this should be an A-Frame entity of type . It also needs to have the src parameter specified with a texture (any texture will do) - see examples. (I should be able to write some code to lift this restriction, but didn't do it yet!)
cameraType	orthographic	Either "orthographic" or "perspective". Determines the type of camera used to render the viewpoint selector cube. Other camera parameters (e.g. position, fov etc.) are not yet configurable.
sequence	'after'	'before', 'after' or 'replace'. Determines whether the camera view is rendered before or after the main scene is rendered, or whether this camera replaces the main camera completely. Typically set to 'after' when rendering to screen, and 'before' when rendering to a plane within the scene (so that the rendered plane is included in the final scene render). If 'replace' is set on any one secondary camera, this will mean that the main camera is not rendered.
quality	"high"	Either "high" or "low". Only applicable when output='plane'. This determines the resolution used for the texture. "low" quality results in a lower-resolution texture, where there will be more noticeable pixelation, but will support a higher frame-rate.

This component provides generic second camera functionality in the scene.

In addition to the properties explicitly configured on the secondary-camera component, the secondary camera will also respond to the following components configured on the same entity.

Component	Effect
position	Used to control the position of the camera
rotation	Used to to configure the orientation of the camera
cursor	If this is configured, it should be configured with the properties "camera: user; canvas: user". The secondary camera will reconfigure the cursor and underlying raycaster to work with this camera and canvas. Note that this functionality depends on an enhancement to cursor.js that is not in A-Frame yet. See "Installation" above.
layer	Used to set which layer the camera views. If a cursor component is configured on this entity, the underlying raycaster will also be configured to use the same layer. (WARNING: I didn't test this function yet. Hopefully it works!)

See the examples for how secondary-camera can be used.

layers

This allows configuration of layers on objects. Layers control whether an object can be seen by a camera, whether it is intersected with a raycaster etc. They are used to allow the viewpoint selector to exist in the main A-Frame scene, without interfering with it.

It takes a single un-named property,

See the THREE.js docs for more background on Layers.

NOTE: Layer 0 is the default layer. In VR, layers 1 & 2 are reserved for the left & right eyes respectively. So if you want to use custom layers, it's best to use layer 3+.

viewpoint-selector

This implements the cube-shaped selector itself. If you use viwpoint-selector-renderer, this sets up viewpoint-selector, and you won't need to use this component directly.

Property	Default value	Description
outer	1	The dimension (in metres) of the rotating cube.
inner	0.6	The dimension (in metres) of the inner faces of the cube.
text	#000000 (black)	The color used for the text on the cube
color	#FFFFFF (white)	The color used for the cube itself
hoverColor	#BBBBBB (light grey)	The color used for the cube segments when the mouse hovers over them
layer	0	The layer used to render the viewpoint selector. Lighting, camera and raycaster must all also use this layer.
syncTarget	None	A DOM selector for a target that should have its rotation synchronized with the viewpoint-selector. Typically this target will be a wrapper for all the objects you want to control with the viewpoint selector. When this target moves, the viewpoint selector moves to match it. When segments of the viewpoint selector are clicked, the viewpoint selector rotates and the target rotates with it.
angleOffset	0 0 0	An angle offset between the viewpoint selector and the syncTarget. Specify this when the default (zero) rotation for the target is not "0 0 0". The viewpoint selector will maintain this offset between itself and the target.
animationTimer	500	The time in msecs taken to rotate to a new fixed position after the user clicks on a segment of the viewpoint selector.

viewpoint-selector-renderer

This configures the viewpoint-selector, and the accompanying cursor, camera, lighting and renderer.

Much of the code here is similar to secondary-camera, and this component could probably be rewritten to use secondary-camera (in which case it could become a lot simpler).

Property	Default value	Description
displayBox	#viewpoint-selector-box	A DOM selector for an HTML element which should be used as a canvas to render the viewpoint-selector.
cameraType	orthographic	Either "orthographic" or "perspective". Determines the type of camera used to render the viewpoint selector cube. Other camera parameters (e.g. position, fov etc.) are not yet configurable.
other parameters...	various, see above.	These parameters are used to configure the viewpoint-selector itself. See above for details.

mirror

This can be used to configure one side of an entity to act as a mirror. This is not currently supported on any other primitive (though it should also work on an that is set up with a plane geometry)

Property	Default value	Description
quality	"high"	Either "high" or "low". This determines the resolution used for the texture. "low" quality results in a lower-resolution texture, where there will be more noticeable pixelation, but will support a higher frame-rate.

In addition to the properties explicitly configured on the mirror component, the plane should be configured as follows:

For an opaque-backed mirror: side or material.side should be set to back. Other material parameters should be set to whatever material effect is desired for the mirror back.
For a mirror that is invisible when viewed from behind, set opacity or material.opacity to 0.

Acknowledgments

The viewpoint selector and viewpoint-selector-renderer components were originally developed in collaboration with Virtonomy (https://virtonomy.io), and formed the foundation for several other components in this repository.

I am grateful to them for giving me permission to release these components as Open Source under the MIT License.

The mirror component is just a thin wrapper around the THREE.js Reflector class, as can be seen in this example.

Implementation Notes

Single vs. multiple WebGL Contexts

Some solutions for secondary cameras that I have seen elsewhere use multiple WebGL contexts (i.e. multiple canvases) to achieve the necessary rendering.

E.g. https://jgbarah.github.io/aframe-playground/camrender-01/

(based on: https://wirewhiz.com/how-to-use-a-cameras-output-as-a-texture-in-aframe/...)

In these components, I've avoided using multiple WebGL Contexts, instead using a single WebGL context, but invoking the render() method multiple times.

With multiple WebGL contexts, the scene, geometry, textures etc. all need to be uploaded to the GPU for each context, which certainly increases memory usage, and I suspect impacts performance as well.

I don't honestly know how great the benefits of using a single WebGL Context are, but I think there ought to be at least some benefit. Maybe I'll try to do some performance comparisons at some point.

Note that even using a single WebGL context, there's still a draw call required for every object, on every camera, so secondary cameras do have a significant performance hit even when using just a single WebGL context - but hopefully less than it would have been otherwise.

add-render-call vs. onBeforeRender()

The mirror component uses THREE.js object3D.onBeforeRender() callbacks, rather than using the add-render-call component.

I've experimented with using object3DonBeforeRender() for secondary-camera but hit issues with recursive render() calls that I couldn't easily solve. I haven't yet understood why the mirror component doesn't hit the same problems.

However I'm interested to see whether moving mirror to use add-render-call might help with the highly pixelated textures in VR (see "Limitations").

add-render-call vs. tick()

The aframe-playground example mentioned above uses the tick() function to perform the pre-rendering to textures.

I've been wondering whether add-render-call is over-complicated, and instead of using this for rendering "before" and "after" the main render step, we could just use the tick(0 and tock() methods already offered by A-Frame - which would be somewhat simpler.

So far I'm not inclined t make that change. Key reasons being:

If we used tick() and tock() processing for the additional render steps, this would get mixed up in terms of scheduling with any other tick() and tock() processing going on in the scene. By inserting the additional processing into the main render() call for the scene, we avoid any issues that might arise when other components' tick() or tock() method acts before we have finished rendering.
By keeping all render calls together, the add-render-call simplifies tracking rendering statistics (as used in the A-Frame stats component), including correctly counting draw calls.. Given that additional cameras do have a significant performance impact, I think it's useful to be able to track accurate statistics for the render processing.

Tests

As well as the examples folder, there is also a tests folder.

This contains modified versions of examples, in order to test specific functions (so far I've focussed on cleanly adding / removing secondary cameras).

These are not in the main examples folder, as they don't offer any particular additional "how to" insight.

Hi @diarmidmackenzie!

I have had a lot of fun experimenting with your aframe-multi-camera tool. I'm currently working to integrate this tool into the next version of our 3DStreet viewer. I am working out of the new-coat-of-paint-2.0 branch in the 3DStreet repository:

https://github.com/3DStreet/3dstreet/tree/new-coat-of-paint-2.0

Here are a few important parts of my implementation so far:

I am importing your code via. CDN in the head of my HTML file.

<script src="https://cdn.jsdelivr.net/gh/diarmidmackenzie/aframe-multi-camera@latest/aframe/cursor.min.js"></script>
<script src="https://cdn.jsdelivr.net/gh/diarmidmackenzie/aframe-multi-camera@latest/src/multi-camera.min.js"></script>
<script src="https://cdn.jsdelivr.net/gh/diarmidmackenzie/aframe-multi-camera@latest/src/viewpoint-selector.min.js"></script>
<script src="https://cdn.jsdelivr.net/gh/diarmidmackenzie/aframe-multi-camera@latest/src/mirror.min.js"></script>

I am also adding divs with ids viewport1, viewport2, etc. to a toolbar that is located in the top part of the page.

<div class="menu-img" id="viewport1" style="border-style: solid; width: 200px; height:130px">
</div>

Finally, I am adding a set of second cameras to my a-scene

<a-entity id="camera1" secondary-camera="outputElement:#viewport1" position="-8 1.6 0" rotation="0 -90 0"></a-entity>
<a-entity id="camera2" secondary-camera="outputElement:#viewport2" position="-8 1.6 0" rotation="0 -90 0"></a-entity>
<a-entity id="camera3" secondary-camera="outputElement:#viewport3" position="-8 1.6 0" rotation="0 -90 0"></a-entity>
<a-entity id="camera4" secondary-camera="outputElement:#viewport4" position="-8 1.6 0" rotation="0 -90 0"></a-entity>

While I think that I am heading in the right direction, I am getting an error that crashes my application

three.js:20378 Uncaught RangeError: Maximum call stack size exceeded
    at setProgram (three.js:20378:1)
    at WebGLRenderer.renderBufferDirect (three.js:19887:1)
    at renderObject (three.js:20278:1)
    at renderObjects (three.js:20254:1)
    at renderScene (three.js:20205:1)
    at WebGLRenderer.render (three.js:20084:1)
    at o.render (multi-camera.js:66:27)
    at PMREMGenerator._textureToCubeUV (three.js:11811:1)
    at PMREMGenerator._fromTexture (three.js:11680:1)
    at PMREMGenerator.fromCubemap (three.js:11619:1)

Any suggestions on how to resolve this issue?

cc @kfarr for context

A-Frame Rainfall effect component

A-Frame Rain component aframe-rain is Rainfall effect component for A-Frame which displays a lot of rain drop/splash objects by using instancing techn

Dec 19, 2021

Get the first frame of a Gif image.

Apr 8, 2022

Minimalist Web XR Location Based Markers for A-Frame 1.3.0

LBAR.js “I understand how the engines work now. It came to me in a dream. The engines don't move the ship at all. The ship stays where it is and the e

Dec 3, 2022

Frame Animation Manager

Clockz.js (demo) Clockz is a Super-smooth Frame Animation manager to help you manage multiples animations in same time. Installation Just include this

Mar 27, 2021

Simple rate-limiter NPM Module used for blocking IPs that exceeds certain number of requests per second in a specific time frame.

API Rate Limiter Zero-Dependencies Simple rate-limiter NPM Module used for blocking IPs that exceeds certain number of requests per second in a specif

Oct 7, 2022

A ready-to-use project template to build custom elements (web components) with Svelte 3 with support and examples for web components, jest, sass, nested components with props, eslinting, stylelinting, Github actions, propagating custom events from shadow-DOM to real-DOM etc.

svelte-webcomponents If you’re using web-components or if you like the project, please ★ this repository to show your support! 🤩 The world's most eas

Nov 9, 2022

True P2P concept for your p2p powered website/app/client. MSC/MEP (Multiple Strategy Concept/Multiple Entry Points)

TRUE P2P CONCEPT - Lets redecentralize the web This repo is just conceptual. Active development of the endproduct (TRUE P2P) happens here https://gith

Mar 29, 2022

⚡🚀 Call multiple view functions, from multiple Smart Contracts, in a single RPC query!

ethers-multicall ⚡ 🚀 Call multiple view functions, from multiple Smart Contracts, in a single RPC query! Querying an RPC endpoint can be very costly

Dec 30, 2022

A React Component library implementing the Base design language

Base Web React Components Base is a design system comprised of modern, responsive, living components. Base Web is the React implementation of Base. Us

Dec 29, 2022

A pure node.js JavaScript Client implementing the MySQL protocol.

mysql Table of Contents Install Introduction Contributors Sponsors Community Establishing connections Connection options SSL options Connection flags

Jan 1, 2023

Example project implementing authentication, authorization, and routing with Next.js and Supabase

Magic Link Authentication and Route Controls with Supabase and Next.js To run this project, To get started with this project, first create a new proje

Dec 11, 2022

Implementing hexagonal architecture on AWS Lambda with Node.js

Developing evolutionary architecture with AWS Lambda Context Agility enables you to evolve a workload quickly, adding new features, or introducing new

Dec 20, 2022

Built a covid-19 trcaker app using React.js implementing hooks and materail UI

Getting Started with Create React App This project was bootstrapped with Create React App. Available Scripts In the project directory, you can run: np

Dec 21, 2021

Venni backend - The backend of the Venni client apps implementing the credit card payments, matching algorithms, bank transfers, trip rating system, and more.

Cloud Functions Description This repository contains the cloud functions used in the Firebase backend of the Venni apps. Local Development Setup For t

Jan 3, 2022

A project implementing a datepicker with format dd/MM/yyyy from scratch.

This is a Next.js project bootstrapped with create-next-app. Getting Started First, run the development server: npm run dev # or yarn dev Open http://

Jan 11, 2022

It consists of a recreation of Twitter, to put into practice both Front-end and Back-end knowledge by implementing the MERN Stack together with other technologies to add more value to the project.

Twitter-Clone_Back-end ✨ Demo. 💻 About the project. 📜 Descriptions. It consists of a recreation of Twitter, to put into practice knowledge of both F

Apr 12, 2022

A Drag-and-Drop library for all JavaScript frameworks implementing an enhanced transformation mechanism to manipulate DOM elements

JavaScript Project to Manipulate DOM Elements DFlex A Drag-and-Drop library for all JavaScript frameworks implementing an enhanced transformation mech

Jan 8, 2023

Configure WhiteSource Bolt for GitHub

Welcome to WhiteSource Bolt for GitHub! This is an onboarding PR to help you understand and configure settings before WhiteSource starts scanning your repository for security vulnerabilities.

:vertical_traffic_light: WhiteSource Bolt for GitHub will start scanning your repository only once you merge this Pull Request. To disable WhiteSource Bolt for GitHub, simply close this Pull Request.

What to Expect

This PR contains a '.whitesource' configuration file which can be customized to your needs. If no changes were applied to this file, WhiteSource Bolt for GitHub will use the default configuration.

Before merging this PR, Make sure the Issues tab is enabled. Once you merge this PR, WhiteSource Bolt for GitHub will scan your repository and create a GitHub Issue for every vulnerability detected in your repository.

If you do not want a GitHub Issue to be created for each detected vulnerability, you can edit the '.whitesource' file and set the 'minSeverityLevel' parameter to 'NONE'.

:question: Got questions? Check out WhiteSource Bolt for GitHub docs. If you need any further assistance then you can also request help here.

opened by mend-bolt-for-github[bot] 0