ARKit – 如何显示来自放置在 SCNPlane 上的虚拟 SCNCamera 的提要?
我使用 ARKit 和 SceneKit 在 AR 空间中放置了一些对象.这很好用.现在我想添加一个额外的相机 (SCNCamera),它放置在场景中的其他地方,由一个公共 SCNNode 附加和定位.它旨在从另一个(固定)角度向我展示当前场景.
I put some objects in AR space using ARKit and SceneKit. That works well. Now I'd like to add an additional camera (SCNCamera) that is placed elsewhere in the scene attached and positioned by a common SCNNode. It is oriented to show me the current scene from an other (fixed) perspective.
现在我想在 i.Ex 上展示这个额外的 SCNCamera 提要.一个 SCNPlane(作为第一个漫反射材料) - 就像一个电视屏幕.当然,我知道它只会显示留在相机焦点中的 SceneKit 内容,而不是 ARKit 图像的其余部分(当然只有主相机才有可能).一个简单的彩色背景就可以了.
Now I'd like to show this additional SCNCamera feed on i.Ex. a SCNPlane (as the diffuse first material) - Like a TV screen. Of course I am aware that it will only display the SceneKit content which stays in the camera focus and not rest of the ARKit image (which is only possible by the main camera of course). A simple colored background then would be fine.
我看过介绍如何在 ARSpace 中的虚拟显示器上播放视频文件的教程,但我需要一个来自我自己当前场景的实时摄像头.
I have seen tutorials that describes, how to play a video file on a virtual display in ARSpace, but I need a realtime camera feed from my own current scene.
我定义了这个对象:
let camera = SCNCamera()
let cameraNode = SCNNode()
然后在 viewDidLoad 我这样做:
Then in viewDidLoad I do this:
camera.usesOrthographicProjection = true
camera.orthographicScale = 9
camera.zNear = 0
camera.zFar = 100
cameraNode.camera = camera
sceneView.scene.rootNode.addChildNode(cameraNode)
然后我调用我的 setup 函数将虚拟显示器放置在我所有的 AR 东西旁边,同时定位 cameraNode(指向物体留在场景中的方向)
Then I call my setup function to place the virtual Display next to all my AR stuff, position the cameraNode as well (pointing in the direction where objects stay in the scene)
cameraNode.position = SCNVector3(initialStartPosition.x, initialStartPosition.y + 0.5, initialStartPosition.z)
let cameraPlane = SCNNode(geometry: SCNPlane(width: 0.5, height: 0.3))
cameraPlane.geometry?.firstMaterial?.diffuse.contents = cameraNode.camera
cameraPlane.position = SCNVector3(initialStartPosition.x - 1.0, initialStartPosition.y + 0.5, initialStartPosition.z)
sceneView.scene.rootNode.addChildNode(cameraPlane)
一切都编译和加载... 显示出现在给定的位置,但它保持完全灰色.我放在场景中的 SCNCamera 根本没有显示任何内容.AR 场景中的其他一切都运行良好,只是我没有从那台相机获得任何信息.
Everything compiles and loads... The display shows up at the given position, but it stays entirely gray. Nothing is displayed at all from the SCNCamera I put in the scene. Everything else in the AR scene works well, I just don't get any feed from that camera.
有没有人有办法让这个场景工作?
Hay anyone an approach to get this scenario working?
为了更好地可视化,我添加了更多打印屏幕.
To even better visualize, I add some more print screens.
以下显示了根据 ARGeo 输入的 SCNCamera 图像.但它需要整个屏幕,而不是像我需要的那样在 SCNPlane 上显示其内容.
The following shows the Image trough the SCNCamera according to ARGeo's input. But it takes the whole screen, instead of displaying its contents on a SCNPlane, like I need.
下一个打印屏幕实际上显示了我使用发布的代码获得的当前 ARView 结果.如您所见,灰色的显示平面保持灰色 - 它什么也没显示.
The next Print screen actually shows the current ARView result as I got it using my posted code. As you can see, the gray Display-Plane remains gray - it shows nothing.
最后一个打印屏幕是蒙太奇照片,显示了我想要的预期结果.
The last print screen is a photomontage, showing the expected result, as I'd like to get.
这是怎么实现的?我在这里遗漏了一些基本的东西吗?
How could this be realized? Am I missing something fundamental here?
经过一番研究和睡眠,我得出了以下可行的解决方案(包括一些无法解释的障碍):
After some research and sleep, I came to the following, working solution (including some inexplainable obstacles):
目前,额外的 SCNCamera 提要未链接到 SCNPlane 上的 SCNMaterial,因为这是最初的想法,但我将使用额外的 SCNView(暂时)
Currently, the additional SCNCamera feed is not linked to a SCNMaterial on a SCNPlane, as it was the initial idea, but I will use an additional SCNView (for the moment)
在定义中,我添加了另一个视图,如下所示:
In the definitions I add an other view like so:
let overlayView = SCNView() // (also tested with ARSCNView(), no difference)
let camera = SCNCamera()
let cameraNode = SCNNode()
然后,在 viewDidLoad 中,我设置了这样的东西...
then, in viewDidLoad, I setup the stuff like so...
camera.automaticallyAdjustsZRange = true
camera.usesOrthographicProjection = false
cameraNode.camera = camera
cameraNode.camera?.focalLength = 50
sceneView.scene.rootNode.addChildNode(cameraNode) // add the node to the default scene
overlayView.scene = scene // the same scene as sceneView
overlayView.allowsCameraControl = false
overlayView.isUserInteractionEnabled = false
overlayView.pointOfView = cameraNode // this links the new SCNView to the created SCNCamera
self.view.addSubview(overlayView) // don't forget to add as subview
// Size and place the view on the bottom
overlayView.frame = CGRect(x: 0, y: 0, width: self.view.bounds.width * 0.8, height: self.view.bounds.height * 0.25)
overlayView.center = CGPoint(x: self.view.bounds.width * 0.5, y: self.view.bounds.height - 175)
然后,在其他一些函数中,我将包含 SCNCamera 的节点放置到我想要的位置和角度.
then, in some other function, I place the node containing the SCNCamera to my desired position and angle.
// (exemplary)
cameraNode.position = initialStartPosition + SCNVector3(x: -0.5, y: 0.5, z: -(Float(shiftCurrentDistance * 2.0 - 2.0)))
cameraNode.eulerAngles = SCNVector3(-15.0.degreesToRadians, -15.0.degreesToRadians, 0.0)
结果,是屏幕底部的一种窗口(新的 SCNView),显示与主场景视图中相同的 SceneKit 内容,通过 SCNCamera 的视角及其节点位置进行查看,这非常好.
The result, is a kind of window (the new SCNView) at the bottom of the screen, displaying the same SceneKit content as in the main sceneView, viewed trough the perspective of the SCNCamera plus its node position, and that very nicely.
在一个常见的 iOS/Swift/ARKit 项目中,这种构造会产生一些副作用,人们可能会遇到这些副作用.
In a common iOS/Swift/ARKit project, this construct generates some side effects, that one may struggle into.
1) 主要是,新的 SCNView 从所需的角度显示 SceneKit 内容,但背景始终是实际的物理相机源.我无法弄清楚如何通过仍然显示所有 SceneKit 内容来使背景成为静态颜色.更改新场景的背景属性也会影响整个主场景,实际上是不想要的.
1) Mainly, the new SCNView shows SceneKit content from the desired perspective, but the background is always the actual physical camera feed. I could not figure out, how to make the background a static color, by still displaying all the SceneKit content. Changing the new scene's background property affects also the whole main scene, what is actually NOT desired.
2) 听起来可能令人困惑,但是一旦包含以下代码(这对于使其工作至关重要):
2) It might sound confusing, but as soon as the following code get's included (which is essential to make it work):
overlayView.scene = scene
整个场景的动画速度(两者)双倍!(为什么?)
the animation speed of the entire scenes (both) DOUBLES! (Why?)
我通过添加/更改以下属性纠正了这个问题,这几乎像正常(默认)一样恢复了动画速度行为:
I got this corrected by adding/changing the following property, which restores the animation speed behavour almost like normal (default):
// add or change this in the scene setup
scene.physicsWorld.speed = 0.5
3)如果项目中有SCNAction.playAudio之类的动作,所有的效果都将不再播放——只要我不这样做:
3) If there are actions like SCNAction.playAudio in the project, all the effects will no longer play - as long as I don't do this:
overlayView.scene = nil
当然,额外的 SCNView 停止工作,但其他一切都会恢复正常.
Of course, the additional SCNView stops working but everything else gets gets back to its normal.