Kiarian Posted April 24, 2020 Share Posted April 24, 2020 I received an email from NVIDIA promoting a new technique for converting a 2D photo into a 3D image. Article quotes:'To convert a single RGB-D input image into a 3D photo, a team of researchers from Virginia Tech and Facebook developed a deep learning-based image inpainting model that can synthesize color and depth structures in regions occluded in the original view."Classic image-based reconstruction and rendering techniques require elaborate capture setups involving many images with large baselines, and/or special hardware,” the researchers stated in their paper, 3D Photography using Context-aware Layered Depth Inpainting. “In this work, we present a new learning-based method that generates a 3D photo from an RGB-D input. The depth can either come from dual-camera cell phone stereo or be estimated from a single RGB image." 'To show the potential of the project, a separate researcher from Google has taken the code and developed a Chrome extension that turns every Instagram post into 3D images. Developers interested in setting up the extension can follow the tutorial to set up and run the project using NVIDIA GPUs on the Google Cloud Platform'Is this something Affinity Photo might have on the horizon? This isn't simply cutting away chucks of an image to then move in a parallax motion, the technique actually looks to give objects 3D properties. Also see YouTube link using this technique on old historic photos. The results are amazing. The picture comes alive! Full link as follows:https://news.developer.nvidia.com/automatically-convert-your-photos-into-3d-images-with-ai/?mkt_tok=eyJpIjoiWmpaa1l6ZGxPREExTXpsaiIsInQiOiJ1MzBYaDJIbjBIK0JlWHptajFZdCtDWTRQUU81UHVsU0Rhb3JNZTkxMHlCaXZlZnkxTzAzWWF6cDdQRXQ2UnRObVVEMXFZU0pzbVdpckdhXC85MlVOTVIrUmJnVnNBUnlvT0p5UGZ6VklkMHFXcCs0aVI0SVZDSzFOdkdRQWlzeUsifQ%3D%3D Patrick Connor and MEB 2 Quote Link to comment Share on other sites More sharing options...
wonderings Posted April 24, 2020 Share Posted April 24, 2020 That is some very cool stuff. Nothing useful for me but the effect is amazing. I know Photoshop has 3D capabilities, I toyed with some 3D models a while back. Does Photos have any 3D capabilities at all? Quote Link to comment Share on other sites More sharing options...
Kiarian Posted April 28, 2020 Author Share Posted April 28, 2020 On 4/24/2020 at 3:22 PM, wonderings said: That is some very cool stuff. Nothing useful for me but the effect is amazing. I know Photoshop has 3D capabilities, I toyed with some 3D models a while back. Does Photos have any 3D capabilities at all? Not that I know of. Definitely something for the future though. The technology isn't too far away though, couple of years I reckon. In the future we'll be able to look behind photos! Quote Link to comment Share on other sites More sharing options...
Andy05 Posted April 28, 2020 Share Posted April 28, 2020 I truly appreciated that they kept those examples, which showed some flaws (like, Queen's diamonds on her dress or Einstein's hair "stuck to the chalk board). But the results look amazing nonetheless. Kiarian 1 Quote »A designer's job is to improve the general quality of life. In fact, it's the only reason for our existence.«Paul Rand (1914-1996) Link to comment Share on other sites More sharing options...
R C-R Posted April 28, 2020 Share Posted April 28, 2020 On 4/24/2020 at 8:40 AM, Kiarian said: Is this something Affinity Photo might have on the horizon? This isn't simply cutting away chucks of an image to then move in a parallax motion, the technique actually looks to give objects 3D properties. If you mean the "deep learning-based image inpainting model" it probably isn't something we will see in AP any time soon unless some third party develops a dynamic library or framework that includes what that model has learned from being trained with a very large sample of images & permits Serif to use it for free or at a very low cost. Quote All 3 1.10.8, & all 3 V2.4.1 Mac apps; 2020 iMac 27"; 3.8GHz i7, Radeon Pro 5700, 32GB RAM; macOS 10.15.7 Affinity Photo 1.10.8; Affinity Designer 1.108; & all 3 V2 apps for iPad; 6th Generation iPad 32 GB; Apple Pencil; iPadOS 15.7 Link to comment Share on other sites More sharing options...
Kiarian Posted April 27, 2023 Author Share Posted April 27, 2023 On 4/28/2020 at 9:17 PM, R C-R said: If you mean the "deep learning-based image inpainting model" it probably isn't something we will see in AP any time soon unless some third party develops a dynamic library or framework that includes what that model has learned from being trained with a very large sample of images & permits Serif to use it for free or at a very low cost. Well, took a few years. 2023, and I think the AI revolution is here. Here is what Apple are doing with using FaceLit technology. Simple 2D images. I don't think it will be long until we see something like this in Affinity Photo and Photoshop.FaceLit: Neural 3D Relightable Faces - Apple Machine Learning Research Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.