Machine Learning: Select Subject

[Juhani]

2 hours ago, fde101 said:

If I am looking at it correctly, it seems to me, at least "subject"ively, that PixelMator Pro currently gives the best results of the three, except with the tree.  Not perfect, though.

Yeah, they've got their model quite good, often beats other tools and everything also happens locally, unlike in Photoshop.

2 hours ago, carl123 said:

PS I'm not sure if Photoshop or Pixelmator has refine capabilities but it would be nice to see a comparison of them compared to Affinity one day

They do, but that's another feature, thus didn't go into that. In my experience though Affinity's matte brush inside the Refine mode is one of the best, especially with hair. However it still struggled a lot with that tree example.

[RichardMH]

Select subject is a bit hit and miss. Not as good as Capture One. Object selection is working better for me. Not very stable though!

I haven't been using Photo much as it hasn't had AI masking. Might get back into it now. 

[Jdtimothy]

This feature was demoed at the Snapdragon conference this week for Windows Copilot+ PCs and said that it uses the NPU (neural processing unit) of the Snapdragon X Elite processor instead of the CPU. 
I can confirm that it is only using CPU and GPU, not the NPU. Will you be adding the ability to offload this to the NPU for quicker and more efficient selections?

[smadell]

I previously (Tuesday morning Oct 22) posted that I had been entirely unsuccessful in installing the ML models on my iPad Air (4th gen). Actually, installing them seemed to proceed just fine, but actually trying to use them caused nearly immediate crashes. This was repeatable, even when I installed one at a time, quit the app, then restarted and installed the second one. Force quitting the app, even deleting and reinstalling, did not result in anything but a crash.

About 30 minutes ago, I updated my iPadOS from 17.x to 18.0.1 (the most recent release currently available). Once this had completed, I started up Affinity Photo and made sure that the ML models were installed (they were). Furthermore, they now worked precisely as they are supposed to. I did not change anything other than update the OS. I don't know if this is expected behavior, or if this is a one-off on my iPad. But, if this is something that others are experiencing I can verify that updating to iPadOS 18 was all it took to go from crashes to expected working order.

[EmT]

12 hours ago, smadell said:

but actually trying to use them caused nearly immediate crashes

Was there anything specific you were doing that triggered the crash? 
We aren't currently aware of any issues specific to iOS 17 so any information you can give us would help us investigate. 

[EmT]

16 hours ago, Jdtimothy said:

This feature was demoed at the Snapdragon conference this week for Windows Copilot+ PCs and said that it uses the NPU (neural processing unit) of the Snapdragon X Elite processor instead of the CPU. 
I can confirm that it is only using CPU and GPU, not the NPU. Will you be adding the ability to offload this to the NPU for quicker and more efficient selections?

We're working very closely with Qualcomm to optimise Affinity's performance on the latest Snapdragon chips and as part of that we supply certain features for them to test. We'll say more about any plans for wider implementation when the time is right.

[Hangman]

Select Subject fails to select the subjects on right-hand facing pages following the first page in a Publisher document...

Steps to Reproduce - Recipe 1

1. Create a new facing-page Publisher document with five pages using the default Master
2. Add an image to all five pages
3. Switch to the Photo Persona
4. Select each image in turn in the Layers Panel
5. Go to Select > Select Subject
6. The image on Page 1 is selected as expected
7. The images on subsequent left-facing pages 2 and 4 are selected as expected
8. The images on subsequent right-facing pages 3 and 5 are not selected

Steps to Reproduce - Recipe 2

1. Create a new multipage Publisher document with two or more pages
2. Add an image to each page
3. Switch to the Photo Persona
4. Select each image in turn in the Layers Panel
5. Go to Select > Select Subject
6. The image on Page 1 is selected as expected
7. Images on subsequent pages are not selected
    

[NotMyFault]

17 minutes ago, Hangman said:

Select Subject fails to select the subjects on right-hand facing pages following the first page in a Publisher document...

AF-4512

[Hangman]

Hi @NotMyFault,

I think this is a different issue...

I can only replicate the issue shown in your screen recording when bleed is applied to the page... the selection offset matches the bleed value based on the top left corner, i.e., a bleed of 20 mm on all sides will offset the selection by X: -20 mm, Y: -20 mm...

[smadell]

3 hours ago, EmT said:

Was there anything specific you were doing that triggered the crash? 
We aren't currently aware of any issues specific to iOS 17 so any information you can give us would help us investigate.

Good morning, @EmT. I don't think there was anything particular or specific. I downloaded the beta from Serif (through Apple's TestFlight app). Once installed and loaded, I opened the Settings and clicked the buttons for the two machine learning models. They both seem to have installed (since the "Uninstall" button became available, and since the button/menu choices in the app also became available). I opened a photo (from one of the Stock sites) and either (i) chose Select Subject from the Selection menu; or (ii) chose the Object Selection button and clicked in an area of the photo.

In the first case ("Select Subject") the dialog appeared indicating that the process was starting. But after about 5-6 seconds, the app went black for a moment and crashed back to the home screen. In the second case (clicking on the photo using the Object Selection button) that click was followed by about 1-2 seconds of nothing and then, again, the screen went black for a moment and the app crashed back to the home screen.

Once iPadOS 18 was installed, the exact same actions invoked the appropriate actions.

[NotMyFault]

I got exactly one crash after installing the models. So it seems advisable to end the app after installation of ML models, before starting to work on documents.

[NotMyFault]

1 hour ago, Hangman said:

Hi @NotMyFault,

I think this is a different issue...

I can only replicate the issue shown in your screen recording when bleed is applied to the page... the selection offset matches the bleed value based on the top left corner, i.e., a bleed of 20 mm on all sides will offset the selection by X: -20 mm, Y: -20 mm...

My document had a bleed set, too.

But I cannot rule out that other possible triggers exists.

[Hangman]

16 minutes ago, NotMyFault said:

My document had a bleed set, too.

My report doesn't relate to bleed and the selection result being out of position it's the fact there is no selection at all...

[Watson Prunier]

Having an option for GPU only would be great! I have the NVIDIA Quadro RTX4000 that can handle these tasks. Taking my CPU and GPU is not a smart idea because nothing else can be done while it is learning and the system crashes.

[NotMyFault]

2 hours ago, Hangman said:

My report doesn't relate to bleed and the selection result being out of position it's the fact there is no selection at all...

I misinterpreted your comment.

The common observation is that it always affects images on the right side. Let the mods take over further analyses if both issues are related or not.

[Hangman]

9 minutes ago, NotMyFault said:

The common observation is that it always affects images on the right side.

The offset selection happens on a non-facing page spread as well...

I see the following with a facing-page document that includes Bleed...

• Image on Page 1 (Right Page) - the selection is offset by the dimensions of the bleed
• Image on Page 2 (Left Page) - the selection is offset by the dimensions of the bleed
• Image across Pages 1 and 2 (Left and Right Pages) - the selection is offset by the dimensions of the bleed
• Image on Page 3 (Right Page) - nothing is selected

[Westerwälder]

Biss dass das Machine Learning richtig funktioniert läuft noch viel Wasser den Rhein oder den River Trent hinab, kurz installiert nach vier versuchen wieder deinstalliert. 

Der Auswahlpinsel ist genauer! Da ist Affinity eine gute Software!

A lot of water still has to flow down the Rhine or the River Trent before the machine learning works properly. It was installed quickly and then uninstalled after four attempts. The selection brush is more precise! Affinity is a good software for that!

[user_0815]

2 hours ago, Westerwälder said:

Biss dass das Machine Learning richtig funktioniert läuft noch viel Wasser den Rhein oder den River Trent hinab, kurz installiert nach vier versuchen wieder deinstalliert. 

Der Auswahlpinsel ist genauer! Da ist Affinity eine gute Software!

A lot of water still has to flow down the Rhine or the River Trent before the machine learning works properly. It was installed quickly and then uninstalled after four attempts. The selection brush is more precise! Affinity is a good software for that!

Jede Reise beginnt mit dem ersten Schritt. 

[manu schwendener]

> AF-4720 - Crash after installing Machine Learning Models when Install is clicked multiple times

Yes. I clicked multiple times because the first time it froze at the start (progress bar opened, but didn't move).

Now trying to install Segmentation always crashes Affinity Photo. 

MacBook Pro (Retina, 13-inch, Early 2015), Mac OS Monterey 12.7.6

The crash generates a long report, do you want it?

---

Bravo for making it opt in and being very clear about what it does.

---

Edit 2.11.2024:

> available on Apple Silicon machines running on a recent macOS

Can confirm.
 

Edited November 2 by manu schwendener

[walt.farrell]

23 hours ago, NotMyFault said:

AF-4512

That's actually AF-4812, from the second post in this topic.

[Hangman]

3 minutes ago, walt.farrell said:

That's actually AF-4812, from the second post in this topic.

For clarification and for anyone encountering Machine Learning selection issues in the 2.6.0 (2805) Beta there are three distinct bugs:

• AF-4512 - Object Selection Tool Selections are off in Photo Persona When Bleed is Applied to the Page
• AF-4812 - Select Subject Fails to Select Subjects on Right-hand Facing Pages Following the First Page in a Publisher Document
• AF-4815 - The Quick Mask Bounding Box is Incorrectly Positioned for Selections on Non-Facing Page Spreads

AF-4512
This is specific to the document having bleed applied. It results in the selection made using the Object Selection Tool and Select Subject being offset negatively by the bleed amount in both the X and Y axes. This affects images on non-facing pages, images on the left and right sides of a facing-page spread and all pages in a multipage spread...

AF-4812
This is specific to using Select Subject (i.e. it doesn't affect the Object Selection Tool) on facing documents in a two-page spread. It only affects odd pages after page 1, i.e., a selection made on page 1 works correctly and selections made on even pages work correctly but selections made on odd pages starting at page 3 are not made at all evidenced by the application of a mask after the selection progress bar...

For multipage spreads, i.e., spreads with more than two pages a selection is made for the image on the first page of the spread but not for any pages after the first page...

AF-4815
This is specific to the position of a selection's bounding box when using the Object Selection Tool, Select Subject, the Selection Brush Tool, the Flood Select Tool or any other selection method...

Once the selection is made and the Quick Mask icon is clicked in the Context toolbar followed by the Move tool the selections' bounding box always appears relative to page 1, i.e.,

For non-facing pages, the selection's bounding box is offset to the right by exactly half the width of the selection but always adjacent to page 1 regardless of the page selected...

For facing-page documents the selections' bounding box is always aligned to page 1, i.e., if a left-facing page selection is made the selections' bounding box for any even page in the document appears on page 1, if a right-facing page selection is made the selections' bounding box appears to the right of page 1 off-canvas...

For multipage spreads the selections' bounding box is always aligned to page 1, i.e., if the selection is made on the first page of any spread the selections' bounding box appears on page 1, if the selection is made on the second page of any spread the selections' bounding box is offset by one page to the right of page 1 off-canvas, if the selection is made on the third page of any spread the selections' bounding box is offset by two pages to the right of page 1 off-canvas and so on...

[Patrick Connor]

Thanks for the clear post @Hangman

[jaxfh]

I'm thrilled to see that Affinity's new beta allows for ML functions to run locally on the machine. This is a fantastic approach for enhancing performance and privacy!

To make the most of this setup, I'd suggest exploring support for Neural Processing Units (NPUs) alongside the CPU and GPU. Many modern systems, like Apple’s M1 - M4, Intel Ultra, AMD AI and Snapdragon X chips, are equipped with NPUs that handle AI-driven processes with impressive efficiency. Utilizing the NPU could provide a significant speed boost for tools like Select Subject and Object Selection, which would lighten the load on the CPU and GPU while accelerating these ML-driven features even further.

NPU support could be a game changer, especially for processes like subject isolation, object recognition, or image segmentation, enhancing both speed and energy efficiency. This is something Affinity could consider for this and future versions.

Thanks to the Affinity team for this forward-thinking approach—I’m excited to see where this goes!

[Patrick Connor]

1 hour ago, jaxfh said:

I'd suggest exploring support for Neural Processing Units (NPUs) alongside the CPU and GPU. Many modern systems, like Apple’s M1 - M4, Intel Ultra, AMD AI and Snapdragon X chips, are equipped with NPUs that handle AI-driven processes with impressive efficiency....

Agreed we are already looking at the NPU chips. Indeed, we're working very closely with Qualcomm to optimise Affinity's performance on the latest Snapdragon chips. We'll say more about any plans for wider implementation when the time is right.

[RM f/g]

Photo crashes every time at the end of the download. Same for Publisher.

Crash log attached.

Affinity Photo 2 Beta-2024-10-27-140101.ips