# What is the difference between Lanczos seperable and non-seperable.

## Recommended Posts

I really need to know, because curiosity. Couldn't find anything else on google. It is quite recent technology it seems? Lots of pdfs etc about the algorithm.

• Which is faster?
• Which is better mathematically?
• Which one should I use :)

Ty.

##### Share on other sites

Nobody seems to know lol

##### Share on other sites

Hi socapex,

I can shed some light on this, Lanczos is one of the slower options as it uses complex algorithmic resampling which will give better results, but will increase the processing time. Separable and non-separable are different ways the algorithm applies itself to the image; separable basically means it is applied independently in the x and y directions.

Hope this helps.

##### Share on other sites

Thanks, Lee.  I was curious about this too.  The link, along with others that I Googled yesterday, told me a lot more than I really needed, or wanted, to know about Lanczos but leaves the questions unanswered, except for speed, in relation to Affinity. Within Affinity, which is better?  Any opinion on that?  TIA

##### Share on other sites

@LilleG exactly, all information I've found was extremely detailed, almost academic. Not that the algorithm theory isn't interesting, I haven't been able to settle on a conclusion: which is better (visually, not mathematically).

On another note, it isn't that slow. It's actually 42 billion times faster than competing software bilinear exports ;)

##### Share on other sites

Since no one knows what the difference is between Lanczos 3 "separable" and "non-separable" may I suggest that Serif put something more meaningful and descriptive into those parenthesis instead?

In my testing, Lanczos 3 (separable) enlargements and reductions a tad sharper than Bicubic, and Lanczos 3 (non-separable) is much sharper.

So I would think the Resample popup should read as follows:

• Nearest Neighbor
• Bilinear
• Bicubic
• Lanczos 3 (sharp)
• Lanczos 3 (sharper)

Because, quite honestly, "separable" and "non-separable" may appeal to the left-brainers, but isn't Affinity Photo appealing to our right-brain creative side?  Yes.  So make the descriptions more understandable, please!

##### Share on other sites

The help file of Affinity Photo should provide a guideline explaining the differences between these resample modes (the pros and cons).

Furthering the idea proposed by JDW, an interrogation mark should be provided directly in the Resample popup that, when clicked, makes appear an info box explaining these differences and —more importantly— proposes the best mode according to the app maker (Serif).

It's indeed peculiar that such basic information is nonexistent in the help file, in the resample popup, or on the web site of Serif!

##### Share on other sites

Furthering the idea proposed by JDW, an interrogation mark should be provided directly in the Resample popup that, when clicked, makes appear an info box explaining these differences and —more importantly— proposes the best mode according to the app maker (Serif).

I wish that it would automatically switch methods based on whether you're shrinking or enlarging an image. Barring that, I wish that anything other than Nearest Neighbor was the default.

##### Share on other sites

I wish that it would automatically switch methods based on whether you're shrinking or enlarging an image. Barring that, I wish that anything other than Nearest Neighbor was the default.

Recent versions of Photoshop have such an "Automatic" setting.  But an auto setting may be more difficult on Affinity Photo because AP has two types of Lanczos:

• Lanczos 3 (sharp) — roughly the same as Photoshop's Bicubic Sharper
• Lanczos 3 (sharper) — this setting is really sharp

So if Serif did implement an AUTO setting, I would assume it should choose Lanczos 3 (sharp) rather than Lanczos 3 (sharper), or maybe such could be a preference setting?

##### Share on other sites

The help file of Affinity Photo should provide a guideline explaining the differences between these resample modes (the pros and cons).

Furthering the idea proposed by JDW, an interrogation mark should be provided directly in the Resample popup that, when clicked, makes appear an info box explaining these differences and —more importantly— proposes the best mode according to the app maker (Serif).

It's indeed peculiar that such basic information is nonexistent in the help file, in the resample popup, or on the web site of Serif!

Furthering the idea of zakoops -

When I read his reply I immediately had before me the thought that it would certainly be an amazing Affinity ability if there was some key/click combination which one could perform when hovering over a tool or an option in the UI which would open the Help resource to just that part which pertained to the tool/option over which one had been hovering.  Unnecessary for the many who are already competent with the software I am sure, but it would be invaluable for the numerous other 'many' who are but beginners.

Probably too enormous an endeavour, too complicated a task, or merely too time consuming an effort to ever be accomplished but it would certainly get new users up and running in short order and help those with bad memories as well were it to ever be implemented.

I tend to have silly ideas like this more than occasionally (probably, more than is healthy).

##### Share on other sites

justwilliam, I think that having a "Help" explanation open automatically would be great, it would be even better to be able to turn it off if you no longer needed it.

##### Share on other sites

Count me among the people who haven't coded resampling filters and didn't know what "separable/non-separable" meant.

* A separable filter is one that is capable of first running in the X axis (width) and then in the Y axis (height). This means it's applied in two steps; first the image is made wider, and then it's made taller. Separable filters are faster than non-separable, because their time usage is proportional to the width (and then the height) of the image, instead of the whole area of the image.

* A non-separable filter is one that runs on both the X and Y axis at the same time. It's applied in a single step and the time usage is proportional to the whole area of the image.

Basically: It's a lot faster to resize an image in two steps using 1-dimensional passes (width first, then height), than to do it all in one step using a 2-dimensional pass (width and height at the same time).

Separable filters have to be specially written to support this two-step operation.

So:

Lanczos (non-separable) = slow

Lanczos (separable) = fast, up to 10x faster

PS: You guys proposing a rename to "sharp" / "sharper" don't get it. It's still the same algorithm. There's no "sharper" setting in one of them. If one of them *looks* sharper it's pure accident due to some algorithm differences between Affinity's 1-D (separable) and 2-D (non-separable) modes.

So I don't know what the proper solution is. A dev would have to explain why they decided to show us both options.

##### Share on other sites

PS: You guys proposing a rename to "sharp" / "sharper" don't get it. It's still the same algorithm. There's no "sharper" setting in one of them. If one of them *looks* sharper it's pure accident due to some algorithm differences between 1-D (separable) and 2-D (non-separable) modes.

Are you a designer or programmer?  If the latter, you're in the wrong place.  You are here among VISUAL DESIGNERS.  That means we go with what we see, not with what theory or scientific data says to us.  If something looks sharper to my graphically trained eye (with Photoshop and Illustrator experience that spans the entire life of those products), then IT IS SHARP.

PERIOD.

##### Share on other sites

@JDW. No... Sharpness is a separate parameter of the filter, regardless of mode...

You could have this:

Lanczos (non-separable; smooth)

Lanczos (non-separable; sharp)

Lanczos (non-separable; very sharp)

Lanczos (non-separable; extremely sharp)

Lanczos (separable; smooth)

Lanczos (separable; sharp)

Lanczos (separable; very sharp)

Lanczos (separable; extremely sharp)

So it makes no sense to rename "non-separable" as "sharper".

I guess what the devs *could* do, is to throw away the slow non-separable version (why is it even there?!), and instead tweak the sharpness parameter of the faster separable version to provide both a smooth and a sharp version of the fast 1D implementation.

The sharpness of a Lanczos filter is defined by the number of "lobes".

Further investigation showed that a Lanczos filter is fully separable, meaning that non-separable and separable are capable of identical output. The existence of the 10-15x slower "non-separable" mode means Affinity seem to have misimplemented Lanczos. There is no reason for the non-separable version of that filter to exist.

If all of this is correct, then it would be best to delete the non-separable (slow 2D version). Keep the separable (fast 1D version). Tweak the number of lobes to provide a few presets with varying sharpness, as such: http://avisynth.nl/index.php/Lanczos_lobs/taps

(Taps = affects the down/upscaling scale; Lobes = affects the sharpness; common values are 2 lobes = the best compromise between sharpness and low amount of ringing, and 4 lobes = more ringing but extra sharpness).

Why non-separable filters are slower: Upscaling in 1D only requires two 4-tap filters. Upscaling in 2D requires a single 16-tap (4*4 taps) filter, which is an order of magnitude slower. There is no reason to have a non-separable version of a Lanczos filter.

##### Share on other sites

Fair enough about your reasons given not to rename it, but the fact remains that one of the two existing settings yields greater sharpness than the other.  And the fact remains that in Photoshop, one similar setting is named "Sharper" because, well, it is.  And that was and is the basis for what I wrote.  If one setting is going to yield greater sharpness than another setting, then mentioning it (which is what Photoshop does), might be worthwhile.  That is all I was saying.

##### Share on other sites

@JDW: I agree that we should have varying sharpness presets. I've edited my post above with more details about how to achieve that.

##### Share on other sites

The separable and non-separable kernels are fundamentally different:

Here is a (rough) analogy using Wolfram Alpha:

Non-Separable:

http://www.wolframalpha.com/input/?i=sinc%28hypot%28x%2Cy%29%29+between+-10+and+10

Separable:

http://www.wolframalpha.com/input/?i=2*sinc%28x%29*sinc%28y%29+between+-10+and+10

We include them both because separable is faster - and some people even prefer the results!

Hope this helps,

Andy.

##### Share on other sites

@Andy Somerfeld:

Fantastic explanation. Visualizing the two filter modes gave me an "a-ha!" moment.

Basically: In the faster, separable 1D mode (width first, then height), it applies the sharpening as a "+" shaped grid pattern over the image. The intensity of the center pixel affects the intensity of the pixels above, below, left and right of it. This will result in an extremely minor "waffle/grid" pattern ariound single contrasting pixels, which can be seen if the sharpening (lobes) is made strong enough that it becomes visible. But normally it won't be seen, so people almost always use separable (1D) lanczos for its speed benefits.

In the slower, non-separable 2D mode (all dimensions at once), it applies the sharpening as an "o" shaped ring around the pixel. The intensity of the center pixel affects the intensity of the pixels in a ring around it. This looks a bit less jarring than having a "+" shaped grid. The resulting ringing is a bit smoother since there's an extra diagonal step in there between each pixel.

Regardless of mode, both versions of Lanczos suffer from "ringing" artifacts, which is the "circle"/"+" shaped halo around contrasting pixels (such as edges of objects) in the images.

So yep, the kernels are different. Since the slow 2D (non-separable) mode applies the sharpening as an "o" ring around the pixel, it will have a stronger effect and cause more sharpness in the image than the "+" pattern of the separable 1D mode.

Andy: What's your lobe setting? The standard 2 lobes for the best tradeoff between ringing/sharpness, I assume? If you don't mind doing some tests, it may be possible to get rid of the non-separable by doing this:

* Separable, lobes = 2, "normal sharpness"
* Separable, lobes = 4, "extra sharpness" -- visibly it might be almost equivalent to the non-separable kernel's sharpness, but much faster; it remains to be seen if the amount of visible ringing is equal to non-separable; if it's worse, then keep the non-separable mode too...

The reason I suggest this, is because every discussion and paper I read through while researching Lanczos filters said that the separable mode and non-separable modes are visibly almost identical. That makes sense, since the extra diagonal pixels of the "o" ring circle compared to the "+" shape don't really matter unless the image has very contrasting areas *and* the noise is amplified enough (such as via a high Lobe setting), so that the more aliased, "grid-like" diagonals become visible. Basically, the ringing of a 2D lanczos filter looks more "antialiased" than the ringing of a 1D lanczos filter, thanks to the extra diagonal steps, but the difference is so small that it shouldn't be noticeable either way (if the two filters use the same number of lobes and taps).

Alternatively, if it turns out that separable with 4 lobes is worse (more ringing) than non-separable with 2 lobes, then why not have these settings:

* Lanczos (separable; normal)
* Lanczos (separable; sharper)
* Lanczos (non-separable; higher quality; normal)
* Lanczos (non-separable; higher quality; sharper)

With 2 (normal) and 4 (sharper) lobes, respectively.

...And lastly, for those following along at home, I have attached two images; one showing the original, and one showing the 'ringing' artifacts of Lanczos (see the weird, gray outlines around the circle lines; that's the filter kernel's effect; remember to zoom in if reading this on mobile). In this case, the lanczos used in this image is a 1D version, which means the strange lines are in a "+" shape around the original pixels. You can confirm this if you zoom in to the left/right sides of the inner circle; there are faint 1 pixel gray dots above/below the black pixels, but not in their diagonals, so it confirms that a 1D lanczos was used. This test image is very unfair, since it's black on white; that means that all of the lanczos sharpening will be visible as tiny, gray dots, which is of course very noticeable. In normal images, and with a lobe count of 2, the ringing is very low.

And if people are interested in the difference between nearest neighbor, bilinear, bicubic and lanczos for enlarging, it's basically as follows from worst to best: Nearest neighbor = jaggy pixels (extreme aliasing), Bilinear = smooth pixels (the values are blended together; produces no ringing), Bicubic = sharper but has visible ringing, Lanczos = sharper than bicubic and rounder handling of diagonal curves than bicubic (less aliasing), similar amount of ringing.

##### Share on other sites

Looks to me that everyones argument is proving a singular point,

Lanczos seperable

Lanczos non-seperable

Has very little meaning to most designers, and do not accurately describe to designers what the end result target is for the image.

Nearest Neighbour

Bilinear

Bicubic

Are likely just as confusing for new users, but since they are established settings in other applications (photoshop for example) they confuse less people.

A more descriptive setting would help clarify the obvious issues people are having.

A little i. info graphic with pop-up with full description would also help inform the user what the settings end result aim is.

HOWEVER:

Ultimately when exporting an image a User generally wants to know:

Quality - ( Low --- ^ --- High)

Sharpness - (Smooth --- ^ ---- Sharp)

File Size Result .........

The actual naming of the equations used is mostly superflous to the user.

Perhaps let advanced users choose which equation they wish to use, and normal users simply choose with a little slider Quality and Sharpness and let Affinity decide the equation.

##### Share on other sites

Why dumb it down?  "Normal" users will either figure it out, or ask.

##### Share on other sites

The names of these filters are well-established in software, precisely because they're the names of the algorithms. It's like when you go to apply an "Unsharp Mask" or a "Gaussian Blur". You have learned what those mean, and you know that they mean the same thing regardless of what application you use. It's not "sharpness" or "blur", which would be too vague and generic.

Unfortunately, people are much less aware of how various scaling filters look. I mentioned the differences at the bottom of my post:

"And if people are interested in the difference between nearest neighbor, bilinear, bicubic and lanczos for enlarging, it's basically as follows from worst to best: Nearest neighbor = jaggy pixels (extreme aliasing), Bilinear = smooth pixels (the values are blended together; produces no ringing), Bicubic = sharper but has visible ringing, Lanczos = sharper than bicubic and rounder handling of diagonal curves than bicubic (less aliasing), similar amount of ringing." (Ringing means "halos" around contrasting edges between colors, such as at the edges of objects.)

I think the best thing Affinity could do is provide a graphical selection screen for beginners, with small, zoomed-in previews of each filter, all using the same test image (not the user's own image). They would then see that Nearest Neighbor = "big, ugly, square pixels; only useful for enlarging pixel art", "Bilinear" = "smoothed out pixels", "Bicubic" = sharp but has halos (ringing) around all contrasting outlines, "Lanczos" = even sharper *and* has smoother handling of diagonal curves (less aliasing/stair-stepping), and has a similar amount of ringing (halos).

Anyone just doing general resizing will probably always want Lanczos. Its quality is very good. If you get too much ringing, use Bilinear resizing instead and manually apply an Unsharp Mask filter with a ringing-free Radius to make it sharp again.

If the resize panel ever gets an "Advanced" mode, I'd also want to have access to 2-lobe, 3-lobe and 4-lobe versions of the filters so that I can choose how sharp or blurry the Lanczos resize should be.

I still hope the "non-separable" (Slow, 2D) version of Lanczos is removed, if indeed proven superfluous (as it was claimed by all the reference books and material I looked through): https://forum.affinity.serif.com/index.php?/topic/9378-what-is-the-difference-between-lanczos-seperable-and-non-seperable/&p=53992

##### Share on other sites

Why dumb it down?  "Normal" users will either figure it out, or ask.

I don't think anyone wants anything "Dumb" down.  Just simplified.

For curious users, like those few here at the forums a simple (Advanced Option) could allow for choosing preferred mathematical algorithm.

I have no problem with the export system as it stands now, I quickly researched it and tested a couple images and figured it out.

However, there are plenty of extremely talented artistic people who would look at those exports and just say, HUH? Choose one randomly and be upset if the result did not match their expectation.

So when my wife complains about such UI deficiencies, I don't roll my eyes and blame her for not understanding mathematical algorithms, I ask her what she wants the result to be and choose from there.

It is really easy for Geeks to criticize average users lack of understanding of complex computer operations.

When you explain these operations in a clear simplified explanation, often people say, "Why doesn't it just say that!"

And THEY are RIGHT.  No matter how far up the hackles get raised by computer programers, engineers, UI Designers. Stop blaming the user.

##### Share on other sites

The original question was "what is the difference between lanzos separable and non-separable".

The answer in all literature and discussions is "no visible difference", if properly implemented:

Eliminating the non-separable version would be the right thing to do.

Affinity's differences in sharpness between the two operating modes are due to misimplementation. There is no reason to keep the 10x slower non-separable version. Instead, provide a second separable preset with more lobes (more sharpness), and voila. Solved. "Lanczos" and "Lanczos (sharper)" in the menu instead of today's confusing setup. Both of those would use separable mode, but with different sharpness settings.

##### Share on other sites

Here are an example what it is called in Iridient Developer -  which also have some more options.

// Magnus

##### Share on other sites

Here are an example what it is called in Iridient Developer -  which also have some more options.

// Magnus

Plain English!

I'm not sure if I'll ever need to go to this list, but at least the waters are less muddy now.

Is there a set of pictures that demonstrate these differences?

## Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

×   Pasted as rich text.   Paste as plain text instead

Only 75 emoji are allowed.

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×

×

• #### Affinity

×
• Create New...