Scripting

[CM0]

I recently made a post in this thread about some scripting capabilities that would be ideal for my use cases. Essentially a live scripting layer to combine the capabilities of procedural textures and equations etc. Detail in link.

https://forum.affinity.serif.com/index.php?/topic/64885-scripting/page/20/#comment-1086758

However, it recently also occurred to me that there seems to be a potential capability that hasn't been utilized in any image editor that would also essentially provide this capability, but with many advantages. That would be to use GLSL fragment shader scripts. I believe you would be able to do all the things requested above, but with far better performance and entire libraries of effects already available as scripts across the web.

Just curious if this has been considered and what are your thoughts? Seems Affinity might could be the first to offer the capability. This might not fit into the first feature set of core scripting, but hopefully something for future consideration that might be a nice unique feature.

[Jon W]

17 hours ago, rui_mac said:

In other applications I use, that allow for scripting (mainly, Cinema 4D and VizRT), besides being able to create scripts that can be executed from a menu listing (Cinema 4D, only), I can also attach a kind of a "tag" to an object that runs scripting code in real time, whenever the scene/document is changed/refreshed.
That way, I can perform scripting operations on the object that the "tag" is attached and also on its children.
Will Affinity scripting abilities only allow for scripts that can be invoked from a menu or will we have a new type of "tag" to attach to our "layers/objects" that can allow us to add abilities/behaviours to any "layer/object"?

We have no plans for layer/object-specific scripts. However, scripts will be able to run in the background and respond to events such as "document changed". 

Could you perhaps give an example of how such a layer/object-specific script would be used in practice?

[Jon W]

11 hours ago, CM0 said:

I recently made a post in this thread about some scripting capabilities that would be ideal for my use cases. Essentially a live scripting layer to combine the capabilities of procedural textures and equations etc. Detail in link.

https://forum.affinity.serif.com/index.php?/topic/64885-scripting/page/20/#comment-1086758

However, it recently also occurred to me that there seems to be a potential capability that hasn't been utilized in any image editor that would also essentially provide this capability, but with many advantages. That would be to use GLSL fragment shader scripts. I believe you would be able to do all the things requested above, but with far better performance and entire libraries of effects already available as scripts across the web.

Just curious if this has been considered and what are your thoughts? Seems Affinity might could be the first to offer the capability. This might not fit into the first feature set of core scripting, but hopefully something for future consideration that might be a nice unique feature.

We are planning to allow custom live filters at some point. It would be possible to write one that used GLSL.

[rui_mac]

4 hours ago, Jon W said:

We have no plans for layer/object-specific scripts. However, scripts will be able to run in the background and respond to events such as "document changed". 

Could you perhaps give an example of how such a layer/object-specific script would be used in practice?

Lets imagine that you attach a script to a layer and that script would keep all objects inside that layer dynamically aligned/distributed, having the position and size of the first and last elements of that layer as the limits.
Or having a script attached to an object, or layer that would automatically create a grid of duplicates of whatever object was dragged into it.
Or a script that would keep an object centered between two other objects, dynamically.
Or a script that would create a specific amount of interpolated shapes between the first two childs of a layer, that would update dynamically as any of the first two children of the layer was edited.
Or a script attached to an object that would distribute clones of another object along the spline of that object. And the objects would dynamically redistribute even when the main curve is edited.
Being able to attach a script to objects allows for many, many possibilities and, all of them, dynamic, instead of having to run a script manually whenever a change is made.

[fde101]

24 minutes ago, rui_mac said:

Lets imagine that you attach a script to a layer and that script would keep all objects inside that layer dynamically aligned/distributed, having the position and size of the first and last elements of that layer as the limits.

You mean like this (done in Designer without any need for a script)?

 

 

[Bryan Rieger]

6 hours ago, Jon W said:

We have no plans for layer/object-specific scripts. However, scripts will be able to run in the background and respond to events such as "document changed". 

Could you perhaps give an example of how such a layer/object-specific script would be used in practice?

I could see layer scripts (or a new layer/group type that runs a script—or a collection of scripts—on all contained objects) being helpful in creating complex behaviours such as grids, constraints, replicators, collisions, distributions, offsets, etc in much the way that Cavalry uses behaviours, VectorStyler uses repeaters and Astute Graphics plug-ins such as Stylism, Randomino, ColliderScribe, and Phantasm enable a vast array of possible live object manipulations. Being able to have layer scripts/behaviour expose properties that users could then visually manipulate (sliders, fields, toggles, etc) to achieve their desired results (without jumping into code) could also create a community/market for these behaviours, in much the way GLSL shader effects support could.

[v_kyr]

59 minutes ago, rui_mac said:

...Being able to attach a script to objects allows for many, many possibilities and, all of them, dynamic, instead of having to run a script manually whenever a change is made.

The question in turn here is then, how to deal the best way with that in a document persistent manner, aka in which way will the scripts need to be distributed together with a doc then (embedding/linking)? - And in case of linking what happens and how to prevent missing script and possible doc file load failures then, as we often see with possible missing image resources so far?

[rui_mac]

26 minutes ago, fde101 said:

You mean like this (done in Designer without any need for a script)?

 

Screen Recording 2023-06-05 at 09.03.29.mov 532.17 kB · 1 download  

 

Yes, I know that simple arranjements of one object between two other object is possible without scripting. But what if there are more objects?
Or if I want  to scale objects (let us imagine that there are 10 objects between the first and last object) so that they will always touch each other?

[rui_mac]

1 minute ago, v_kyr said:

The question in turn here is then, how to deal best with that in a document persistent manner, aka in which way will the scripts need to be distributed together with a doc then (embedding/linking)? - And in case of linking what happens and how to prevent missing script and possible doc file load failures then, as we often see with missing image resources so far?

The scripts that would be attached to objects, would be saved within the file. At least that is what Cinema 4D and VizRT do.

[Bryan Rieger]

2 hours ago, v_kyr said:

The question in turn here is then, how to deal best with that in a document persistent manner, aka in which way will the scripts need to be distributed together with a doc then (embedding/linking)? - And in case of linking what happens and how to prevent missing script and possible doc file load failures then, as we often see with missing image resources so far?

I think these would need to be handled much like any other resource; embedded in the file, distributed along side as a linked resource, or (perhaps) even accessed via an app/suite wide library (ie. assets, brushes, etc). If a script/behaviour is missing the user is notified and the layer/group is rendered in a manner that communicates the problem (ie. layer is drawn with a checkerboard pattern and a missing script icon), along with means to update, relink or replace the missing resource. This 'unable to render' state could also be useful in communicating scripts/behaviours with errors that prevent their execution from completing and thus block rendering.

[v_kyr]

39 minutes ago, rui_mac said:

The scripts that would be attached to objects, would be saved within the file. At least that is what Cinema 4D and VizRT do.

How do they handle other in scripts possibly defined external resources if those are missing on a system?

 

38 minutes ago, rgba said:

I think these would need to be handled much like any other resource; embedded in the file, distributed along side as a linked resource, or (perhaps) even accessed via an app/suite wide library (ie. assets, brushes, etc).

The later is IMHO not that much practicle here, importing/exporting scripts from a general app repository (ie. like assets, brushes, etc are handled), as that needs some more efford in terms of concrete script identification via some unique ID handling.

46 minutes ago, rgba said:

If a script/behaviour is missing the user is notified and the layer/group is rendered in a manner that communicates the problem (ie. layer is drawn with a checkerboard pattern and a missing script icon), along with means to update, relink or replace the missing resource. This 'unable to render' state could also be useful in communicating scripts/behaviours with errors that prevent their execution to complete without throwing errors, hence blocking rendering.

As scripts (hopefully) will have the capability to reuse/call other scripts & external resources too, some foolprove execution error handling management will be either way essential here. - Though not sure here how your last sentence is meant here, aka "...with errors that prevent their execution to complete without throwing errors", since if no error are thrown anywhere, the rendering engine still has to determine somehow that rendering is not possible and tell what and where the overall cause of that state is.

[rui_mac]

6 minutes ago, v_kyr said:

How do they handle other in scripts possibly defined external resources if those are missing on a system?

All the code is self contained inside the "tag". Meaning, the script within the "tag" that is attached to the object will not call other scripts that are coded and show up in the menu. They are separate things, even if they use the same language. The scripts that show up in the menu, are almost like any other command that we can choose from the menus.
The scripts in the "tags" are scripts that are executed every time the document changes or gets updated.

[Bryan Rieger]

54 minutes ago, v_kyr said:

As scripts (hopefully) will have the capability to reuse/call other scripts & external resources too, some foolprove execution error handling management will be either way essential here. - Though not sure here how your last sentence is meant here, aka "...with errors that prevent their execution to complete without throwing errors", since if no error are thrown anywhere, the rendering engine still has to determine somehow that rendering is not possible and tell what and where the overall cause of that state is.

Sorry, poorly worded sentence. If a script cannot complete it will of course throw an error communicating this. If rendering is blocked by the error(s) then there should be some visual way to indicate that the layer cannot be rendered. After Effects does something similar with footage that cannot be found within the composition.

[David Cake]

On 5/12/2023 at 9:03 PM, fde101 said:

All code is ultimately interpreted at some level.  When it is interpreted by hardware we call it "native" and when it is interpreted by software we call it "interpreted" or "emulated" but in the end it is being interpreted at some level.

I don’t understand why you put so much effort into these arguments that for all practical discussion of the topic at hand seem deliberately obfuscatory. 

Yes, Rosetta and similar technologies exist. Yes, JIT compilation makes the distinction between interpretation and compilation less clear, especially when incorporated into language design (eg Julia), yes internal intermediate byte code format exists in some compiled languages, yes some virtual machines byte codes have been hardware implementations. But none of this is very relevant to discussion of why you would want either an interpreted scripting language or a compiled binary API as an extension language for a native application, and most is theoretical at best, positing whole technologies that would need to be created. 

[Bryan Rieger]

44 minutes ago, v_kyr said:

The later is IMHO not that much practicle here, importing/exporting scripts from a general app repository (ie. like assets, brushes, etc are handled), as that needs some more efford in terms of concrete script identification via some unique ID handling.

Probably, but it depends on the implementation. If a general script library is used then some sort of package manager and id (hash, url, git repo, com.domain.script, etc) will be required. There's lots of prior art for inspiration here.

[v_kyr]

49 minutes ago, rui_mac said:

All the code is self contained inside the "tag". Meaning, the script within the "tag" that is attached to the object will not call other scripts that are coded and show up in the menu. They are separate things, even if they use the same language. ...

AFAI have seen here, Cinema 4D uses Python for scripting, thus they are at least be able to load/import other own & third party modules, which then can offer additional fuctionality here. So they already offer and can make use of additional imported Python modul scripts!

38 minutes ago, rgba said:

Sorry, poorly worded sentence. If a script cannot complete it will of course throw and error communicating this. If rendering is blocked by the error(s) then there should be some visual way to indicate that the layer cannot be rendered. After Effects does something similar with footage that cannot be found within the composition.

Ah Ok now I catched what you mean. - Yes that should be possible/doable too here then for Affinity scripting, as far as that layer based script execution access will be implemented by the scripting dev team. Further Adobe After Effects does use JS scripting too.

[rui_mac]

This is just a very simple example that I coded now in Cinema 4D. The script is attached to the top object, called "Null". This would be the equivalent of a layer or another object in Affinity. Then, no matter how many objects are set as children of the top object, the script always distributes them evenly between the position of the first and last child. As you all can see in the movie, I can add as many objects I want, and they will always distribute evenly. And it is fully dynamic.

[Petar Petrenko]

After long and in-depth analyzing of this thread I came to the conclusion that the final demand is:

WE WANT AFFINITY TO ALLOW US TO RE-CODE ALL THE APPS,
SO NO ONE CAN RECOGNIZE THE ORIGINAL.
NOTHING MORE, NOTHING LESS.

[v_kyr]

24 minutes ago, rui_mac said:

This is just a very simple example that I coded now in Cinema 4D. The script is attached to the top object, called "Null". This would be the equivalent of a layer or another object in Affinity. ...

Looks good and I personally like the fact that they use Python as an scripting language for C4D. - Your shown sample just imports the "c4d" module, I assume you can import other Python modules if needed/wanted there too, so also make use of your own written modules then and thus can enhance all as individually needed.

And as I can also see, you do perform the whole as far as there are more than two child objects in that object group via a precheck, though there radius1 + radius2 are never used and thus the two expressions can be removed or commented out. 😉

[rui_mac]

4 minutes ago, v_kyr said:

Looks good and I personally like the fact that they use Python as an scripting language for C4D. - Your shown sample just imports the "c4d" module, I assume you can import other Python modules if needed/wanted there too, so also make use of your own written modules then and thus can enhance all as individually needed.

And as I can also see, you do perform the whole as far as there are more than two child objects in that object group via a precheck, though there radius1 + radius2 are never used and thus the two expressions can be removed or commented out. 😉

Yes, you are right in all accounts.
The radius1+radius2 was to distribute between the outer perimeters of the first and last disks... I started the code for that, but gave up as this was only a simple example and I decided that I could simply interpolate between the center positions 😉

The c4d library contains all the internal C4D librarys. I can import just the ones I want, instead of importing the whole thing. And, of course, I can import all python libraries, if I need something specific.

[kimtorch]

8 hours ago, David Cake said:

I don’t understand why you put so much effort into these arguments that for all practical discussion of the topic at hand seem deliberately obfuscatory. 

I agree, I'd much rather see examples of people's current scripting rather than trying to redesign or expand something which doesn't yet exist. If people would like to speculate on how a scripting engine could be called or which language it should use then they're free to start a new topic or write directly to Affinity. The entire purpose of this thread (which I started) was to enquire if Affinity were going to support scripting and what people are currently using which they would need to 'change teams'.

It's frustrating enough having waited this long for a glimmer of hope to see constant divergence to obscure topics and references. As I've said previously, it's fine to want 'everything' but it means it's never released. Tell them what's important - ie: what you're currently doing by script - then by all means submit your dreams once the rest of us are quietly getting their work done.

[CM0]

On 6/5/2023 at 5:50 AM, Jon W said:

We are planning to allow custom live filters at some point. It would be possible to write one that used GLSL.

Nice, that is fantastic! Really excited for these new capabilities. GLSL will make for some impressive demo reels of Affinity new features when the time comes.

[Bryan Rieger]

16 hours ago, kimtorch said:

…The entire purpose of this thread (which I started) was to enquire if Affinity were going to support scripting and what people are currently using which they would need to 'change teams'.

I think one of the problems here is how each of us interprets 'scripting' today.

In the early days of InDesign, and when Quark ruled the DTP world, scripting was usually thought of in relation to automation; enabling a script to complete a series of mundane, repetitive tasks that enabled users to focus on more high-value work. In the years since, the concept of 'scripting' has morphed considerably. While we still create scripts to automate repetitive tasks, we now also create scripts to create new tools, workflow and functionality that extend our existing tools, enabling us to perform complex tasks that would have taken hours or days, in seconds and minutes. Going beyond simply creating new tools and functionality (Cinema 4D, Illustrator, etc), scripts can also be utilized to create entirely new generative works (Processing Blender, Rhino, etc).

From here it's not a huge leap to see AI as the evolution of scripting. We use it today to perform tedious, repetitive tasks, (object selection, masking, etc) and to create new tools (for writing, style transfer, content-aware fill, etc) as well as to create entirely new works (Firefly, etc). Even the way we create 'scripts' is changing. While we still write scripts in various languages, we also now have tools such as Apple's Shortcuts which employ a Scratch 'block-like' interface, or tools such as Unity, Unreal, etc which also provide visual node-based/flow-control scripting tools.

I think that while focusing on matching existing scripting support from other workflows is helpful in the short-term, it could quickly lead to a situation where user expectations have moved far beyond what was considered state-of-the-art just a few years ago, and risk leaving the Affinity suite feeling further dated, and increasingly irrelevant.

In the wise words of a fellow Canadian “…skate to where the puck is going to be, not where it has been”.

[Jon W]

On 6/5/2023 at 4:46 PM, rui_mac said:

This is just a very simple example that I coded now in Cinema 4D. The script is attached to the top object, called "Null". This would be the equivalent of a layer or another object in Affinity. Then, no matter how many objects are set as children of the top object, the script always distributes them evenly between the position of the first and last child. As you all can see in the movie, I can add as many objects I want, and they will always distribute evenly. And it is fully dynamic.

Thanks for the explanation. Whilst dynamic layout is certainly an interesting idea, it's somewhat outside the scope of scripting itself - it would have to be considered as a completely new feature. Although we have no plans to allow general scripts to be embedded in documents (primarily due to security issues), if we were to add such a "dynamic layout" feature then I imagine we would use embedded scripts (with limited, layout-specific capabilities). 

[Jon W]

Thanks everyone for all your suggestions - please keep them coming! They are all useful. 

Initially, we'll be concentrating on exposing existing functionality - primarily, manipulation of documents and their contents. 

Then later we will be able to move on to some of the "new feature" things that have been suggested: custom tools / shapes / brush engines / personas etc.