Scripting

[debraspicher]

@kimtorch I would say @Seneca's post is more relevant to your discussion and points there. People get touchy about criticism on these boards, but you're entitled to give feedback just like the others. Serif's goal on these boards seems to now be to make the process more open to taking in feedback during development, so I'm sure that your discussion is already welcome here. To your point, I would not be surprised if they are not already looking to that large of a scope given the amount of talk over missing functionality/lost productivity. Scripting would certainly help to fill in some of these gaps until the same features are present in the vanilla apps.

The availability/readability of scripts also give them a much better idea over how users are actually using their programs and can inform their design decisions overtime. Notably it would help show which features are important, or are not as important, in the grand scheme. An open system would easily repay itself in time, imv.

[v_kyr]

7 minutes ago, kimtorch said:

Given I was the one who started this discussion almost 5 years ago - and explicitly mentioned things like PDF creation, tagged text, importing into documents etc - I think it's reasonable for me to have an opinion on scripting capabilities.

I hope they do bring scripting capabilities to Affinity Photo, but with over 40 years experience in the printing and publishing industry, I have a very good idea of where priorities for automation should lie.

So you saw at least that they now seem to have a scripting core and are continously working on a JS API. The later will have to address APub of course in the same manner as ADe and APh (...as the apps and the file format share either way internally most of the functionality, even if not everything is from every app accessable). Further text handling and it's formating is mandantory for APub and if they have and keep some JS based fs module, then one should hopefully be able to do own I/O based scripting too!

 

 

[Seneca]

On 3/24/2023 at 4:32 PM, Tim France said:

We've got the callback version of read, write etc. too.  We aren't going to be node, nor are we going to try and be a web browser, but we will likely draw inspiration from both.

This is great.

We all know how quickly this JS landscape is shifting under our feet.

First we've had Node, then Deno and now a new kid on the block Bun, written in Zig language, extremely fast.

 

On 3/24/2023 at 2:05 PM, Tim France said:

I thought I'd give you a quick progress update and let you see some of things we've been working on at Serif Labs.

Having quickly read some of the code samples shown in the video my conclusion is as follows (I may be completely wrong here, of course):

1. We will have a Macro Recording facility - brilliant.

2. We will have Scripting Facility - brilliant.

3. Not shown and mentioned elsewhere, we will have a C based APIs - brilliant.

One thing I'm not clear about is whether we will be able to write our scripts in TypeScript as well as in JavaScript.

In theory this should be no problem as TypeScript is a superset of JavaScript. To be able to give scripts a .ts extension and run it as well .js files would be a big bonus in my humble opinion. Basically what I am hoping for is to have an environment that would transpile TS to JS automatically. For larger scripts and for those who want to go the TypeScript way this would be really great.

I'm sure the team at Affinity has all this covered.

[v_kyr]

8 hours ago, Seneca said:

First we've had Node, then Deno and now a new kid on the block Bun, written in Zig language, extremely fast.

• Framework Bun: "Faster than anyone would believe" – Interview with Jarred Sumner

[Tim France]

 

15 hours ago, kimtorch said:

They're far better off getting feedback before they're fully committed to a wrong direction.

Couldn't agree more, that's one of the reasons we've put these examples up.  However I should re-emphasise that the examples don't represent everything that's possible with scripting, even at this early stage.  It's ok to find the examples underwhelming:

23 hours ago, Tim France said:

I'm not claiming any of these examples are particularly useful on their own; they're just to demonstrate some basic scripting functionality.

Note: "just to demonstrate" and "basic"  But like I said, constructive feedback is welcomed, just don't worry that this is all you're going to get.

16 hours ago, Seneca said:

And secondly, as far as I remember the team is also working on a C based extension system. For companies that need greater automation this plug-in system might be more suitable.

This next bit is mainly for any devs who might be reading this.

The Javascript API is built entirely on top of the C API.  Actually, it's built on top of a header-only C++ SDK that minimally wraps the C API so we don't have to worry about explicitly deallocating any handles that might be used.  We've not cheated and called directly into the Affinity backend from the Javascript layer, it's all going through the C++ wrappers and therefore through the public C API functions.  This is great for testing, as tests written in Javascript exercise all abstraction layers.  We will likely release the C++ headers along with the C headers.

There is a low-level Javascript API that is as similar to the C API as possible, which has the benefit of reducing the amount of documentation we have to write.  There are some differences between function signatures, but essentially they are the same, with each C function "projected" into Javascript land.  We then provide a set of JS files that wrap those exposed JS functions into ES6 classes and hide a lot of the boring boiler-plate that no-one cares about.

For maximum performance, you'd use the C or C++ layer.  For convenience, you'd use the high level JS layer, which is what the examples use. For Javascript consumers who want to design a different OOP layer, they are free to require() the API functions directly and wrap them up however they like.  I would say that as a dev coming from a mainly C/C++ background, I was surprised at how performant the Javascript levels have turned out to be.

 

[CM0]

6 minutes ago, Tim France said:

For maximum performance, you'd use the C or C++ layer.  For convenience, you'd use the high level JS layer, which is what the examples use. For Javascript consumers who want to design a different OOP layer, they are free to require() the API functions directly and wrap them up however they like.  I would say that as a dev coming from a mainly C/C++ background, I was surprised at how performant the Javascript levels have turned out to be.

 

Yes, this is great. Javascript performance has improved to the point some serious professional software can now be built on that foundation. The most popular code editor VSCode for instance.

Can you comment on the Javascript engine? Does it support WASM? as that would be another opportunity for very high nearly C level performance through the JavaScript engine?

[Seneca]

11 minutes ago, CM0 said:

Can you comment on the Javascript engine? Does it support WASM? as that would be another opportunity for very high nearly C level performance through the JavaScript engine?

Hello @CM0,

I know this question is for Tim to answer, but my 2-cents are that we don't really need Wasm, since we will have C-based APIs. If we didn't have that, Wasm would be something worth considering. Also, as Tim says, the JavaScript performance is so good that, to have another layer based on Wasm would unnecessarily bloat the app.

[CM0]

9 minutes ago, Seneca said:

Hello @CM0,

I know this question is for Tim to answer, but my 2-cents are that we don't really need Wasm, since we will have C-based APIs. If we didn't have that, Wasm would be something worth considering. Also, as Tim says, the JavaScript performance is so good that, to have another layer based on Wasm would unnecessarily bloat the app.

But we get WASM for free if it is in the JavaScript engine. It is widely supported. It is also portable, so no compiling for specific architectures and opens up the use of any language such as high performance RUST. 

[Seneca]

15 minutes ago, CM0 said:

But we get WASM for free if it is in the JavaScript engine. It is widely supported. It is also portable, so no compiling for specific architectures and opens up the use of any language such as high performance RUST. 

Opening up the API to different language through WASM would definitely be very interesting indeed. 🙂

EDIT: On the other hand the C based APIs will allow other language to plug into that and it doesn't have to be through WASM.

 

[Aisthetikos]

I love plugins, but unfortunately I have tragic experiences with them not being updated (discontinued) or the companies behind them closing down. And that you then have to say goodbye to the tools again when they no longer work with, for example, Affinity. It's serious when it's serious functionality in your serious workflows.

So I hope for plugins that can provide extra functionality, e.g. for companies with special needs, who on their own maintain their own scripts for their own workflows and needs, but that Serif itself builds necessary functionality for everyone into the programs and maintains it continuously.

For me, this is far more important than the technical platform.

But crazy good news - that gives better opportunities for everyone. Just with the above in mind.

[v_kyr]

21 hours ago, Aisthetikos said:

I love plugins, but unfortunately I have tragic experiences with them not being updated (discontinued) or the companies behind them closing down. And that you then have to say goodbye to the tools again when they no longer work with, for example, Affinity.

Not sure if you meant here more overall related to Affinity, but it's all of not much surprise. Developing Plugins in order to add/offer some additionally functionality to a specific software is a time consuming process (which costs time & money) and depending on the sort of Plugin, the question is always also how much people will make use of it, or buy that then and if it's overall worth the effords. Add to that the fact that most target system also evolve and may change interfaces etc. over time, so an individual Plugin has to be also changed/updated and adapted to this situation continiously.

In case of Affinity (...to be more specific APh and it's plugin mechanism is meant here) it actually relies entirely on Adobe's PS API, but does functional wise only support a minimal subset of that. So the majority of the Plugins build with that are first of all meant for Adobe PS and I personally don't know of any one (Plugin) which is primarily/explicitely only build-up for Affinity's usage here. - Further Affinity's usage for the Adobe PS API parts are entirely undocumented (no guide, no examples etc.) from Serif's side here, meaning you have to rely on Adobe docs and examples for PS and so have to find out by yourself what from all that is applicable at all for APh. - So the weak point here lies entirely on Serif's side and most third party developers actually won't spend their scarse time to deal with Affinity's shortcomings here!

22 hours ago, Aisthetikos said:

So I hope for plugins that can provide extra functionality...    ... For me, this is far more important than the technical platform.

Without a good and well build "technical platform" and also a good documentation (... API overview, guides, usage examples, ... etc.) of that, you won't get the rest. It's very closely related and one cannot be achieved without the other!

[debraspicher]

@Aisthetikos Yes, sadly, it's inevitable scripts break overtime. Especially programs that are undergoing so many changes all at once. Affinity does have one major advantage in its pockets it seems that the foundation of all 3 programs are built more or less the same, share a similar code base, hence being able to use the same functionality from one program to the next. This means (hopefully) that script aka extension developers really only have to target one code engine. So that makes updating scripts, also reusing/repurposing them, so much simpler than having to work within the confines of multiple programs. Anyway, I always recommend to someone who relies on scripts themselves that they pick up some script writing capabilities themselves, jic.

[chradja]

..

Edited March 26, 2023 by chradja
Wrong quote

[chradja]

On 3/25/2023 at 4:12 PM, CM0 said:

But we get WASM for free if it is in the JavaScript engine. It is widely supported. It is also portable, so no compiling for specific architectures and opens up the use of any language such as high performance RUST. 

WASM is a completly different engine than javascript. Although it is capable of executing javascript code in a sandbox.

[CM0]

11 minutes ago, chradja said:

WASM is a completly different engine than javascript. Although it is capable of executing javascript code in a sandbox.

WASM is built directly into most modern JavaScript engines. V8 for example which is the engine used for VSCode editor also supports WASM.

https://www.infoq.com/news/2021/03/v8-9-webassemly/

[v_kyr]

3 hours ago, CM0 said:

... V8 for example which is the engine used for VSCode editor also supports WASM. ...

You better link to the V8 project page of Google’s open source high-performance JavaScript and WebAssembly V8 engine, since that comes along with much more actual informations about it's WASM integration and current state. - Further in order to make the assembly language for execution in web browsers, also accessible outside of the web, there's the "WebAssembly System Interface" (WASI). So WebAssembly can also be used outside of browsers via the WASI interface. The wasmtime command (see also it's GitHub home) then starts a stand-alone runtime environment for WebAssembly. The runtime environment for WebAssembly can also be embedded in different languages and there is also a micro runtime with reduced memory usage for that.

However even all interesting, especially here from the performance side of view, the whole might possibly then add too much complexity, additional needed diskspace and also an additional bunch of mem overhead (... not to forget some more third party tool dependencies then too) for an overall maybe by Serif simpler planned Affinity JS based scripting implementation. - So let's wait and see what and how they will finally implement it and whether it then roughly meets our expectations.

 

[CM0]

55 minutes ago, v_kyr said:

e "WebAssembly System Interface" (WASI). So WebAssembly can also be used outside of browsers via the WASI interface.

V8 can execute webassembly modules, I'm not suggesting a standalone WASM. However, using V8 would mean that JS plugins could be written that use WASM modules for performance critical tasks.

[v_kyr]

9 minutes ago, CM0 said:

V8 can execute webassembly modules

I know, why do you think I wrote "... high-performance JavaScript and WebAssembly V8 engine ... about it's WASM integration and current state" and referenced to it's homepage. - WASI is interesting for hooking on via other languages than JS to WASM here.

[CM0]

13 minutes ago, v_kyr said:

I know, why do you think I wrote "... high-performance JavaScript and WebAssembly V8 engine ... about it's WASM integration and current state" and referenced to it's homepage. - WASI is interesting for hooking on via other languages than JS to WASM here.

ok, np. Just how it followed in context was not clear. thanks.

[fde101]

On 3/25/2023 at 10:37 AM, Tim France said:

The Javascript API is built entirely on top of the C API.  Actually, it's built on top of a header-only C++ SDK that minimally wraps the C API so we don't have to worry about explicitly deallocating any handles that might be used.  We've not cheated and called directly into the Affinity backend from the Javascript layer, it's all going through the C++ wrappers and therefore through the public C API functions.

Great... in this case, please make the JavaScript support a plugin itself, in such a way that developers can make similar plugins for other scripting languages.

[Tim France]

On 3/25/2023 at 2:47 PM, CM0 said:

Can you comment on the Javascript engine?

Sorry, for a few reasons that's not something I can get into at this stage.

On 3/24/2023 at 10:01 PM, Frozen Death Knight said:

Also, will switching modes be supported as well?

Similar to StudioLink, Photo-specific scripting functionality will be available from any product as long as you have a Photo license enabled.

[Floor]

I'm very excited about scripting support, can't wait to play with it! The previews look promising.

If there is one thing I could request it would be the ability to run/trigger a script from outside of the Affinity app UI. For example I'm the developer of a font manager app for Mac, and it would be great to be able to adjust fonts and text properties from within the external app. Currently using the macOS Accessibility api to do something similar, but with scripting it can be made much more user friendly.

Sketch includes a `sketchtool` binary that can run any plugin and pass some context (for example json). It's fine if the user would need to explicitly 'install' a plugin for security, but it would be nice if such a plugin/script can be triggered without any further user interaction. Another option would be an Apple Script handler that can kickstart a plugin/script and pass context, but that's not cross platform of course.

[Seneca]

6 hours ago, Floor said:

If there is one thing I could request it would be the ability to run/trigger a script from outside of the Affinity app UI. For example I'm the developer of a font manager app for Mac, and it would be great to be able to adjust fonts and text properties from within the external app. Currently using the macOS Accessibility api to do something similar, but with scripting it can be made much more user friendly.

I asked for a similar question earlier.

Here's the answer given by TonyB:

On 5/9/2022 at 2:47 PM, TonyB said:

On 5/9/2022 at 2:39 PM, Seneca said:

Thank @TonyB for that information.

One thing that I'd love to know, if possible, is how you are going to deal with automation which remains platform specific.

Apple during the last Year WWDC announced that the future of automation on MacOSX is Shortcuts (replacing AppleScript). I have no doubts that on this year's WWDC, just next month, we will see a bigger push towards that.

Windows, no doubt, have their own technology. 

Are there any plans to take advantage or each platform unique automation technologies?

Expand  

We will support a very limited set of platform specific automation functions with one of those exposed functions being able to pass a Javascript snippet that will then have full access to Affinity. 

Hope that makes sense.

[symbiont]

i don't understand the usefulness of this demo. where are the scripts that can control properties of the tools that are already in the programs. like accessing gradient, lines, color values, layers, etc. i'm looking forward to scripts that can automate a lot of the repetitiveness, rather then image generation to some 80's looking fractals. this is not looking good or hopeful.

however after seeing how layers can be renamed in a later post, i'll just wait and see what the team will showcase what scripting can do in some future update. is scripting coming to version 2?

[Seneca]

23 minutes ago, symbiont said:

i'm looking forward to scripts that can automate a lot of the repetitiveness, rather then image generation to some 80's looking fractals.

I would respectfully suggest to read (again) page 17 of this thread before jumping into any conclusions.