Thanks for writing this up and for raising the question. For starters, I don't think this is ever going to be an easy question to answer in general. Mypy, uv, pdm, and even Python itself all have at least one Claude commit at this point, so I don't think it's possible to have a dependency chain with absolutely no AI-vulnerable code, nor do I think it's the best use of energy to pursue a literal zero there.
Rather, I'd suggest that a good goal is to completely and totally eliminate AI antifeatures (AI-encumbered in the taxonomy I've been trying to popularize) while minimizing AI-vulnerable code as much as is reasonable. To that effect, my immediate gut reaction is that PoDoFo may not meet the bar at which we should eliminate it as a dependency? Should it get to that point, and I sincerely hope that it does not, your earlier suggestion of pinning to a version without AI-vulnerable code may be the best path forward. Vendoring may also be reasonable, given that PoDoFo is fairly close to a pure CMake project?
Sorry... I got distracted enough with trying to build wheels that I forgot to check socials here, that's totally on me. Anyway, I think that makes sense, yeah... testing the modules as they are post-compilation seems like a good strategy.
I've looked into Python/WASM as a possible plugin mechanism to allow either non-Python plugins, or decoupling plugins from the venv used by Calibre itself, and it definitely seems feasible? That said, some of the RS code seems to go into web contexts that may or may not yet support WASM, I don't know how well QtWebEngine plays with WASM at moment, most of what I've seen goes (weirdly enough) the other way, embedding Qt in WASM contexts.
I really like the idea, but I think I fall more towards what you note, that it's more risky at this point than TypeScript, and a higher bar to contribution.
Yeah, no, a lot of this comes back to the "what should Arcalibre be" question, but I'm increasingly getting to the point where an archival fork doesn't make as much sense as I'd have hoped. The Calibre codebase is really fragile, and in ways that make it difficult to snapshot without making significant changes. I'll say more in the other thread shortly.
In any case, yeah, TypeScript seems like a good path forward. It's a much lighter-weight and better-maintained dependency (even if it is Microsoft). There's a lot of JS code in the form of .pyj modules, such that having some compile-time guarantees makes a lot of sense.
Similarly, I like the idea of replacing modules one at a time. I've not gotten the tests down to a point where I can run them, but hopefully the existing test suite should help in porting RS → TS?
It's both, but when run as a build dependency, it's used with a custom QPA engine ("headless") that seems to have troubles lying to QtWebEngine about Vulkan support. I would guess that the rationale is that since QWE was already a runtime dependency, re-using it in a headless build context was probably reasonable at the time.
As another addendum, I'm significantly more convinced of the need to decommission RapydScript-NG after seeing that it's effectively a vanity fork by Kovid, and very likely to be modified in ways that break Arcalibre in the future. In general, I think it's a good strategy to cut dependencies on Kovid's other projects such that Arcalibre can be independently maintained and developed.
Maybe taking things kind of in synthesis, going to Loomio might make sense once things are bigger — until then, for keeping records of decisions, maybe something like recording governance decisions in issues and PRs on rereading/governance makes the most sense?
As a quick addendum, I forgot to note that the RapydScript-NG compiler currently is shipped as a partially minified blob that is then used to self-host the compiler build (as per bin/rapydscript --self). That in turn adds complexity to the task of removing the implicit dependency on opaque blobs.
Thanks for writing this up and for raising the question. For starters, I don't think this is ever going to be an easy question to answer in general. Mypy, uv, pdm, and even Python itself all have at least one Claude commit at this point, so I don't think it's possible to have a dependency chain with absolutely no AI-vulnerable code, nor do I think it's the best use of energy to pursue a literal zero there.
Rather, I'd suggest that a good goal is to completely and totally eliminate AI antifeatures (AI-encumbered in the taxonomy I've been trying to popularize) while minimizing AI-vulnerable code as much as is reasonable. To that effect, my immediate gut reaction is that PoDoFo may not meet the bar at which we should eliminate it as a dependency? Should it get to that point, and I sincerely hope that it does not, your earlier suggestion of pinning to a version without AI-vulnerable code may be the best path forward. Vendoring may also be reasonable, given that PoDoFo is fairly close to a pure CMake project?
Sorry... I got distracted enough with trying to build wheels that I forgot to check socials here, that's totally on me. Anyway, I think that makes sense, yeah... testing the modules as they are post-compilation seems like a good strategy.
Awesome, thank you so much! I started working on getting unit testing up and running in case that's helpful to an RS → TS port.
I've looked into Python/WASM as a possible plugin mechanism to allow either non-Python plugins, or decoupling plugins from the venv used by Calibre itself, and it definitely seems feasible? That said, some of the RS code seems to go into web contexts that may or may not yet support WASM, I don't know how well QtWebEngine plays with WASM at moment, most of what I've seen goes (weirdly enough) the other way, embedding Qt in WASM contexts.
I really like the idea, but I think I fall more towards what you note, that it's more risky at this point than TypeScript, and a higher bar to contribution.
Yeah, no, a lot of this comes back to the "what should Arcalibre be" question, but I'm increasingly getting to the point where an archival fork doesn't make as much sense as I'd have hoped. The Calibre codebase is really fragile, and in ways that make it difficult to snapshot without making significant changes. I'll say more in the other thread shortly.
In any case, yeah, TypeScript seems like a good path forward. It's a much lighter-weight and better-maintained dependency (even if it is Microsoft). There's a lot of JS code in the form of
.pyjmodules, such that having some compile-time guarantees makes a lot of sense.Similarly, I like the idea of replacing modules one at a time. I've not gotten the tests down to a point where I can run them, but hopefully the existing test suite should help in porting RS → TS?
It's both, but when run as a build dependency, it's used with a custom QPA engine (
"headless") that seems to have troubles lying to QtWebEngine about Vulkan support. I would guess that the rationale is that since QWE was already a runtime dependency, re-using it in a headless build context was probably reasonable at the time.As another addendum, I'm significantly more convinced of the need to decommission RapydScript-NG after seeing that it's effectively a vanity fork by Kovid, and very likely to be modified in ways that break Arcalibre in the future. In general, I think it's a good strategy to cut dependencies on Kovid's other projects such that Arcalibre can be independently maintained and developed.
https://github.com/kovidgoyal/rapydscript-ng/issues/15
Maybe taking things kind of in synthesis, going to Loomio might make sense once things are bigger — until then, for keeping records of decisions, maybe something like recording governance decisions in issues and PRs on rereading/governance makes the most sense?
Thanks for kicking this off! And yeah, I think it's a great idea to kind of brainstorm what some new features might be, especially if we decide to reposition Arcalibre as more than an archival fork.
Some initial thoughts off the top of my head:
As a quick addendum, I forgot to note that the RapydScript-NG compiler currently is shipped as a partially minified blob that is then used to self-host the compiler build (as per
bin/rapydscript --self). That in turn adds complexity to the task of removing the implicit dependency on opaque blobs.