nbconvert/pandoc is very limited in how it can be extended because it only relies on jinja templating rather than a proper AST.
You couldn't achieve a lot of the things that sphinx/jupyter book can now do with that.
Trust me I know because, before starting work on https://github.com/executablebooks/MyST-Parser I used the nbconvert/pandoc approach to create https://github.com/chrisjsewell/ipypublish ;-)
This is interesting, so do you think the JupyterBook solution could/should replace nbconvert?
I was always a bit frustrated by the way nbconvert pulled in pandoc as a dependency, but then didn't make use of any of its power (pandoc surely has a powerful AST). Instead they re-implemented filters and templates themselves. It struck me that it would have been much cleaner to write a proper 'ipynb > AST' Pandoc reader, and then expose pandoc AST filters and templates.
I was a bit suspicious of moving away from pandoc to sphinx, because sphinx seems markedly less powerful and general than pandoc. However, you seem to think it's the opposite! Does Sphinx have an AST? I can't see any reference?
pandocs AST is actually a little limited, in terms of extensability, because it has to resove to JSON. The scorreia.com/software/panflute/code.html is a good python API for it.
In fact, pandoc now does have an ipynb reader and writer. You can convert between ipynb and any of the formats pandoc supports, and you can manipulate the AST using filters.
My main concern with jupyter is that I prefer orgmode (and by a lot), but I know that it's niche and most people don't even heard about it ipnbs are preferred form of notebook exchange.
I will check how good translation to/from orgmode is then.
If the rest of the documentation workflow is in python, it would remove the need to install another entirely separate scripting language runtime to do those checks on every developer laptop, CI system, etc.
