Synchronous XHR is a really bad way to do this. The performance will be terrible for anything but the smallest module graphs.
But the TypeScript compiler API is synchronous, so there's a problem.
What you want to do is asynchronously walk the import graph, resolving import specifiers along the way with something like es-module-lexer or TypeScript's light parser, then when all the input files are collected, pass them through a compiler host to the compiler.
> But the TypeScript compiler API is synchronous, so there's a problem.
But it doesn't use DOM APIs, so you can run it in a worker without any issue. Monaco (the embeddable distribution of VSCode) does that.
The even bigger issue is that the TypeScript compiler is gigantic — like 10 MB, which is just a nonstarter for something you'd need to embed in every page of your site.
The compiler is about 730k compressed, not 10 MB. You can also lazy load it so that you show code before being able to run it.
Being in a worker only causes the synchronous fetches to not block the main thread. It's still terrible for performance as you lose all ability to load files in parallel.
It's really not that difficult to pre-traverse the import graph and fire off as many parallel fetches as the browser will allow.
Thanks, you're right, I was looking at the unminified version. Minified is 3.4 MB uncompressed, 730 KB compressed. That's still crazy considering that you could avoid it by compiling your code on the server (which can be extremely fast if you disable type-checking) rather than shoving that responsibility onto your user's browser. Might be reasonable for a big web application with megabytes of other scripts, but not for a normal website.
Oh, I wouldn't use something like this for the main code of a page. You should absolutely just compile and bundle the TS once, rather than on every page load.
But there are cases where you want to be able to run arbitrary TypeScript in the browser - in our case it was inline editable code samples - and for that running the TS compiler efficiently in a worker is great, and 730k isn't that bad. You probably also have 500kB - 1MB for a decent code editor too.
That's totally fair. The OP seems to be intended for your main page logic:
> TypeScript is still second-class citizen with regards to browser adoption, there is a proposal to fix that, but until then we have to use tooling, bundlers, build steps that are an impediment for when you want to quickly create a short demo or PoC.
(Of course, just for "a short demo or PoC," but will anyone be motivated to rip it out before that's no longer feasible?)
So I assumed you were talking about something similar. But using this approach to compile user code makes a lot of sense.
Sychronous XHR also seems a symptom here of compiling Typescript to CommonJS which as a very synchronous module system is also the wrong module system to choose for a browser application. All the modern browsers support ESM great and asynchronously load them just fine.
The compiler API has nothing to do with the module format for the compiler itself.
The issue is that the CompilerHost and LanguageServiceHost interfaces expect a synchronous filesystem API, and downstream from that the compiler internals all expect synchronous access to files.
There's a very long standing issue open to make the API async, but I'm pretty sure it's obsolete now in the face of the tsgo work.
I've tried this before, with a service worker[1] that intercepts TS/X-ish requests, and directing them over to sucrase[2] to compile to JS, before being loaded by the browser. Unfortunately sucrase seems to be no longer maintained.
If you are interested in TypeScript for the browser, you might also like --erasableSyntaxOnly option of typescript >5.8. The only tool needed for it is the typescript compiler itself, so toolchain is kept to a minimum.
I'm wondering if it would be useful/possible to run the compilation in a service worker that would intercept requests for *.ts and compile them in the worker?
I've seen an alternative approach, where the TS code is sent for compilation to a dedicated server - dismissed that idea as over engineered, but then I learned that the swc WASM package clocks in at over 5MB.
> tsbro solves this by completely bypassing the browser's import system using synchronous XHR, transpile with swc wasm and a sophisticated ESM-to-CJS transpiler so that synchronous require is used everywhere:
Neat! I've been working on a custom element for running tests in the browser, and was thinking of wiring up swc to prevent having to compile the tests from ts before running them. This library seems like it would serve better than trying to maintain something myself!
Now, I am all for it except the fact that I've heard that some people have actually recommended the wayland protocol + wasm to be a better alternative really.
And I personally feel like (I may be wrong) that at the end your proposal and the wayland proposal might be the same..
The problem with wayland protocol/your proposal is the fact that such things have already been tried (java applets) and they were insecure, and accessibility was a mess, so reverting back to it does feel like a massive chaos since javascript was created to solve that problem..
I am not a js advocate, honestly I wish that ephemeral running of apps cross platform becomes genuinely easy (in my mind nix-shell comes)
There is htmx which is nice too I guess but I think I still need some js to sprinkle in some more interactivity/animations.
Astro with htmx / islands architecture kinda feels the best, imagine using svelte/vue/react and htmx+golang in the same project..
I think what people want is the ability to write in another language while retaining first class access to the DOM without too much performance penalty, and not a boxed-in arrangement like applets.
tsc's code is mostly the type-checker, you want to look for a "transpiler" here, so embedding either swc, esbuild, sucrase or the like to handle the process of converting for you. I've never heard of one written in C++ but that may exist.
Synchronous XHR is a really bad way to do this. The performance will be terrible for anything but the smallest module graphs.
But the TypeScript compiler API is synchronous, so there's a problem.
What you want to do is asynchronously walk the import graph, resolving import specifiers along the way with something like es-module-lexer or TypeScript's light parser, then when all the input files are collected, pass them through a compiler host to the compiler.
This is what the Lit team's Playground Elements do, which compile files on a worker for embeddable interactive code samples: https://github.com/google/playground-elements
> But the TypeScript compiler API is synchronous, so there's a problem.
But it doesn't use DOM APIs, so you can run it in a worker without any issue. Monaco (the embeddable distribution of VSCode) does that.
The even bigger issue is that the TypeScript compiler is gigantic — like 10 MB, which is just a nonstarter for something you'd need to embed in every page of your site.
The compiler is about 730k compressed, not 10 MB. You can also lazy load it so that you show code before being able to run it.
Being in a worker only causes the synchronous fetches to not block the main thread. It's still terrible for performance as you lose all ability to load files in parallel.
It's really not that difficult to pre-traverse the import graph and fire off as many parallel fetches as the browser will allow.
> The compiler is about 730k compressed
Thanks, you're right, I was looking at the unminified version. Minified is 3.4 MB uncompressed, 730 KB compressed. That's still crazy considering that you could avoid it by compiling your code on the server (which can be extremely fast if you disable type-checking) rather than shoving that responsibility onto your user's browser. Might be reasonable for a big web application with megabytes of other scripts, but not for a normal website.
Oh, I wouldn't use something like this for the main code of a page. You should absolutely just compile and bundle the TS once, rather than on every page load.
But there are cases where you want to be able to run arbitrary TypeScript in the browser - in our case it was inline editable code samples - and for that running the TS compiler efficiently in a worker is great, and 730k isn't that bad. You probably also have 500kB - 1MB for a decent code editor too.
That's totally fair. The OP seems to be intended for your main page logic:
> TypeScript is still second-class citizen with regards to browser adoption, there is a proposal to fix that, but until then we have to use tooling, bundlers, build steps that are an impediment for when you want to quickly create a short demo or PoC.
(Of course, just for "a short demo or PoC," but will anyone be motivated to rip it out before that's no longer feasible?)
So I assumed you were talking about something similar. But using this approach to compile user code makes a lot of sense.
Read the last paragraph
Sychronous XHR also seems a symptom here of compiling Typescript to CommonJS which as a very synchronous module system is also the wrong module system to choose for a browser application. All the modern browsers support ESM great and asynchronously load them just fine.
The compiler API has nothing to do with the module format for the compiler itself.
The issue is that the CompilerHost and LanguageServiceHost interfaces expect a synchronous filesystem API, and downstream from that the compiler internals all expect synchronous access to files.
There's a very long standing issue open to make the API async, but I'm pretty sure it's obsolete now in the face of the tsgo work.
I've tried this before, with a service worker[1] that intercepts TS/X-ish requests, and directing them over to sucrase[2] to compile to JS, before being loaded by the browser. Unfortunately sucrase seems to be no longer maintained.
[1]: https://github.com/altbdoor/sucrase-build-iife
[2]: https://github.com/alangpierce/sucrase
If you are interested in TypeScript for the browser, you might also like --erasableSyntaxOnly option of typescript >5.8. The only tool needed for it is the typescript compiler itself, so toolchain is kept to a minimum.
I'm wondering if it would be useful/possible to run the compilation in a service worker that would intercept requests for *.ts and compile them in the worker?
I've seen an alternative approach, where the TS code is sent for compilation to a dedicated server - dismissed that idea as over engineered, but then I learned that the swc WASM package clocks in at over 5MB.
I love the name BTW.
That is how this works.
> tsbro solves this by completely bypassing the browser's import system using synchronous XHR, transpile with swc wasm and a sophisticated ESM-to-CJS transpiler so that synchronous require is used everywhere:
Yes, but it's doing all that locally. When I ran the code it downloaded a 5MB file, namely https://esm.sh/@swc/wasm-web@1.13.2/wasm_bg.wasm.
It's also not using a service worker.
Neat! I've been working on a custom element for running tests in the browser, and was thinking of wiring up swc to prevent having to compile the tests from ts before running them. This library seems like it would serve better than trying to maintain something myself!
I usually feel more concern whenever more features get stuffed into the browser.
It is an accumulation of complexity that, for backwards compatibility, we get stuck with.
The browser should be simple.
If the focus just stayed on making wasm better for web development, folks can use any language they want and the API surface area can stay small.
Now, I am all for it except the fact that I've heard that some people have actually recommended the wayland protocol + wasm to be a better alternative really.
And I personally feel like (I may be wrong) that at the end your proposal and the wayland proposal might be the same..
The problem with wayland protocol/your proposal is the fact that such things have already been tried (java applets) and they were insecure, and accessibility was a mess, so reverting back to it does feel like a massive chaos since javascript was created to solve that problem..
I am not a js advocate, honestly I wish that ephemeral running of apps cross platform becomes genuinely easy (in my mind nix-shell comes) There is htmx which is nice too I guess but I think I still need some js to sprinkle in some more interactivity/animations.
Astro with htmx / islands architecture kinda feels the best, imagine using svelte/vue/react and htmx+golang in the same project..
I think what people want is the ability to write in another language while retaining first class access to the DOM without too much performance penalty, and not a boxed-in arrangement like applets.
Is there a solution to get ts compiler embeddable into c++ project that uses v8, so it could compile the code on the fly?
Seems tsc itself requires node, but surely an api that takes a ts file as a string and returns a ts file as a string should be possible?
tsc's code is mostly the type-checker, you want to look for a "transpiler" here, so embedding either swc, esbuild, sucrase or the like to handle the process of converting for you. I've never heard of one written in C++ but that may exist.
swc exists as wasm which any browser should be able to execute.
Perhaps you could use or build a C interface for swc, a TS compiler written in rust.
they could use swc-wasm-web as tsbro also uses swc-wasm-web
Hot damn
How does this differ from esm.sh/tsx?
It doesn't send source code to esm.sh, but does it locally in the browser