Vite Integration

[dillonkearns]

What and Why

I've previously talked about my goal to have elm-pages be lean and unopinionated regarding building and bundling non-Elm assets: #148. elm-pages used to be built on Webpack, and I didn't want to expose any webpack internals because:

1. I didn't want to couple elm-pages to Webpack, and
2. I didn't want to expose internals with caveats where the user would have to take care not to break internals

So the goal was to be opinionated about how to build and optimize an Elm application (to consider that a core responsibility of elm-pages), and then let the user deal with bundling and optimizing the rest of their assets.

I think this is the right overall direction, but Vite has changed things in a couple of important ways:

1. Vite has an approach of using standards to process files in an intuitive way. For example, you don't need a special configuration for TypeScript, it will process TypeScript out of the box by just stripping out TS-specific syntax to transpile it quickly. You don't have to define a complex config because it has a philosophy of trying to transform assets in a standardized way.
2. Vite is quite fast and lean
3. Vite exposes a much simpler plugin architecture, and also a simpler SSR API which elm-pages can use as a framework (other frameworks like SvelteKit and Astro are building around that feature as well)
4. There seems to be a growing consensus that Vite is the standard approach to bundling now, and the ecosystem is growing quickly

I've got a Vite integration working smoothly. I'm really pleased with how snappy it is and how convenient it is to get that nice bundling for production and fast dev server handling for non-Elm resources.

I just need to clean up a few things and then I'll push that up. But there are a few design decisions that come up with this new integration.

New Questions for the Vite Integration

• Should I just expose the entire Vite config? Seems like a lot of frameworks do that and just give an error if the framework has a merge conflict with the framework-specific Vite config (example: https://github.com/sveltejs/kit/blob/922eadf1843aa03c2a10b0652719e209f0288789/packages/kit/src/core/dev/index.js#L79). Fortunately, I'm not using Vite for the Elm compilation stuff and pre-rendering, so it doesn't really interact with it too much so should be fairly safe.
• Should the entrypoint files be hardcoded to specific files (index.js and style.css? What if you want index.ts or style.scss?)?
• One possibility is to let the user do import "style.css", but does that cause problems for an extra network waterfall chain? What about <link> preload directives, should those be automated, or not?
• Should it be possible for the user to add more entry point files, or should it be a pre-defined list? If the user should be able to add more entry point files, what are the advantages? And how would they include them (as HTML, or programmatically through Elm, or through a JSON configuration or Vite configuration file)?

This is also related to #195.

I would love to hear people's thoughts on this. This will open up some design decisions that will need to be explored, so I'd love to hear people's ideas and use cases around this.

[ffigiel]

I think an html template would be the most straightforward way to manage entrypoints and other global configuration like 3rd party scripts/fonts.

On the other hand, this approach may lead users to skip the awesome Head elm module and put everything in the html template. Even worse, they might put something in the body and end up confused since elm would remove it.
To some extent, this problem could be mitigated with instructions/comments in the template. It's not a great solution though

[ffigiel]

speaking of the Head module, perhaps it would be enough to allow users to define their entrypoints using that? for example head.Stylesheet or head.Script. or you'd have a separate tags for entrypoints and external resources, e.g. head.LocalScript (to be processed by vite) and head.ExternalScript (to be inserted as-is)

[ffigiel]

I don't believe Vite generates preload directives at all

oh! this means I misunderstood your earlier comment, sorry. Yes, that makes sense.

So if I got that SSR guide you linked right: broadly speaking, the user (programmer) could pass some metadata to elm-pages, which in turn will pass it to vite, which then generates html with preload directives for the specified resources.
Sounds like a good plan. Is this what you said you wanted few comments above - to have users express these things in elm and let vite handle the details?

[TheFedaikin]

Hopefully I am not shoehorning myself into this discussion.

1. Allowing the user to include global stylesheets and scripts seems like it could easily be done in an intuitive way. Page-specific styles would also be doable.

I think global script with arbitrary position in the head would be ideal for me and my #267 use-case. I think it's also great for stuff such as analytics, because it will allow you to flexibly add it in any scenario (I think there might be a case where you want it ASAP even before your Elm app loads, hence current index.js might not be enough).

It does concern me a bit to allow users to include page-specific scripts, because once a script is loaded you can't "undo" that, which means that you could have unintuitive behavior where a page depends on a script but doesn't include it, but it seems to work in your testing because you visit a page that does include that script first. My gut feeling is that that would be a footgun.

It's potentially dangerous in the Elm world and I can't think realistically about the use-case where you can't stub it out (for example add logic to global script that will only be executed on the specific route/event). It's useful for optimizing for size, but potential problems that it might cause, especially considering that upsides are not that high, are not worth it, imo.

[dillonkearns]

Thanks for the conversation @megapctr, and thanks for chiming in @TheFedaikin! I appreciate the discussion and the use cases. I'm glad to have all the perspectives here so we can have a thoughtful design since it's hard to backtrack on these types of APIs once they're in the wild.

I've been thinking about it a bit more, and I'm starting to think that it might make sense to give users a way to customize what is in the <head> in a somewhat low-level way. Then they could include <script> tags in any way that you could imagine (without a specific API exposed by elm-pages), but without a foot-gun for doing unintuitive things like loading for a specific route and having behavior depend on the order you navigate through pages.

Right now there is an HTML string that is pre-processed by Vite in both the dev server and production build. It's defined in this JS file:

elm-pages/generator/src/pre-render-html.js

Lines 52 to 81 in fd99a48

	function templateHtml() {
	return /* html */ `<!DOCTYPE html>
	<!-- ROOT --><html lang="en">
	<head>
	<!-- PLACEHOLDER_PRELOADS -->
	<script defer src="/elm.js" type="text/javascript"></script>
	<script defer src="${pathJoin(
	__dirname,
	"../static-code/elm-pages.js"
	)}" type="module"></script>
	<link rel="stylesheet" href="/style.css" />
	<meta charset="UTF-8" />
	<meta name="viewport" content="width=device-width,initial-scale=1" />
	<title><!-- PLACEHOLDER_TITLE --></title>
	<meta name="generator" content="elm-pages v${cliVersion}" />
	<meta name="mobile-web-app-capable" content="yes" />
	<meta name="theme-color" content="#ffffff" />
	<meta name="apple-mobile-web-app-capable" content="yes" />
	<meta
	name="apple-mobile-web-app-status-bar-style"
	content="black-translucent"
	/>
	<!-- PLACEHOLDER_HEAD_AND_DATA -->
	</head>
	<body>
	<div data-url="" display="none"></div>
	<!-- PLACEHOLDER_HTML -->
	</body>
	</html>`;
	}

I'm considering whether it might make sense to have an option to override part of this template from the elm-pages.config.mjs file.

My gut feeling is that it would make sense to make the red portion here customizable:

+<!DOCTYPE html>
+<!-- ROOT --><html lang="en">
+  <head>
+    <!-- PLACEHOLDER_PRELOADS -->
+    <title><!-- PLACEHOLDER_TITLE --></title>
-    <script defer src="/elm.js" type="text/javascript"></script>
-    <script defer src="${pathJoin(
-      __dirname,
-      "../static-code/elm-pages.js"
-    )}" type="module"></script>
-    <link rel="stylesheet" href="/style.css" />
-    <meta charset="UTF-8" />
-    <meta name="viewport" content="width=device-width,initial-scale=1" />
-    <meta name="generator" content="elm-pages v${cliVersion}" />
-    <meta name="mobile-web-app-capable" content="yes" />
-    <meta name="theme-color" content="#ffffff" />
-    <meta name="apple-mobile-web-app-capable" content="yes" />
-    <meta
-      name="apple-mobile-web-app-status-bar-style"
-      content="black-translucent"
-    />
+    <!-- PLACEHOLDER_HEAD_AND_DATA -->
+  </head>
+  <body>
+    <div data-url="" display="none"></div>
+    <!-- PLACEHOLDER_HTML -->
+  </body>
+</html>

That would allow the user to not load the Elm bundle at all if they wanted an HTML-only site, or to tweak the default meta tags.

As tempting as it sounds to do an Elm-based API for loading script tags, the more I think about it the more it seems like overkill to have to expose all the possible ways to load scripts. Having it as a string that is pre-processed by Vite seems like it supports all the use cases you would need in a reasonable way, without needing a complex API to describe all the possible ways you could include script tags or link tags.

I actually have some additional thoughts about styles, but I think that's a separate conversation. In a nutshell, I think it would be cool to have a way to include elm-css styles and have them split out into generated CSS and removed from the Elm bundle, and code-split by route for the production build.

So I think having a way to customize that part of the template HTML in the elm-pages.config.mjs file will be a pretty good starting point for this next release. Any thoughts on that?

[dillonkearns]

Oh yes, and for reference, this is what RemixJS's root.tsx file looks like. It also gives you low-level control over the starter template, though through JSX instead of a raw string.

https://github.com/remix-run/remix/blob/48b5f36cb1b9f2a43acd615fa97520df6cbe2995/examples/jokes/app/root.tsx#L46-L67

[dillonkearns]

To circle back, here's a discussion thread to discuss more ideas about the preload links: #337. The head tags template is now customizable in the v3 beta like so: https://github.com/dillonkearns/elm-pages-v3-beta/blob/6a067671200f896c2a9b10a31aab307f88943973/examples/docs/elm-pages.config.mjs#L12-L15.

[ffigiel]

Users might want to specify their entrypoints in a particular order, e.g. load the minimal css (or even inline it) before the full stylesheet, or load some script before the elm app because it's needed for init flags

[rolojf]

I think that to let the user do import "style.css" is a good idea in the index.ts or index.js.
That way, if the user would like to import "style.scss" instead or (as is my case) would like to use tailwindss with it's jit compiler. Vite would know what to do.

The way I do it now is with a script in package.json with concurrently. I feel that having Vite and no taking advantage of its capabilities is a waste.

I don't know if it causes extra network waterfall chain - I really don't get what you mean with this.

Not sure, just a point of view. Regards.