Blog

Why we're migrating Zacks Insight from React to Datastar

Last year I joined Zacks Intelligence as the lead engineer. The nice thing about joining in a leadership position is that you can impose your will and preferences on everybody else!

I inherited zacksinsights.com which was built on a Python backend of microservices and a React frontend. Since taking over this project, I have consolidated all the microservices into a monolith and we are currently migrating the frontend from React to Datastar.

In this post, I just want to go over the rationale for this decision and share our positive experience with Datastar. Hopefully it will be useful for others who might find themselves in a similar position.

Context

The team at Zacks Intelligence is very small. At the time of making this decision, we had one backend engineer and one frontend engineer. While Zacks Intelligence is part of the larger Zacks organization, we operate like a small startup since we are an independent subsidiary. So in this context, development speed is the number one priority over all other choices. We need to be able to quickly release new features, iterate on new experiments and test new theories as quickly as possible while maintaining a high quality experience.

At first, our biggest bottleneck was the fact that our application had several microservices despite serving only a small number of users and having a tiny development team. Consolidating all of these into one monolith paid back dividends as it simplified all aspects of backend development. In recent years, there have been good conversations about returning to the monolith structure, so I won't dwell too much on this point. My only recommendation to small teams like ours is that there's very little reason to develop microservices. A monolith can be deployed in different configurations. For example, one codebase that serves as the API gateway, the background worker process, etc., is just a much more practical way to approach the problem for many different reasons.

After resolving our backend issues, it became clear that our development speed was hampered by our frontend/backend split. Primarily, any useful feature in a user-facing product is driven by the interface, and any sufficiently complex feature is going to need to coordinate with the backend. Having to take into account this split just complicated our development unnecessarily. Features which sounded simple in discussions would end up taking longer than necessary to develop.

We also started to experience regressions in core functionality such as authentication. The root cause of the problem here is that for some of these core primitives, we would have to duplicate the management of them both in the backend and the frontend. Our backend had a concept of an authenticated user and their permissions and that sort of thing, but then the frontend would have to have a parallel system to handle authentication and view access on the frontend. In essence, it was clear that we had two sufficiently complicated apps that coordinated rather than having one unified system.

With a small team of just two, that meant that we were naturally divided into frontend and backend by experience or preference. If someone put in the effort to clean up and improve core things on one side, the other could not benefit. For example, if we improved error handling, error reporting and monitoring on the backend, doing the same on the frontend would be a completely separate project of equivalent scope. It was clear that we were duplicating effort, and the quality between the two implementations would often be different.

We also were maintaining two technologies: the Python stack and the JavaScript stack. Coincidentally, around that time we were discussing hiring, and we started to ask ourselves, "Do we need a frontend focused developer or do we need a backend focused developer?" Making that decision meant that one domain or the other would be likely understaffed.

It was at this time that I took advantage of my position as lead engineer to introduce my prejudices against the modern front-end stack and explore other options.

SPA vs. MPA

Based on my experiences, I was confident we needed to move away from an SPA to an MPA. I recognize that by allowing the backend to drive the UI, we would gain a lot of advantages. Primarily:

  • We could simplify our internal org to explicitly be a "Python shop" While not needing React or JavaScript specialists
  • By allowing the backend to drive the interface, team members could collaborate more effectively rather than work in parallel verticals
  • Based on my past experiences, MPAs tend to be simpler to maintain and simpler to reason about

Since our product is standard enough to work in an MPA, this was a natural choice.

Why Datastar

In the past, I built many MPA websites using classic HTTP patterns, such as form submissions, with frameworks like Django or Flask. In more recent years, I had been experimenting with HTMX, using it for internal tools at previous positions. However, I had hit the limitations of the patterns offered by HTMX. At some point on most of those projects, I either had to concede and adapt to the limitations of HTMX or break out of it and introduce a new dependency like Mithril JS or Alpine JS. In both cases, it just left an unsatisfactory feeling that I was missing something.

I don't recall how I came across it. I believe it was through one of the essays on the HTMX site, but I stumbled onto Datastar.

I have to be completely honest. When I first started learning Datastar, I did not understand it. The concepts behind Datastar, the JavaScript library, are elegantly simple. There's really not much to it, and if you walk through the guide on the website, you can complete it in a few minutes. However, I think it's very hard to appreciate what it brings to the table because it completely gets out of your way while enabling powerful patterns.

Since Datastar is a thin layer for the interface, ironically, most of the learning is about better ways to do things on the backend. For a little bit more than a year now, I've had the pleasure of participating in the Datastar Discord, which has enlightened me on new ways to approach backend development. Before this, when I thought of backend-driven development, I was thinking in the old style of HTML form submissions and page redirects. Thanks to the Discord I was exposed to newer ideas such as SSE, Fat Morph, CQRS and more.

Armed with this knowledge, I recognize that our team could move to the MPA style of web development while still retaining the reactivity and other frontend experiences that we still needed.

Datastar POC

Picking a new technology for the first time in a production environment always relies on a bit of faith. Since I hadn't used Datastar on a real application yet, I was going with my gut based on what I had learned from talks, discord, etc. I think for these situations, the best thing to do is to just start, pick some low-risk areas, and start developing. That's exactly what I did.

We had a need for a simple feature that we would use internally among business partners that would be a simple form submission that would trigger a background task and would need to wait until the task was completed. In classic MPAs, this is kind of kludgy to do, involving page refreshes and redirects, and might look something like this:

import asyncio
import uuid
from fastapi import FastAPI, Form, BackgroundTasks
from fastapi.responses import HTMLResponse, RedirectResponse

app = FastAPI()

# In-memory task store: {task_id: {"status": "pending"|"done", "result": ...}}
tasks: dict = {}

FORM_HTML = """
<h2>Submit a Job</h2>
<form method="POST" action="/submit">
  <input name="payload" placeholder="Enter something" required>
  <button type="submit">Run</button>
</form>
"""

STATUS_HTML = """
<h2>Task {task_id}</h2>
{body}
"""

async def slow_task(task_id: str, payload: str):
    await asyncio.sleep(5)  # simulate work
    tasks[task_id] = {"status": "done", "result": f"Processed: {payload.upper()}"}

@app.get("/", response_class=HTMLResponse)
async def index():
    return FORM_HTML

@app.post("/submit")
async def submit(background_tasks: BackgroundTasks, payload: str = Form(...)):
    task_id = str(uuid.uuid4())[:8]
    tasks[task_id] = {"status": "pending", "result": None}
    background_tasks.add_task(slow_task, task_id, payload)
    return RedirectResponse(url=f"/status/{task_id}", status_code=303)

@app.get("/status/{task_id}", response_class=HTMLResponse)
async def status(task_id: str):
    task = tasks.get(task_id, {"status": "not found", "result": None})
    if task["status"] == "done":
        body = f'<p>Done! Result: <strong>{task["result"]}</strong></p><a href="/">Back</a>'
    else:
        body = '<p>Working... (auto-refreshing)</p><meta http-equiv="refresh" content="2">'
    return STATUS_HTML.format(task_id=task_id, body=body)

While this is fine, it's a bit out of place in 2026. You would be hard-pressed to find a designer who would naturally design this flow.

Thankfully, Datastar makes this kind of thing just as easy, if not easier in a lot of cases. Especially if you use the official SDKs which provide a lot of conveniences. Here's what the equivalent of this might look like using Datastar.

import asyncio
import uuid
from fastapi import FastAPI
from fastapi.responses import HTMLResponse, StreamingResponse
from datastar_py.fastapi import DatastarResponse, ReadSignals, ServerSentEventGenerator as SSE
from app import worker

app = FastAPI()

tasks: dict = {}

def render(status="idle", result=""):
    if status == "working":
        content = "<p>Working...</p>"
    elif status == "done":
        content = f"<p>Done! Result: <strong>{result}</strong></p>"
    else:
        content = ""
    return f"""
    <main>
        <input data-bind-payload placeholder="Enter something">
        <button data-on-click="@post('/run')">Run</button>
        {content}
    </main>
    """

async def slow_task(task_id: str, payload: str):
    await asyncio.sleep(5)  # simulate work
    tasks[task_id] = {"status": "done", "result": payload.upper()}

@app.get("/", response_class=HTMLResponse)
async def index():
    return render()

@app.post("/run", response_class=StreamingResponse)
async def run(signals: ReadSignals):
    payload = signals.get("payload", "")
    task_id = str(uuid.uuid4())[:8]
    tasks[task_id] = {"status": "pending", "result": None}

    async def stream():
        tasks[task_id]["status"] = "working"
        yield SSE.patch_elements(render(status="working"))

        worker.enqueue_task(slow_task(task_id, payload))
        await worker.wait_for_task_result(task_id)

        yield SSE.patch_elements(render(status="done", result=tasks[task_id]["result"]))

    return DatastarResponse(stream())

Committing to Datastar

Since we were bought into Datastar at a philosophical level and since our experiments and POC went really well, we decided to commit to Datastar. We decided to entirely rebuild our website to be 100% Datastar, dropping React entirely. So far, we are happy with the decision and have not regretted it, but there has been some learning along the way.

At this stage, I don't have compelling before and afters or metrics or that sort of thing, so this will be very much a "trust me, bro" moment. Heck, we haven't even deployed the new version of the site yet! However, the most important thing I can say is that rebuilding the site with Datastar has been easy. The code is less complex than our old frontend/backend split, and it has just been much more pleasurable to develop. It has also empowered us to implement more ambitious features that were previously assumed to be too complex (e.g. "multiplayer" chat).

The most important part is that it solved all of the original pain points for our team.

  • We now have one code base where multiple developers cross-contribute and build upon each other's work more naturally in Python
  • We also are moving much faster to build features that previously took more time when orginally built in React.
  • The code is simpler to maintain and reason about for our small team

As a positive side effect, despite this not being one of our main goals, the site is actually faster and more reliable. It's a lot easier for us to unit, integrate and end-to-end test as well.

Future

We're still in the middle of the migration. There's probably some interesting things to share and discuss about limitations, for example, having to reach for web components for more complex features or the side effect of latency. However, honestly, these trade-offs are so marginal and the gain so outsized that it doesn't even feel worth it to delve into those topics. After we've completed the migration and deployed to production for a bit of time, it would be interesting to revisit this topic and explore some of the data on our side on a more technical level.

In the meantime, I hope that sharing this experience might open up some people to explore the tool and try a new way of website development. Without too much hyperbole, Datastar revived my interest and enjoyment in building websites, something that had been waning for the past many years.

A big thank you to Delaney, Ben and the other contributors!