Installation feedback

[domdomegg]

Hey team. This is not really a structured bug / feature request, but is some notes I took down when installing Mathesar. May be useful for optimising the onboarding experience.

1. Went to mathesar.org. I can't remember exactly how I found you. I think I was looking for Postgres interface that was friendly for non-super-technical people, and ideally made technical people more productive. Possibly via Googling for this, then stumbling onto your Hacker News launch thread?
2. Clicked 'I'm a developer' then 'Play with Live Demo' to get a feel for what Mathesar is all about. Enjoyed using the demo - it loaded quickly and I got a flavour for the features in Mathesar.
3. Given Mathesar seemed like a good product for our use case, I wanted to install a demo version and user test it with other people at my organisation to get feedback on whether they also found it useful and accessible. To be most realistic I wanted to attach it to a Postgres database that looked very similar to ours. I didn't think I could do this in the demo, so I decided to try to install it.
4. After the demo, I went back to the developer page and clicked 'Install Mathesar' which took me to https://docs.mathesar.org/
5. Now I was presented with many options, and wasn't really sure which one to pick. I picked Docker Compose, largely just because it was first in the list. If there is a 'good' way to do it, then maybe it's worth adding (recommended) to that option - or generally having a clearer nudge towards how to decide between these methods.
6. On this page, there are a lot of steps that seem irrelevant unless I want to do something. It feels like these should be hidden by default, or have the option to add them later (e.g. custom domain can probably be postponed until you've actually got the main thing up and running via localhost or similar).
7. I followed the steps to set up mathesar on the page. The wget commands didn't work because I'm on a Mac (it's also not pre-installed on Windows). Yes, I could install it with brew or similar, but annoying to have to do this. Maybe using curl (which also would mean the renaming step wasn't necessary) would make this more cross-platform compatible (e.g. curl -LO https://github.com/centerofci/mathesar/raw/0.1.2/docker-compose.yml && curl -Lo .env https://github.com/centerofci/mathesar/raw/0.1.2/.env.example).
8. I thought I needed to tweak a bunch of the env files to get things working. I'm now actually not so sure, but even now am fairly confused. I reckon this starter guide can just 'get things up and running in general', and then direct people to 'how to connect your own database' and 'how to set up a custom domain' and 'how to run on a custom port'
9. The page also directed me to setting up Caddy environment variables. I wasn't clear on how necessary this was, and it felt a bit clunky to be a separate step because I'd just been editing the environment variables in the same file for other things.
10. Connection URL problems: I had put in a connection string URL for a Postgres database hosted on neon (https://neon.tech/). This requires the endpoint id to be passed in and SSL to be enabled. (https://neon.tech/sni). This should be done automatically by newer versions of Postgres, but it seems Mathesar is using a version where this must be done manually. Entering these values in manually appears to have made some progress (the migrations succeeded) but ultimately I could not get the Django connection to work at all. I also seemed to get a lot of errors when not providing a connection port, despite using the default 5432. Ultimately I gave up trying to connect it to a real database and connected it to the provided Postgres database and copied the data across, given I was just doing an evaluation.
11. Hard to diagnose failure: When I tried going to the server locally, I just got various 500 errors with no extra information. This was hard to debug, and I think was due to the connection problems. Ideally, it would have worked with the connection URL above, but if it did fail it should have shown me a message saying 'Mathesar can't connect to your database. Review the environment variables X and Y to ensure the connection works. I think also the instructions by default run docker compose without logs (-d` parameter) so it's harder to figure out where things are failing.
12. When I did manage to get the UI to load (by abandoning a custom DB), I was presented with a login screen. By this point because I'd been debugging database things for a while I forgot I needed to setup a superuser (especially because it's hidden amongst a huge number of other complex instructions, just after you've jumped off to another page to configure Caddy). At this point, I think it would have been reasonable for Mathesar either to present a UI to set up this admin user OR at least state no admin user had been configured and direct me at the docs for setting up an admin user. Instead I got quite confused about what the username / password was that Mathesar wanted, and tried jumping around the docs to find this.
13. To bulk load data into the new database, I wanted to connect to it with another Postgres client. But the standard docker compose config doesn't support this. I figured out I needed to add a ports block to expose the port on the host machine, but it might be useful to include the in the docker-compose template (e.g. as commented out code) as I imagine this would be a common thing to do?
14. After getting into the database, and adding my tables next to the one I created in the UI, I couldn't see them in the UI. I read https://docs.mathesar.org/configuration/connect-to-existing-db/, but couldn't find out how to force a sync for quite a while until finding https://docs.mathesar.org/user-guide/syncing-db/. Feels like there should maybe a link to here from 'Connect to an existing database server'. On this in particular, it feels like it would be useful to have this automatically run e.g. every 1 minute, to keep things in sync between different tools. I don't know how compute intensive this on larger databases though.

[kgodey]

Thank you so much for this detailed feedback, @domdomegg! This is really helpful for us.

Making installation easier and more intuitive is one of our top priorities. Our original installation documentation was based off our local development setup, which turned out to be a mistake. We're now rethinking installation from the ground up, we're working on some improvements for the next release, and we're going to be continuing to simplify in the release after that – here are our meeting notes about that in case you're curious.

Your post has a lot of things we could pull out into smaller action items, we'll work on creating GitHub issues to track that work next week.

[domdomegg]

Glad it's useful! Happy to be a guinea-pig on setting up Mathesar again in future :)

I think if I was doing this again, I might just record a screencast of the entire experience - may be a helpful alternative to a full user interview (would also include a text summary of the key nice and pain points alongside it).