I believe I did:
sudo apt-get install astrometry.net
per this: indilib.org/about/ekos/alignment-module.html
I did not do:
sudo apt-get install kstars-bleeding astrometry.net

Is there a way to check that I installed the program correctly?

Can check on both astrometry.net and kstars-bleeding with the 'apt-cache policy':

astroberry@astroberry:~ $ apt-cache policy astrometry.net
astrometry.net:
Installed: 0.76+dfsg-3+b1
Candidate: 0.76+dfsg-3+b1
Version table:
*** 0.76+dfsg-3+b1 500
500 raspbian.raspberrypi.org/raspbian buster/main armhf Packages
100 /var/lib/dpkg/status

astroberry@astroberry:~ $ apt-cache policy kstars-bleeding
kstars-bleeding:
Installed: 6:3.4.0
Candidate: 6:3.4.2
Version table:
6:3.4.2 500
500 www.astroberry.io/repo buster/main armhf Packages
*** 6:3.4.0 100
100 /var/lib/dpkg/status

Let's also take a look at the permissions of the astrometry files themselves:

astroberry@astroberry:~ $ ls -lh /usr/share/astrometry/
total 9.7G
-rw-r--r-- 1 nobody nogroup 101M Dec 7 2014 index-4202-00.fits
-rw-r--r-- 1 nobody nogroup 104M Dec 7 2014 index-4202-01.fits
... etc ...

astrometry looks good:
Installed: 0.67+dfsg-1

kstars-bleeding (presumably this is the issue?
Installed: (none)

permissions:
total 2.3G
drwxr -xr -x 3 root root 4.0K Dec 7 2014 doc
-rw-r--r-- 1 vince vince 179M Dec 7 2014 index-4201-00.fits
etc..

You must have Kstars from the OS's repository installed? It's possible that version isn't recent enough to support offline astrometry plate solving.

So yes, best get the Kstars from the ppa in the links previously.

To help in daylight and without need of attaching mounts and such, you can create an additional Ekos profile. I call mine "Simulators" and I add a bunch of things like
Telescope Simulator for the mount
CCD Simulator for the imaging camera
Guide Simulator for the guide camera
That allows you to do a certain amount of "playing around".

Thanks for your help! I installed kstars-bleeding. Will test tonight and report back.

Yeah... so still doesn't work. Let's go over my setup real quick:

INDI server running on RPi3. Now includes the astrometry driver which i confirmed is running.
I've confirmed that the astrometry program is installed as is Kstars (but I haven't done anything with them... should I be launching Kstars on the pi?)
Kstars/EKOS client is a Windows 10 machine.
The equipment profile has Astrometry as an Aux driver.
We've confirmed the index files are where they are supposed to be and permissions are ok.
I get the same behavior: it captures a picture, sends it to the solver and then solver fails.

What else can I try?

On Windows, Offline is not available, so you have to select remote and select Astrometry driver in your equipment profile.

Yes, that’s what I’m doing and still solver fails. Any ideas how to troubleshoot?

I didn't realize you were accessing the RPi via KStars running in Windows; I thought you were running KStars directly on the RPi but via a VNC or browser interface from Windows.

I haven't yet set up a remote astrometry solver. As you're running Windows on your workstation, you don't have much choice in the matter, but I wouldn't normally set it up with the remote astrometry server running on the Pi as its processing power is limited compared to workstations. I mean I have it running locally on the Pi, but that's to ensure I have it when I'm in the middle of nowhere.

I'll have to look into how to set this up myself.

After an hour or so at this using my desktop machine as the putative astrometry server and a laptop as the client doing astrometry look-ups, all I get are solver failed errors.

I had the indi astrometry driver loaded and enabled on the server profile (with Telescope Simulator and CCD Simulator) and the same on the client, with it connecting to the server's IP address. Everything appears to load up fine on the client machine. From the server machine, I can do a local (i.e. offline) "Resolve and Slew" on an existing FITS but when I try that on the client with "remote" set it just fails.

Is there a guide somewhere detailing exactly what has to be configured for remote astrometry to work, on both the server machine and the client machines?

Ok, will give up on this. I installed ASTAP on my Windows machine (client) and it works great! It is blazing fast - solved in 0 seconds. Works for me. Thanks for your help.