fulfill auths error/deprovision failed

wizza

Hi, sorry a bit unsure how to title this as I am a novice with regards to certificates and so it's not clear where the process is breaking. I'm sure it's something simple I haven't comprehended so I hope you don't mind me posting.

I have followed the "Getting Started" documentation, currently unable to generate a certificate.

Here is how my configuration appears:
Provider
ACME Server
ACME Account
Certificate

If I try to place a new order for this certificate, I get the following in the log:
4/16/2025, 9:28:59 AM, error, challenges/solver.go:101, challenges: deprovision failed (porkbun: unknown record ID for '_acme-challenge.pi*****.home.mydomain.net.' '***************************************8Qz4') 4/16/2025, 9:27:59 AM, info, orders/fulfilling_do.go:100, orders: fulfilling worker 2: order 5 done 4/16/2025, 9:27:59 AM, error, orders/fulfilling_do.go:99, orders: fulfilling worker 2: fulfill auths error: porkbun: failed to create record: status: 400 message: {"status":"ERROR","message":"Invalid domain."} 4/16/2025, 9:27:57 AM, info, orders/fulfilling_do.go:24, orders: fulfilling worker 2: ordering order id 5 (certificate name: pi*****.home.mydomain.net, subject: pi*****.home.mydomain.net)

I downloaded Lego, set my Porkbun API keys and ran the following:
lego --server=https://acme-staging-v02.api.letsencrypt.org/directory --email myemail@example.com --dns porkbun -d pi*****.home.mydomain.net run

This successfully generated the certificate.

Maybe the issue relates to the fact I am trying to generate a certificate for a host on a subdomain of my domain that I own through Porkbun? I can see in Porkbun when I manage my DNS there are two prebuilt TXT entries for _acme-challenge.mydomain.net, do I need to manually create additional entries here or something?

gregtwallace

Subdomains should be fine and you should not need to manually make any DNS records. The fact it works with the underlying application but not in CW is really perplexing me.

I did some Googling and see some threads re: porkbun and a similar error message. Are you using CNAMEs or wildcards by chance?

Also, I know the docs suggest not using acme.sh if you can use acme-lego, but can you try acme.sh and see if that works?

Also also, doubt this is the issue, but maybe worth trying: quote the environment var values, e.g., PORKBUN_API_KEY="pk1_..."

wizza

gregtwallace
Thanks for the quick reply! I tried acme.sh which gave me a bit more in the logs:
dns_checker: check _acme-challenge.pi*****.home.mydomain.net: propagated: 0 (0%, min: 100%), will check again in 15.3s
It kept repeating this over and over with increasing time between checks. This led me to performing an nslookup for the host which was not found...

So the issue was with the configuration of my local DNS where I use a pi-hole.
I have a pi-hole running with conditional forwarding to a local DNS server that manages DNS records for my local network (home.mydomain.net). Any lookups that fall into this subdomain are redirected by my pi-hole to this local DNS server (and not forwarded on from there if not found).
Therefore my pi-holes were redirecting lookups of _acme-challenge.pi*****.home.mydomain.net to my local DNS server and so they were never reaching porkbun. acme.sh seemingly correctly created the TXT records but then couldn't complete the lookup, whereas lego never created the TXT records in the first place.
Now both acme.sh and lego have succeeded after changing the DNS server for Cert Warden to a public provider (1.1.1.1)

This doesn't seem to be an issue running the lego command from a device other than Cert Warden that is using my pi-hole for DNS, like what I referenced in my original post. Any ideas why?

And finally, do you have a suggested method to bypass the need to change the DNS server for my Cert Warden instance? Is that where domain aliases might come in, where I can direct requests to my base domain mydomain.net instead so queries are not routed locally?

gregtwallace

This doesn't seem to be an issue running the lego command from a device other than Cert Warden that is using my pi-hole for DNS, like what I referenced in my original post. Any ideas why?

Yes. Annoyingly, the go-acme/lego code performs its own DNS lookups. I didn't find a convenient way around this so what I ended up doing was figuring out how to specify the DNS servers it would use. The path of least resistance was to set them to the system's DNS servers.

acme.sh doesn't have this problem so it relies solely on cert warden's DNS checker. You can specify the dns servers you want to use for that in the config, but the defaults are public servers. Did you have an internal server specified manually here? If not, I'm not sure why acme.sh would have been failing to find the records.
see: https://github.com/gregtwallace/certwarden-backend/blob/4113facfd001686bbf93ee3b019e64ef12ebb738/config.default.yaml#L51

And finally, do you have a suggested method to bypass the need to change the DNS server for my Cert Warden instance? Is that where domain aliases might come in, where I can direct requests to my base domain mydomain.net instead so queries are not routed locally?

If you use acme.sh and specify public servers in dns_checker (either use default or specify the ones you like), I think it should work.

wizza

gregtwallace
Ah, thanks for the context, that is making sense more sense now.

I probably should have mentioned this as well sorry, I'm not using it as a docker instance but rather it is running on a Proxmox LXC on Debian. It was set to use the Proxmox DNS servers (my pi-hole) which I manually overrode to public servers. I can specify public servers to make it work which is fine, but if I can avoid this customisation and change some configuration in Cert Warden to make it work then I think I would prefer that. Unless of course public DNS servers are specifically recommended by yourself to use in Cert Warden for this or some other reason 🙂

gregtwallace

The dns lookup is to determine when to tell the acme server to lookup the records (that is, it verifies they exist before telling the server they exist).

If you don’t want that, you can’t avoid dns use with go-acme. You could use acme.sh and then for dns_checker you can specify a sleep time. This will prevent the dns lookup and instead just assume the records are ready after the specified sleep time. Just set something that works for your records / provider.

Draw back would be if they end up having a slow day or other problem where the records don’t propagate as expected, you could end up with random unexpected errors and not know why.

wizza

gregtwallace
I've settled on go-acme/lego and using public DNS servers for now, maybe I will tinker with it later. Thanks for the suggestions.

Unrelated topic, but to avoid creating a new thread for what is probably a simple problem, I've tried creating a "serverdefault" cert which has had success but it doesn't seem to have taken automatically.

In the log:
4/16/2025, 4:01:05 PM, error, orders/fulfilling_do.go:208, orders: fulfilling worker 2: failed to load app's new https certificate (cannot load https certificate, server is in http mode)

If I try to access https://<certwarden>:4055 I get no connection. I tested and the server is not listening on port 4055, is there anything special I need to do to make HTTPS work for a non-docker install?
root@certwarden:~# lsof -i COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME master 278 root 13u IPv4 22642437 0t0 TCP localhost:smtp (LISTEN) master 278 root 14u IPv6 22642438 0t0 TCP localhost:smtp (LISTEN) sshd 3380 root 3u IPv6 22642000 0t0 TCP *:ssh (LISTEN) certwarde 3928 certwarden 9u IPv6 22655925 0t0 TCP *:4050 (LISTEN)

gregtwallace

Did you restart the application after generating the valid cert? It doesn't automatically switch from http to https.

wizza

gregtwallace
I thought I had, however I guess not... 🥲 reminder to self to RTFM!
Working now.
Thanks for your help!

woodsb02

Thanks guys - I've been facing this issue for a few weeks now, and finding this post has helped me solve it.

I my case, I'm running certwarden in podman, was seeing these same errors, and was struggling to troubleshoot it.

I tried to fix it by editing the /app/data/app/config.yaml inside the container to add:

challenges:
  dns_checker:
    dns_services:
      - primary_ip: 9.9.9.9
        secondary_ip: 149.112.112.112

But this didn't work - even after restarting the certwarden container.

Instead, I edited my Podman Quadlet to add:

DNS=9.9.9.9
DNS=149.112.112.112

And then after doing a systemctl daemon-reload and restarting the container, certificates were once again renewing.