Knex: Timeout acquiring a connection

I’m getting some errors in my logs that some anchoring fails due to a bad request from the clay server. It seems like the server was overloaded or something as it says the pool is probably full. I think maybe this could cause the CACAO expired errors we’ve been dealing with before, so wanted to report this in case it needs to be looked into. These are the logs:

Oct 30 19:06:30 discourse-graph-demo ceramic[31031]: [2022-10-30T19:06:30.471Z] WARNING: Anchor failed for commit bagcqceraxek6uqj3bxpvgbujwqdt7ao26w2unzoig5fzfmviyzzfwow5wf3q of stream kjzl6kcym7w8y6m7gc6xlapspgeo260wlsa6dhi8ryq4ajvljtnbx1fly28fj01: HTTP request to 'https://cas-clay.3boxlabs.com/api/v0/requests/bagcqceraxek6uqj3bxpvgbujwqdt7ao26w2unzoig5fzfmviyzzfwow5wf3q' failed with status 'Bad Request': {"error":"Loading request status for CID bagcqceraxek6uqj3bxpvgbujwqdt7ao26w2unzoig5fzfmviyzzfwow5wf3q failed: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call?"}
Oct 30 16:46:06 discourse-graph-demo ceramic[31031]: [2022-10-30T16:46:06.213Z] WARNING: Anchor failed for commit bagcqcera2jdme27qzuan5w3vn7tkzqsfk2dstnvl2tw3nmuvxub4d3bsrd4q of stream kjzl6kcym7w8y9c18xtd4e4tcm5m9qx263rql9tz7u5q59etcp0suxmiood5xwo: HTTP request to 'https://cas-clay.3boxlabs.com/api/v0/requests/bagcqcera2jdme27qzuan5w3vn7tkzqsfk2dstnvl2tw3nmuvxub4d3bsrd4q' failed with status 'Bad Request': {"error":"Loading request status for CID bagcqcera2jdme27qzuan5w3vn7tkzqsfk2dstnvl2tw3nmuvxub4d3bsrd4q failed: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call?"}
Oct 30 16:22:41 discourse-graph-demo ceramic[31031]: [2022-10-30T16:22:41.625Z] WARNING: Anchor failed for commit bagcqcera7sv7okibecewmyr32xx3uwvcj2j62e7anuats4hf32j54gynexna of stream kjzl6kcym7w8y5idp4mfqave7vs9ose3h76llhwmuwe6xnpzt8qhjpejvhpr3a4: HTTP request to 'https://cas-clay.3boxlabs.com/api/v0/requests/bagcqcera7sv7okibecewmyr32xx3uwvcj2j62e7anuats4hf32j54gynexna' failed with status 'Bad Request': {"error":"Loading request status for CID bagcqcera7sv7okibecewmyr32xx3uwvcj2j62e7anuats4hf32j54gynexna failed: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call?"}
2 Likes

Thanks so much for reporting these, @nax :pray:t3:

We’ll take a look.

As you asked for the other issue you reported, mind adding this to Github issues as well?