Terminate functions running but module hook not firing

[Brandonito]

So, first, to clear up the first go-to questions:

1. PHP and MySQL Error logging is enabled both through WHMCS, Apache and PHP itself, and in the case of Apache and PHP, is also piped to a Graylog cluster
2. I have logModuleCall --everywhere-- to help debug issues like this. Our tblmodulelog is enormous (about 5 GB after 6 months) but our DB server is optimized for writes.
3. Crons are running properly
4. Nowhere in our custom code do we explicitly set "Cancelled;" we only set "Terminated"
5. Failed transactions are not a denominator - some have them, some dont
6. With the exception of a small number of edge cases (which are all 'read' and no 'write'), we use the API over manual DB voodoo even if it means increased complexity or lower performance

Environment:

Ubuntu 16.04, updated weekly, on AWS EC2
Amazon RDS (MySQL vanilla)
Apache2
PHP 7.0 (FPM)
ionCube (latest)
authorize.net CIM as gateway
WHMCS kept up to date
 

Ok, so onto the problem.

We have a number of custom modules (written by me), but most importantly, we've written a module that helps us to enforce contract agreements - the idea is, we have a few products that we place on discount in exchange for a lower price; in exchange, the customer agrees to a one-year term with a penalty should they cancel or stop paying before the term is up. Anyway, semantics at this point, becuase WHMCS doesn't even get to the point where it's supposed to trigger the terminate call, even though it takes all the steps prior to that.

So, the normal flow is like this:

1. User signs up, is presented with contract terms, then consents to them and enters their payment info;
2. After a few months, customer stops paying (NSF on card, expired card etc. but this applies only to non-customer initiated cancellations);
3. Customer is notified their payment failed.
4. After 3 days, their account is suspended;
5. 33 days after the payment fails, the account is terminated, and the contract is breached;
6. A new invoice is generated that includes the early cancellation fee;
7. The system attempts to charge the customer through the payment gateway.

But, it seems sometimes what's been happening is this:

1. User signs up, is presented with contract terms, then consents to them and enters their payment info;
2. After a few months, out of the frickin blue, the customer accounts flops over to "canceled," no cancellation request is made, the customer did not log in, no log entries are recorded, and no module termination triggers are fired!
3. The customer is completely unaware this has happened, as termination triggers have not fired.

Things I've checked:

1. Manual termination action (Works)
2. Activity logs (Nothing recorded)
3. PHP errors (Nothing recorded)
4. MySQL errors (Nothing recorded)
5. Gateway log (Some have declined, some don't, no errors)
6. Crons (Tested and confirmed working)
7. Automation configuration - auto suspension and auto termination are checked
8. Database log tables directly in case it wasnt recording the UserID correctly (through a query like 'SELECT * FROM tblgatewaylog WHERE data LIKE '%<userid>%'; -- all log tables checked, nothing obvious appears)

From a code perspective:

1. We have a provisioning module which fires a manual hook
2. This hook is intercepted by the contracts "addon." This works when the terminate is called manually.
3. However, about 50% of the time, the provisioning module terminate call isn't even firing! The product flips to 'Canceled' but there's nothing in the logs.
4. Reiterating, nowhere in our code do we actually set 'Canceled;' it's only terminate

Am I missing anything??

I could try and decouple the custom module, but I don't see the point, as it doesn't even trigger the provisioning module terminate call in the first place.