Usage metrics not updating

Kamran · January 8, 2020

Greetings,

I'm developing a module to setup a user on my server and sync their usage data to whmcs. I've successfully contextualized usage metrics now the big issue here is how whmcs track usage metrics my progress so far is below:

I've created a new class which implements "ProviderInterface" below is the code for "metrics" function

 public function metrics()
     {
         return [
             new Metric(
                 'disk',
                 'Disk Utilized',
                 MetricInterface::TYPE_SNAPSHOT,
                 new MegaBytes,
                 new Usage(0)
             )
 			];
     }

usage function is

public function usage()
     {
         $serverData = $this->apiCall('stats');
         $usage = [];
         foreach ($serverData as $data) {
             $usage[$data['username']] = $this->wrapUserData($data);
         }
         return $usage;
     }

tenantusage function is

public function tenantUsage($tenant)
     {
         $userData = $this->apiCall('user_stats');

         return $this->wrapUserData($userData);
     }

helper wrapuserdata function is:

private function wrapUserData($data)
     {
         $wrapped = [];
         foreach ($this->metrics() as $metric) {
             $key = $metric->systemName();
             if ($data[$key]) {
                 $value = $data[$key];
                 $metric = $metric->withUsage(
                     new Usage($value)
                 );
             }
             $wrapped[] = $metric;
         }
         return $wrapped;
     }

And lastly helper apicall function which delibrately sends a value somewhat emulating an api call for now.

private function apiCall($action)
     {
         if($action == "stats"){
            return [
                [
                "username" => "kamran@domain.com", //this represents my username which is provided to whmcs
				"disk" => 100
            ]
            ];
         }
         else{
            return [
                [
                "disk" => 100
            ]
            ];
         }
     }

Okay so far in the above mentioned code, date and time updates but usage details are not updating. BTW I'm checking metrics using module parameters using $params["metricStats"][0]["currentvalue"]

So please help me with the below mentioned questions.

What is the correct way to return "usage()" function including what should be the identifier of the tenant(i.e. the key of value usage object where key should be serviceid?accountid?userid? of the tenant)?
Whenever cron job "TenantUsageMetrics" runs only "usage()" function is invoked and never "tenantusage()" why?
Also if my syntax is wrong a possible example would be tremendous.

Thank you for hearing me out!

Lobster · January 16, 2020

I'm also joining this question

WHMCS David · January 20, 2020

Hi @Kamran

First, in your main module file do you define a function ending in _MetricProvider ? This function is responsible for returning an object that implements \WHMCS\UsageBilling\Contracts\Metrics\ProviderInterface. It is this object that WHMCS will use to get metrics. Since you mention that the cron is calling usage(), I presume you have it...I just thought it was important to mentions since you do not.

Following up to that, as well as your questions:

The usage() method of the ProviderInterface is invoked during the cron in order to collect all stats at once. The idea is to have one method WHMCS invokes; if you server supports providing all "tenant" (aka user, or domain) stats at one API call, then great; otherwise your module can get the list of individual records however is most optimal for your API.

The return value from usage()should be an array. The key of the array is the "tenant" and the value is an array of \WHMCS\UsageBilling\Contracts\Metrics\MetricInterface. WHMCS maps the service's domain field to the tenant.

The tenantUsage() method is only invoked within the context of a service. So for example, when you are viewing a client's service in the admin area, and you press the refresh icon in the stats table, this will invoke that single method (and an other tenant stats on that server will not be refreshed...just the tenant associated with this service you are viewing). You do see the table for metric stats on the service page, right? If you don't see it, you might have an issue with the tenant value and how it is mapped to the service...It looks like you are using kamran@domain.com as the tenant value, but I suspect this is not the value in the service's domain field. My guess is this is your issue.

I would not recommend trying to observe value change using $params. In fact, I don't even follow what you're doing. After running the TenantUsageMetrics cron task, or clicking the refresh button, you should see changes on the service's stats page.

In your example, in the metrics method, you return an array with one Metric. This is good, however, you should not instantiate this with "new Usage(0)". Simply, do no pass that last parameter. The constructor for Metric will know how to treat "no usage data". This should not cause any issue, however, it's better to let the system deal with "no usage data" that stub it with "usage of zero".

Hopefully this helps you get on your way. Let me know how it goes.

Kamran · January 20, 2020

Hi @WHMCS David

Thank you for responding as this is quite critical for my employer, much appreciated!

I understand your concern but I've returned MyMetricsProvider(implements ProviderInterface). Secondly now I'm following you on the tenant id part and I'm using service properties to set username as well as domain and then keying the domain name so my new api call function is this

private function apiCall($action)
    {
        if($action == "stats"){
            return[
                [
                    "username" => "kamran2.com",
                    "disk" => 99
                ]
                ];
        }
        return[
            [
                "disk" => 100
            ]
            ];
    }

And as per your advised method I'm not passing any Usage object on metrics function, everthing else is the same as what I've provided on my original post but still I'm not getting the desired output even on the admin area the result I see on the admin area is provided below

Also whenever I click on refresh now tenantUsage() function didn't run(I verify using module logs I've placed logModuleCall() in both Usage() and TenantUsage()) also I tried to check if the database stores the values correctly so below is the result for:

tblserver_tenants

| 19 |         4 | kamran2.com          | 2020-01-20 20:43:02 | 2020-01-20 21:01:27 |

tbltenants_stats

| 57 |        19 | disk           | snapshot |  99.000000 | 1579554086.759813 |          0 | 2020-01-20 21:01:26 | 2020-01-20 21:01:26 |
+----+-----------+----------------+----------+------------+-------------------+------------+---------------------+---------------------+

So clearly values are tracked correctly right? Also I didn't modify or inset anything on the database and let WHMCS do the job I just checked.

WHMCS David · January 20, 2020

@Kamran,

I'm glad to see that the usage is being stored based on the cron task. At this point, since the refresh is not triggering a method call, I would guess that there's something not aligned with the service and the server.

Please try something like the following:

select id, server, domain, username from tblhosting where domain = 'kamran2.com';

You should end up with just one record, and it should show the value tblhosting.server as 4 (as indicated in by your output of tblserver_tenants), and the tblhosting.id should be the id the same as the id parameter in the URL of the client service you are trying to observe the value in.

You are running v7.9.1, right?

Kamran · January 21, 2020

@WHMCS David

The problem is partially SOLVED huge thanks to @Lobster (who was in touch on direct message)for his help so my problem was I needed to use the below mentioned code in the metadata(This is not included in the sample provisioning code)

'ListAccountsUniqueIdentifierField'=> 'domain',

Now TYPE_SNAPSHOT works fine along with TYPE_PERIOD_MONTH but I'm facing problems with TYPE_PERIOD_DAY my code is still the same only change is that now my metrics function becomes

 public function metrics()
    {
        return [
            new Metric(
                'dsk',
                'Disk',
                MetricInterface::TYPE_PERIOD_DAY,
                new GigaBytes()
            ),
            new Metric(
                'vm',
                'VMs',
                MetricInterface::TYPE_SNAPSHOT,
                new WholeNumber("vms")
            )

        ];
    }

everything works fine except the metric disk which is TYPE_PERIOD_DAY let me know please if you have any suggestions.

By the way @WHMCS David glad to know WHMCS really cares about it's clients and their needs your and @Lobster's help is much appreciated!

Thanks for hearing me out..

WHMCS David · January 21, 2020

Hi @Kamran,

Glad you're making progress.

The TYPE_PERIOD_DAY should not be used at this time; only documented in the class level docs because it is reserved for a potential future design consideration. There are many factors to daily cyclical remote metrics that are not in the current implementation and first iteration of the Usage Billing feature.

RyanK · January 23, 2020

On 1/20/2020 at 5:26 AM, WHMCS David said:

The return value from usage()should be an array. The key of the array is the "tenant" and the value is an array of \WHMCS\UsageBilling\Contracts\Metrics\MetricInterface. WHMCS maps the service's domain field to the tenant.

As far as I'm aware this isn't documented. To the contrary, the documentation (https://developers.whmcs.com/provisioning-modules/usage-metrics/) indicates that the "tenant" should be the "username".

I'm having a hard time understanding why usage metrics would use the domain name instead of something more generic? I have services without a domain name that I'd like to display usage metrics for. I just assumed the tenant would be the serviceid since it's a unique identifier that every service has.

Please update the documentation. I certainly wouldn't have guessed that the tenant ID is the service's domain name, and without documentation all I can do is guess.

Edited January 23, 2020 by RyanK

WHMCS David · January 23, 2020

Hi @RyanK

Thanks for the feedback. Yes, the documentation with respect to what you mention, as well as what was discussed with @Kamran /this thread earlier, will be updated to clarify these new features.

RyanK · January 24, 2020

@Kamran or @WHMCS David, can you clarify the usage of

'ListAccountsUniqueIdentifierField'=> 'domain',

I didn't find any mention of this field at https://developers.whmcs.com/provisioning-modules/meta-data-params/

My best guess is that ListAccounts can be substituted for any module function name, so I tried setting:

function mymodule_Metadata() {

'MetricProviderUniqueIdentifierField' => 'id',
.....

function mymodule_MetricProvider($params) {
    return new MyMetricsProvider($params);
}

My hope was that this would use the service's ID field as the identifier, as that's how I've been exporting my metrics. My metrics are being recorded, just nothing happens and tenantUsage isn't called when view the service page.

mysql> select id, server, domain, username from tblhosting where id=11;
+----+--------+--------+----------+
| id | server | domain | username |
+----+--------+--------+----------+
| 11 |      1 |        |          |
+----+--------+--------+----------+
1 row in set (0.00 sec)


mysql> select * from tblserver_tenants where tenant=11;
+----+-----------+--------+---------------------+---------------------+
| id | server_id | tenant | created_at          | updated_at          |
+----+-----------+--------+---------------------+---------------------+
|  4 |         1 | 11     | 2020-01-23 11:04:11 | 2020-01-23 23:33:14 |
+----+-----------+--------+---------------------+---------------------+
1 row in set (0.00 sec)


mysql> select * from tbltenant_stats where tenant_id=4;
+----+-----------+-------------+----------+-------------+-------------------+------------+---------------------+---------------------+
| id | tenant_id | metric      | type     | value       | measured_at       | invoice_id | created_at          | updated_at          |
+----+-----------+-------------+----------+-------------+-------------------+------------+---------------------+---------------------+
| 16 |         4 | cpu         | snapshot |   73.125000 | 1579822394.941318 |          0 | 2020-01-23 11:04:11 | 2020-01-23 23:33:14 |
| 17 |         4 | memory      | snapshot | 7670.000000 | 1579822394.942335 |          0 | 2020-01-23 11:04:11 | 2020-01-23 23:33:14 |
| 18 |         4 | disk        | snapshot | 5262.000000 | 1579822394.943267 |          0 | 2020-01-23 11:04:11 | 2020-01-23 23:33:14 |
| 19 |         4 | allocations | snapshot |    2.000000 | 1579822394.944155 |          0 | 2020-01-23 11:04:11 | 2020-01-23 23:33:14 |
| 20 |         4 | databases   | snapshot |    0.000000 | 1579822394.945082 |          0 | 2020-01-23 11:04:11 | 2020-01-23 23:33:14 |
+----+-----------+-------------+----------+-------------+-------------------+------------+---------------------+---------------------+
5 rows in set (0.00 sec)

How would I configure the server module so that it will use the service ID to look up the tenant usage stats?

Thanks!

Edited January 24, 2020 by RyanK
formatting

WHMCS David · January 24, 2020

@RyanK

it is not possible in the current implementation to use service.id to look up the tenant usage. Using that service field would be extremely challenging because the server almost certain doesn't know what auto incrementing id in WHMCS's database table tblhosting is for a service it is hosting (vps/vm/shared/otherwise). There must be a common field that is known to both the WHMCS service and the remote server (ala the server module). In WHMCS 7.9, the feature is meant to use the service field 'domain'; more below.

"MetricProviderUniqueIdentifierField" doesn't exist, only "ListAccountsUniqueIdentifierField". "ListAccountsUniqueIdentifierField" is pretty new and will be itemized in the documentation revisions I spoke about in my last post. At this time of this writing, Usage Billing's utilization of "ListAccountsUniqueIdentifierField" was designed to support only the value of 'domain', but with future considerations for other service fields. If this metadata key is not present, the core logic should implicit use that field. If that field in the service is empty, the system will assume there is no tenant stats recorded to map to (and thus nothing will be rendered on the service page). [fwiw, there was a bug in pre-production that affected this behavior; make sure you're running the latest 7.9]

At this time you must have a value in service.domain and it must be a key of the array returned by MetricInterface->usage() for the feature to work.

RyanK · January 24, 2020

Well that's unfortunate. I filed a feature request to make the field configurable for servers that don't use domain names.

https://requests.whmcs.com/topic/make-field-to-use-for-tenantusage-configurable

As mentioned, ideally I wouldn't need to store an identifier in the domain field, since this seems like it would be confusing to customers.

Thanks for the explanation!

Lobster · January 24, 2020

20 hours ago, RyanK said:

Well that's unfortunate. I filed a feature request to make the field configurable for servers that don't use domain names.

https://requests.whmcs.com/topic/make-field-to-use-for-tenantusage-configurable

As mentioned, ideally I wouldn't need to store an identifier in the domain field, since this seems like it would be confusing to customers.

Thanks for the explanation!

Have you actually tried to change the value of

'ListAccountsUniqueIdentifierField' => 'domain',

into this:

'ListAccountsUniqueIdentifierField' => 'id',

And then in your usage() method return array with keys, that are the same as service ids?
For instance:

public function usage()
{
	return [
		1 => $this->wrapUserData([
			'disk' => 321,
			'bandwidth' => 123,
		]),
		2 => $this->wrapUserData([
			'disk' => 321,
			'bandwidth' => 123,
		]),
		3 => $this->wrapUserData([
			'disk' => 321,
			'bandwidth' => 123,
		]),
	];
}

Where keys 1, 2 and 3 are service ids respectively.

So, according to that scenario, WHMCS first runs provisioningmodule_CreateAccount() function, where you are already able to get 'serviceid' from the $params array and pass it further (via api, for instance).
On the other side you can save it to keep the relation between whmcs entity and the created entity on your "server" side.
And when WHMCS calls usage() function you just need to return the array mapped by appropriate keys. It is obviously not the perfect solution, but it should work.

P.S. The key 'ListAccountsUniqueIdentifierField' was previously described here https://developers.whmcs.com/provisioning-modules/server-sync/.
So if you don't have a server sync feature integrated within your provisioning module, I believe you can safely use it for usage stats.

On 1/23/2020 at 8:35 PM, RyanK said:

Please update the documentation. I certainly wouldn't have guessed that the tenant ID is the service's domain name, and without documentation all I can do is guess.

Totally agree with this statement.

Mike Polyakovsky · July 29, 2020

On 1/24/2020 at 3:49 PM, WHMCS David said:

At this time you must have a value in service.domain and it must be a key of the array returned by MetricInterface->usage() for the feature to work.

Looks like we can use custom fields from 7.10 to look up the tenant per https://developers.whmcs.com/provisioning-modules/meta-data-params/

ListAccountsUniqueIdentifierField - The following values are supported: domain, username, customfield.yourFieldName

That kind of work for me with the TenantUsageMetrics crontask, however I can't make it work for the "Refresh Now" button - tenantUsage doesn't seem to be triggered in the first place. I get the following response:

{"success":false,"error":"Tenant Identifier or Server ID missing"}

It works fine when using domain or username options btw, but the customfield doesn't work as expected. Am I missing smth?

Mike Polyakovsky · August 4, 2020

On 7/29/2020 at 2:23 PM, Mike Polyakovsky said:

Looks like we can use custom fields from 7.10 to look up the tenant per https://developers.whmcs.com/provisioning-modules/meta-data-params/

ListAccountsUniqueIdentifierField - The following values are supported: domain, username, customfield.yourFieldName

That kind of work for me with the TenantUsageMetrics crontask, however I can't make it work for the "Refresh Now" button - tenantUsage doesn't seem to be triggered in the first place. I get the following response:

{"success":false,"error":"Tenant Identifier or Server ID missing"}

It works fine when using domain or username options btw, but the customfield doesn't work as expected. Am I missing smth?

Btw, I've created support ticket and they confirmed the defect and opened up a new case to have this reviewed for future releases.

Usage metrics not updating

Recommended Posts

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Join the conversation

Similar Content

Recently Browsing 0 members

Important Information