Whois logs

[yggdrasil]

Is this just me or domain searches are not logged anymore in the WHOIS lookup log in the admin side?

I don't see them in the database either.

If I call them with an API, they are registered, but not if I search a domain manually from the domain register form. Strangely I do see some searches, but its definitely not registered all of them.

[brian!]

I think it depends on the lookup provider - standard whois should log them all (including alternatives), but namespinning definitely doesn't log every domain it searches... it doesn't necessarily even log the specific domain that was entered to be searched for! 🙄

[WHMCS Dan]

On 11/07/2020 at 21:14, yggdrasil said:

Is this just me or domain searches are not logged anymore in the WHOIS lookup log in the admin side?

I don't see them in the database either.

If I call them with an API, they are registered, but not if I search a domain manually from the domain register form. Strangely I do see some searches, but its definitely not registered all of them.

The WHOIS log only shows WHOIS checks performed, as noted by @brian! other lookup providers such as registrars or WHMCS Namespinner use alternative methods to check domain availability so the checks will not be logged in the WHOIS Log.

[yggdrasil]

So there is no way to see all domain searches?

I used this in the past to keep an eye on traffic for domains, but Brian! is correct, it only works for some standard searches. I have the public Whois page disabled, so it's not from people doing manual searches on my installations but for some specific domains that use the standard Whois the are logged, everything else is not. This seems very inconsistent.

Is there a way to have all searches registered in the Whois log? Otherwise, it makes this more or less useless.

 

[WHMCS Dan]

I can certainly understand why you'd want to see a log of domain searches, this can be useful data to understand your clients.

The best thing to do here would be to raise a Feature Request for this here: https://requests.whmcs.com we can then gather community support for such a feature and look at implementing this in a future release.

[brian!]

9 hours ago, yggdrasil said:

This seems very inconsistent.

what is weird is let's say i'm using Namespinning, and I search for whmcsdan.com in the client area - even though it's available and it's specifically what i'm searching for, that isn't logged in the whois logs, but whmcsdan.co.uk is logged twice - even though I only searched for the .com once and there are spotlight and suggested TLDs listed in the results and setup before .co.uk ?

I suspect that it's been like this for a long while.

6 minutes ago, WHMCS Dan said:

The best thing to do here would be to raise a Feature Request for this here: https://requests.whmcs.com we can then gather community support for such a feature and look at implementing this in a future release.

or to put it a more accurate way - originally, when standard whois was the only lookup option, the logging worked as expected and described...

https://docs.whmcs.com/System_Utilities#Activity_Logs

Quote

WHMCS logs all activity, admin logins, gateway communications, sent and received email communications, and domain lookups. This allows you to monitor and track all the activity taking place inside your WHMCS system.

however, when WHMCS introduced Namespinning (aka eNom) and Lookup Providers years ago, WHMCS coded them so poorly (at least NS) that it doesn't even log what is being searched correctly... i'd file that as a bug introduced by WHMCS developers rather than something that needs to be feature requested - surely we haven't reached the stage of users being required to create feature requests to fix bugs. 🙄

btw - if I lookup whmcsdan.com in v7.8.3 or v7.9.2, then I get the following (and on all versions there are more spotlight listed TLDs that are returning results, but are not being listed in the logs).

I would have thought with Namespinning, at the very least it should log any domain that was specifically searched for, and/or the first available result it found - even if it didn't list the rest of the alternative / suggestions it also looks up ??

although I appreciate that the default inclination of WHMCS will now be to alter the documentation as opposed to fixing the underlying issue... Dan, do try to ensure those above you resist that urge and encourage them to fix issues rather than masking that they even exist.

[WHMCS Dan]

Thanks for raising the inconsistency with our docs page, I've gone ahead and amended our docs to make clear it is WHOIS Lookups which are logged, not domain lookups to help avoid confusion in future. 

I can also see we don't seem to have an explanation of the WHOIS fallback checking when using WHMCS NameSpinning or a registrar lookup provider, so I've raised a case internally to get this documented and I'll explain how this works briefly here.

When you select WHMCS Namespinning or a registrar lookup provider, often these will not support all TLDs and so we fallback to using WHOIS lookups to check if the domain is available.

Because this is a WHOIS lookup, this is then logged, but the domains which are checked via WHMCS Namespinning or a registrar lookup provider are not logged as this uses a different method to check the availability.

This fallback functionality is actually the result of another feature request:  https://requests.whmcs.com/topic/lookup-provider-fallback-to-classic-whois-lookup 

I have discussed this with my manager to confirm, but as adding logging for non-WHOIS lookup providers would be new functionality a Feature Request is the best way to get this raised so we can track interest and consider putting this on the development schedule. 

 

[brian!]

6 minutes ago, WHMCS Dan said:

Thanks for raising the inconsistency with our docs page, I've gone ahead and amended our docs to make clear it is WHOIS Lookups which are logged, not domain lookups to help avoid confusion in future. 

and you think all WHMCS users will know the difference?

8 minutes ago, WHMCS Dan said:

When you select WHMCS Namespinning or a registrar lookup provider, often these will not support all TLDs and so we fallback to using WHOIS lookups to check if the domain is available.

very strange that eNom don't support .uk lookups natively.

16 minutes ago, WHMCS Dan said:

This fallback functionality is actually the result of another feature request:  https://requests.whmcs.com/topic/lookup-provider-fallback-to-classic-whois-lookup 

one vote and it got implemented... yet other requests can get tens or hundreds of votes and can take 7+ years to be added or not implemented at all ?

Quote

While we do not implement features based solely on their popularity or age, requests that get a lot of traction in our feature requests tracker do get reviewed by our team.

one vote is traction? 😲

I suspect i've been around this place long enough to know that you were going to undertake this feature anyway, and just marking it as completed gives the impression that it was the request that generated the idea... I don't believe that it did, but if it's true, then FR is in a worst state than I thought.

besides, didn't Namespinning always do that, e.g it had a fallback to whois ? it was the lookup providers that didn't originally... and i've not been talking about lookup providers, only NS and whois.

[bear]

1 hour ago, brian! said:

one vote and it got implemented... yet other requests can get tens or hundreds of votes and can take 7+ years to be added or not implemented at all ?

Quote

While we do not implement features based solely on their popularity or age, requests that get a lot of traction in our feature requests tracker do get reviewed by our team.

one vote is traction? 😲

I suspect i've been around this place long enough to know that you were going to undertake this feature anyway, and just marking it as completed gives the impression that it was the request that generated the idea... I don't believe that it did, but if it's true, then FR is in a worst state than I thought.

I just wanted to quote this part, because of just how absurd the truths are behind it. One vote traction....another with dozens of votes and many years...closed for lack of interest. Yup, checks out. 🙈

[WHMCS Dan]

After further researching some of the inconsistencies described in this thread, I was able to reproduce that when searching for some extensions such as .co.uk this appears to be logged in the WHOIS Log despite being supported by the NameSpinner service.

I have now raised case #CORE-14854 with our developers in order to have this inconsistency reviewed for future releases. Whilst I cannot provide an estimated time for completion for this, once we resolve cases and push features they are available at our change log, here:

http://changelog.whmcs.com/

[yggdrasil]

3 hours ago, WHMCS Dan said:

After further researching some of the inconsistencies described in this thread, I was able to reproduce that when searching for some extensions such as .co.uk this appears to be logged in the WHOIS Log despite being supported by the NameSpinner service.

I have now raised case #CORE-14854 with our developers in order to have this inconsistency reviewed for future releases. Whilst I cannot provide an estimated time for completion for this, once we resolve cases and push features they are available at our change log, here:

http://changelog.whmcs.com/

But how will it be left in the future? To log searches or not log them? I would prefer to have all searches logged, regardless if they are from the API registrar, namespinner, or fall back to the standard Whois. As I use all 3 of combinations. In case a domain is not supported in a registrar, WHMCS falls back to the standard Whois file which I use mostly for country level domains, those are also logged, hence why I was confused that some searches showed up but not others, so it's not only for .uk but other ccTLDs as well.

Edited July 16, 2020 by yggdrasil

[brian!]

17 hours ago, yggdrasil said:

But how will it be left in the future? To log searches or not log them?

I think you have been given the answer, e.g the current situation continues, e.g *some* searches will be logged, e.g those directly via whois, or those that use whois on fallback.

i'm inclined to agree with you that this is next to useless if you're using multiple lookup methods and not solely standard whois - either log nothing or log everything (regardless of lookup method).

but what you end up with are selective logs (the worst of all worlds), e.g you can't tell that someone searched for whmcsdan.com on your site unless a) eNom or the LP don't support that TLD and the whois search gets logged, or b) the user later purchases the domain.... actually, not even b) because they could have searched for whmcsdan.net, but chose the .com instead if provided as an alternative suggestion.

17 hours ago, yggdrasil said:

it's not only for .uk but other ccTLDs as well

it will be for any TLD that eNom / LP don't sell... or at least should be, but who knows which other tlds that eNom sell, that NS is falling back on whois for.

if you were using external forms to pass a domain to the domain register page, then they could be easily logged with a hook, but logging NS directly in the cart would be a lot more involved from the outside.

[yggdrasil]

100% agree. A partial log is worse than no log at all since now you are basing your decisions on incorrect and false information. This would have been an interesting sales tool. Being able to market research what people search or even help potential customers but I guess it's another feature from WHMCS that works partially or is half broken.

Its not really helpful or useful at all in the current state. All it does it just fill up your SQL tables with junk data because it's not even registering the real TLD someone searched but just the alternative suggestion.

The Whois log like its name tells you and I incorrectly assumed for years, should register EVERY single domain search someone types on your WHMCS installation. Regardless of the source, API, Whois search or typing it on the domain search box directly. That is the whole idea of a log, to have centralized unified information that either helps you debug something or provides valuable information. With incorrect or partial data its just worthless. 😪😪😪😪

 

Edited July 20, 2020 by yggdrasil