28,000 users with 'firstname' field altered, no support response.

[Phobic]

Due to the exploit spread today by WHMCS, and the fact that no email has even arrived yet warning us of this absolutely massive, elementary mistake by the WHMCS team, our database was exploited earlier this evening, with 28,000 accounts having their firstname field authored, and then us receiving an email containing all md5 passwords that were stored on our system.

 

We contacted support, who asked us to update our whmcs installation. We specified in our first mail that we updated immediately after being made aware of this exploit, following the attack carried out.

 

Considering an exploit at the level of a full database scrape being made available is something that would bring an absolute end to almost every whmcs-powered business (and whmcs itself), I expected a more thorough, and professional response - at least one at a level that would imply our ticket was actually read and acknowledged.

 

What's actually happening at WHMCS at the moment? Are people drunk at the wheel?

 

And I would also strongly recommend everyone who has a WHMCS account to change their password immediately - if db scraping is as easily carried out by the released script, you can guarantee that the WHMCS database is in the hands of several people at this stage.

[Phobic]

With this, we have also found that every single 'password' column in our database was also altered.

 

From reading through the code on the exploit, the code for this exploit should never, ever have existed in the first place. It was either put in place by a Computer Scientist on his first day in college, or put in as a backdoor for WHMCS. I would hedge my bets on the latter.

 

I would expect WHMCS to find themselves as destroyed as the companies affected by this - if not by reputation, than by legal action from the business owners who found themselves shut down tonight.

[Blueberry3.14]

I do share your frustration (in theory), and I figured some new exploit was in the wild, since we've noticed a massive and sudden surge of a new exploit attempt on our WHMCS install. None were successful, however, and every single one of them blocked correctly by Mod_Security. Do you have Mod_Sec?

 

Also, and I know you're already frustrated to the edge, and I get that (I've been there too), but do you have backups of your database? You should, daily ones. You can restore from a backup, all is not lost.

Edited October 3, 2013 by Blueberry3.14

[Phobic]

Our backup server went down due to HDD Failure - we're around a week and a half without backups, but we're still trying to recover data from our old hard drive. Due to the nature of the failure, the RAID 1 backup did us no good - as corrupted data got mirrored on the second drive.

 

While we're working on getting this back, there's no quick fix. We do have Mod_Sec running and fully up to date; so I don't know why that didn't kick into action. The script kiddie in question actually contacted me, demanding money for a fix; after informing him he didn't use a proxy, and that I'm aware of what script he used, he began apologising. He's in the UK, so pursuing charges against him is a possibility.. but as he appears to be a juvenile (who admitted he didnt know what the script would actually do), I doubt that'll go far. And it definitely won't assist our customers either.

 

Right now the only solution I can really consider is mass-resetting all passwords to a random string, emailing on the updated password along with information on the attack to our clients, and consider a new billing platform. If you read through the exploit, it's very explicit on accepting AES_Encrypt data - it really looks like an intentional backdoor. I can't see any use for it otherwise.

[Blueberry3.14]

Damn. When it rains, it pours. I agree, a mass-resetting of passwords is probably your best bet for a first step.

[SeanP]

May I ask if you are using the branded or unbranded version of WHMCS? I invested in the unbranded version, because I always felt it was way too easy for people to search "Powered By WHMCS" and possibly find your implementation of the software. Whenever there is a security vulnerability like this, I'm sure that is one of the main ways people find WHMCS installs. I'm sorry this has happened to you. I feel bad for you.

[searley]

May I ask if you are using the branded or unbranded version of WHMCS? I invested in the unbranded version, because I always felt it was way too easy for people to search "Powered By WHMCS" and possibly find your implementation of the software. Whenever there is a security vulnerability like this, I'm sure that is one of the main ways people find WHMCS installs. I'm sorry this has happened to you. I feel bad for you.

 

 

Dont think this makes much difference, most hackers/spammers/fraudsters find us by searching for a file nam like submitticket.php or clientarea.php or any file name that is almost exclusive to whmcs

[Shazan]

I do share your frustration (in theory), and I figured some new exploit was in the wild, since we've noticed a massive and sudden surge of a new exploit attempt on our WHMCS install. None were successful, however, and every single one of them blocked correctly by Mod_Security. Do you have Mod_Sec?

Could you share the mod_security rule(s) that blocked the attempts to use this exploit?

 

Thanks.

[Alex - Arvixe]

Dont think this makes much difference, most hackers/spammers/fraudsters find us by searching for a file nam like submitticket.php or clientarea.php or any file name that is almost exclusive to whmcs

 

Many find it via the powered by line branding. I don't believe it is common for an attack to be targeted via specific URLs which may or may not be exclusive to WHMCS.

 

@OP - Very sorry to hear of your situation, the fact this happens as well as your backups not functioning is the worst that can happen. Good luck in this situation!

[paperweight]

I downloaded the patch yesterday and installed it about 90 seconds after WHMCS posted the info on their Twitter feed. But I was lucky because I was monitoring everything and awake at the moment.

 

WHMCS's email to me about the patch was received in my email box about 7 hours after the Twitter post went up. So had I only been monitoring my email, I would too be screwed

 

Most of our operations and development staff are in countries that have Twitter blocked, so by default we do not usually monitor Twitter -- I really wish WHMCS could up the ante on their email deliveries and get a faster SMTP to get these vital messages out earlier!

[ReflexWeb]

Hi all, I didn't read the email from WHMC until this morning 04/10/2013 applied patch straight away now shows version 5.2.8

 

Short while later i get a n email -

 

Subject = WHMC User Details Change

Body = Client ID: 113 - Brian Catallo has requested to change his/her details as indicated below:

 

Client ID: 114 - Brian Catallo has requested to change his/her details as indicated below:

 

First Name: 'Brian' to 'AES_ENCRYPT(1,1), firstname=(SELECT GROUP_CONCAT(id,0x3a,username,0x3a,email,0x3a,password SEPARATOR 0x2c20) FROM tblclients where id = 1)'

Last Name: 'Catallo' to '1'

Company Name: '' to '1'

Address 1: '1827 Westfall Rd' to '1'

Address 2: '' to '1'

City: 'Rochester' to '1'

State: 'North Carolina' to '1'

Postcode: '14618' to '1'

Phone Number: '3363549555' to '1'

Default Payment Method: '' to ''

 

If you are unhappy with any of the changes, you need to login and revert them - this is the only record of the old details.

 

FOLLOWED BY another

 

Client ID: 114 - AES_ENCRYPT(1,1), firstname=(SELECT GROUP_CONCAT(id,0x3a,username,0x3a,email,0x3a,password SEPARATOR 0x2c20) FROM tblclients where id = 1) 1 has requested to change his/her details as indicated below:

 

First Name: 'AES_ENCRYPT(1,1), firstname=(SELECT GROUP_CONCAT(id,0x3a,username,0x3a,email,0x3a,password SEPARATOR 0x2c20) FROM tblclients where id = 1)' to 'AES_ENCRYPT(1,1), firstname=(SELECT * FROM (SELECT COUNT(id) FROM tblclients) as x)'

Default Payment Method: '' to ''

 

If you are unhappy with any of the changes, you need to login and revert them - this is the only record of the old details.

 

 

AND SO ON

 

I have .htaccess on my admin folder

I have updated my WHMC admin login details

I have updated my WHMC database passwords

 

The patch for 5.2.7 to 5.2.8 obviously didnt stop the above from happening, server sad that WHMCS have not responded to my support ticket about this!!!!!!!!

[SeanP]

Dont think this makes much difference, most hackers/spammers/fraudsters find us by searching for a file nam like submitticket.php or clientarea.php or any file name that is almost exclusive to whmcs

 

Sure, they can always find WHMCS some way, but I don't want a sign on website saying "I'm running WHMCS!!!"

[Phobic]

A very long night indeed.

 

I pointed out to WHMCS that we could grab firstname data back from all customer's welcome mails - and they assisted in writing up a script for that, which thankfully assisted me in getting some sleep. WHMCS also assisted me in writing up a mass password reset script.. which would have actually crashed out php as it was unthrottled, so after writing up some quick throttle control for it, everything went well.

 

However, from a quick twitter search, I can see a lot of hosts got people slightly sharper than the script kiddies who found us - entire databases getting scraped, all tables getting dropped etc. Still a catastrophic mistake on WHMCS's part; a lot of businesses came to a swift end last night, due to what I'm still having trouble seeing as being anything more than a back door. Very disheartening

 

For those asking - we use unbranded WHMCS.

[Darthkatzs]

Getting pushed around by support, saying "they do not know of any backdoor" and completely ignoring my question as to what the function of that code was then. "We take security very seriously", yet they have exploits that even a 14 year old would notice.

[bear]

Many find it via the powered by line branding. I don't believe it is common for an attack to be targeted via specific URLs which may or may not be exclusive to WHMCS.

Starting a few hours after this first broke, I'd been seeing lines in the logs of someone looking for all the main WHMCS files programmatically (a second or two apart) from single IPs. We'd already disabled access at that point.

 

Doesn't take long to build a list of targets for a closer look based on URLs.

 

To the OP:

Sorry to hear you were hit with this, but wanted to remark on the apparent growth of your customer base. You'd posted in March you were dealing with "a few thousand" clients, and here you are mentioning 28K around half a year later. Pretty awesome growth there.

[ditto]

WHMCS's email to me about the patch was received in my email box about 7 hours after the Twitter post went up. So had I only been monitoring my email, I would too be screwed

 

I don't know why, but I never ever receive these emails from WHMCS, I have not received any of those emails in years. The only emails I receive from WHMCS is replies to my support tickets and invoice emails. All extra emails they send out when things like this happen, I never receive those.

 

Instead I subscribe to the RSS feed in blog.whmcs.com - but I wonder why I never receive the mass emails from them ... not for years ...

[zoilodiaz]

Well since WHMCS a HOLE bug, we have to keep entering in the forum every day and check the blog every day.

[Blueberry3.14]

WHMCS's email to me about the patch was received in my email box about 7 hours after the Twitter post went up. So had I only been monitoring my email, I would too be screwed

 

Admittedly I haven't gone back to check the times that the blog post was written verses Twitter (I've given up on expecting anything timely from WHMCS via Twitter, it figures that once I do that they come through), but I get their blog via RSS. Often that's faster than waiting for an email, since emails are sent in batches.

 

You could go even further and use IFTTT to set up an alert (i.e. send a notification to your phone via SMS if WHMCS makes a blog post with certain words, like security issues, etc). IFTTT rocks for that.

[Blueberry3.14]

Could you share the mod_security rule(s) that blocked the attempts to use this exploit?

 

Thanks.

 

Surely.

 

SecRule REQUEST_HEADERS:User-Agent "(?:\b(??:indy librar|snoop)y|microsoft url control|lynx)\b|d(?:ownload demon|isco)|w(?:3mirror|get)|l(?:ibwww|wp)|p(?:avuk|erl)|cu(?:sto|rl)|big brother|autohttp|netants|eCatch)" \
       "chain,log,auditlog,msg:'Request Indicates an automated program explored the site',id:'990011',severity:'5'"

 

and

 

# SQL injection
SecRule REQUEST_FILENAME|ARGS|ARGS_NAMES|REQUEST_HEADERS|XML:/*|!REQUEST_HEADERS:Referer "@pm insert xp_enumdsn infile openrowset nvarchar autonomous_transaction print data_type or outfile inner shutdown tbcreator @@version xp_filelist sp_prepare sql_longvarchar xp_regenumkeys xp_loginconfig xp_dirtree ifnull sp_addextendedproc xp_regaddmultistring delete sp_sqlexec and sp_oacreate sp_execute cast xp_ntsec xp_regdeletekey drop varchar xp_execresultset having utl_file xp_regenumvalues xp_terminate xp_availablemedia xp_regdeletevalue dumpfile isnull sql_variant select 'sa' xp_regremovemultistring xp_makecab 'msdasql' xp_cmdshell openquery sp_executesql 'sqloledb' dbms_java 'dbo' utl_http sp_makewebtask benchmark xp_regread xp_regwrite"         "phase:2,t:none,t:urlDecodeUni,t:htmlEntityDecode,t:replaceComments,t:compressWhiteSpace,t:lowercase,pass,nolog,skip:1,id:1500007"
SecAction phase:2,pass,nolog,id:999501,skipAfter:959001
SecRule REQUEST_FILENAME|ARGS|ARGS_NAMES "(?:\b(??(?:elect\b(?:.{1,100}?\b(??:length|count|top)\b.{1,100}?\bfrom|from\b.{1,100}?\bwhere)|.*?\b(?(?:ump\b.*\bfrom|ata_type)|(?:to_(?:numbe|cha)|inst)r))|p_(??:addextendedpro|sqlexe)c|(?:oacreat|prepar)e|execute(?:sql)?|makewebtask)|ql_(?:longvarchar|variant))|xp_(?:reg(?:re(?:movemultistring|ad)|delete(?:value|key)|enum(?:value|key)s|addmultistring|write)|e(?:xecresultset|numdsn)|(?:terminat|dirtre)e|availablemedia|loginconfig|cmdshell|filelist|makecab|ntsec)|u(?:nion\b.{1,100}?\bselect|tl_(?:file|http))|group\b.*\bby\b.{1,100}?\bhaving|d(?:elete\b\W*?\bfrom|bms_java)|load\b\W*?\bdata\b.*\binfile|(?:n?varcha|tbcreato)r)\b|i(?:n(?:to\b\W*?\b(?:dump|out)file|sert\b\W*?\binto|ner\b\W*?\bjoin)\(?:f(?:\b\W*?\(\W*?\bbenchmark|null\b)|snull\b)\W*?\()|a(?:nd\b ?(?:\d{1,10}|[\'\"][^=]{1,10}[\'\"]) ?[=<>]+|utonomous_transaction\b)|o(?:r\b ?(?:\d{1,10}|[\'\"][^=]{1,10}[\'\"]) ?[=<>]+|pen(?:rowset|query)\b)|having\b ?(?:\d{1,10}|[\'\"][^=]{1,10}[\'\"]) ?[=<>]+|print\b\W*?\@\@|cast\b\W*?\()|(?:;\W*?\b(?:shutdown|drop)|\@\@version)\'(?(?:qloledb|a)|msdasql|dbo)')" \
       "phase:2,capture,t:none,t:htmlEntityDecode,t:replaceComments,t:compressWhiteSpace,t:lowercase,ctl:auditLogParts=+E,log,auditlog,msg:'SQL Injection Attack',id:'950001',tag:'WEB_ATTACK/SQL_INJECTION',logdata:'%{TX.0}',severity:'2'"
SecRule REQUEST_HEADERS|XML:/*|!REQUEST_HEADERS:Referer "(?:\b(??(?:elect\b(?:.{1,100}?\b(??:length|count|top)\b.{1,100}?\bfrom|from\b.{1,100}?\bwhere)|.*?\b(?(?:ump\b.*\bfrom|ata_type)|(?:to_(?:numbe|cha)|inst)r))|p_(??:addextendedpro|sqlexe)c|(?:oacreat|prepar)e|execute(?:sql)?|makewebtask)|ql_(?:longvarchar|variant))|xp_(?:reg(?:re(?:movemultistring|ad)|delete(?:value|key)|enum(?:value|key)s|addmultistring|write)|e(?:xecresultset|numdsn)|(?:terminat|dirtre)e|availablemedia|loginconfig|cmdshell|filelist|makecab|ntsec)|u(?:nion\b.{1,100}?\bselect|tl_(?:file|http))|group\b.*\bby\b.{1,100}?\bhaving|d(?:elete\b\W*?\bfrom|bms_java)|load\b\W*?\bdata\b.*\binfile|(?:n?varcha|tbcreato)r)\b|i(?:n(?:to\b\W*?\b(?:dump|out)file|sert\b\W*?\binto|ner\b\W*?\bjoin)\(?:f(?:\b\W*?\(\W*?\bbenchmark|null\b)|snull\b)\W*?\()|a(?:nd\b ?(?:\d{1,10}|[\'\"][^=]{1,10}[\'\"]) ?[=<>]+|utonomous_transaction\b)|o(?:r\b ?(?:\d{1,10}|[\'\"][^=]{1,10}[\'\"]) ?[=<>]+|pen(?:rowset|query)\b)|having\b ?(?:\d{1,10}|[\'\"][^=]{1,10}[\'\"]) ?[=<>]+|print\b\W*?\@\@|cast\b\W*?\()|(?:;\W*?\b(?:shutdown|drop)|\@\@version)\'(?(?:qloledb|a)|msdasql|dbo)')" \
       "phase:2,capture,t:none,t:urlDecodeUni,t:htmlEntityDecode,t:replaceComments,t:compressWhiteSpace,t:lowercase,ctl:auditLogParts=+E,log,auditlog,msg:'SQL Injection Attack',id:'959001',tag:'WEB_ATTACK/SQL_INJECTION',logdata:'%{TX.0}',severity:'2'"
SecRule REQUEST_FILENAME|ARGS|ARGS_NAMES "\b(\d+) ?= ?\1\[\'\"](\w+)[\'\"] ?= ?[\'\"]\2\b" \
       "phase:2,capture,t:none,t:htmlEntityDecode,t:replaceComments,t:compressWhiteSpace,t:lowercase,ctl:auditLogParts=+E,log,auditlog,msg:'SQL Injection Attack',id:'950901',tag:'WEB_ATTACK/SQL_INJECTION',logdata:'%{TX.0}',severity:'2'"
#
SecRule REQUEST_FILENAME|ARGS|REQUEST_HEADERS|XML:/*|!REQUEST_HEADERS:Referer "@pm user_objects object_type substr all_objects mb_users column_name rownum atttypid substring object_id user_group user_tables pg_attribute user_users column_id user_password attrelid object_name table_name pg_class"         "phase:2,t:none,t:urlDecodeUni,t:htmlEntityDecode,t:replaceComments,t:compressWhiteSpace,t:lowercase,pass,nolog,skip:1,id:1500008"

 

That's assuming those hitting my WHMCS install are using the same exploit as others are experiencing. In looking for the code that's being used on Pastebin, I found number of similar ones, and another WHMCS exploit being sold for $6k on another site, etc. This latest one that our server is being hit with is different from previous Injection attempts, but the above Mod_Sec rules catches them.

 

EDIT: The ones hitting us are probably not the same as the OP's. Ours didn't even try to register as a user first, and it's my understanding that's required for the 5.2.7 exploit to work. Then again, they just be not too bright.

 

There's also the template hack that this one stops:

 

SecRule ARGS|ARGS_NAMES "@validateByteRange 1-255" \
"deny,log,auditlog,msg:'Invalid character in request',id:'960901',severity:'4',t:urlDecodeUni,phase:2"

 

And my .02 on the branding vs non-branding debate.... I think non-branding helps, it's worth the additional $5 a month. It weeds out a few low-level, lazy script kiddie/Google Dorkers who use

intext:"Powered by WHMCompleteSolution" OR inurl:"submitticket.php‎"‎

, but the "submitticket.php" is still going to be found.

 

The more popular any software application gets, the more it's going to be targeted. WordPress is a prime example.

 

EDIT: The above Mod_Security rules will help with standard SQL Injection attempts, but are UNTESTED against the current exploit that changes user names using AES_ENCRYPT.

 

 

This rule may help (credit goes to Patrick of Rack911 )

 

SecRule REQUEST_URI|ARGS|REQUEST_BODY "AES_ENCRYPT" "id:31337,phase:4,log,deny,msg:'WHMCS Fail'"

 

Just make sure you don't have another rule by the same number.

Edited October 4, 2013 by Blueberry3.14

[Blueberry3.14]

Since having a valid login/user account to a WHMCS is critical to this exploit being run, I'm surprised WHMCS isn't recommending everyone disable client registration ("Other" tab, Allow Client Registration, Tick this box to allow registration without ordering any products/services). It's not foolproof, because they could still place and order and register, but it's a lot less drastic than taking your entire billing system offline, like I'm seeing recommended in many forums.

 

I applied the patch yesterday as soon as it was released, but I've also disabled new client registration for now.

[brian!]

I don't know why, but I never ever receive these emails from WHMCS

I think that I usually do, but I still haven't received one about this issue - I was following the forum when the initial post appeared, so I was able to quickly upgrade, but still waiting for their email telling em to do so! (unless WHMCS are so on the ball that they are aware of my upgrade and feel it not relevant to tell me!)

[Blueberry3.14]

I think that I usually do, but I still haven't received one about this issue - I was following the forum when the initial post appeared, so I was able to quickly upgrade, but still waiting for their email telling em to do so! (unless WHMCS are so on the ball that they are aware of my upgrade and feel it not relevant to tell me!)

 

Do you have *@whmcs.com whitelisted?

 

from:	 WHMCS <noreply@whmcs.com> via messaging-master.net 
reply-to:	 noreply@whmcs.com
to:	 <removed>
date:	 Thu, Oct 3, 2013 at 1:09 PM
subject:	 Security Advisory Announcement
mailed-by:	 messaging-master.com
signed-by:	 messaging-master.net

 

They've changed broadcast email services, but I only have *@whmcs.com whitelisted, and have never had a problem receiving their emails.

[brian!]

thanks for the info - I have got whmcs.com whitelisted - it's the same address that all the forum thread updates use.

 

however, going to messaging-master website redirects to critsend.com... and there is an email in the server logs from critsend.com, but it got rejected as junkmail - "JunkMail rejected - sender85.critsend.com [xxx.xxx.xxx.xxx]:48924 is in an RBL, see Blocked - see http://www.spamcop.net/bl.shtml?xxx.xxx.xxx.xxx"

 

so it was sent, but got rejected before getting to the whitelist - i'll have to look into this further to ensure that we receive future announcements... thanks again.

[Blueberry3.14]

thanks for the info - I have got whmcs.com whitelisted - it's the same address that all the forum thread updates use.

 

however, going to messaging-master website redirects to critsend.com... and there is an email in the server logs from critsend.com, but it got rejected as junkmail - "JunkMail rejected - sender85.critsend.com [xxx.xxx.xxx.xxx]:48924 is in an RBL, see Blocked - see http://www.spamcop.net/bl.shtml?xxx.xxx.xxx.xxx"

 

so it was sent, but got rejected before getting to the whitelist - i'll have to look into this further to ensure that we receive future announcements... thanks again.

 

Oy. I didn't check to see if messaging-master redirected to anything, I was just happy to see it was something other than critsend.com.

 

From my support ticket of Oct 5, 2012:

 

Considering all the security events over the past 6 months, don't you think it would be wise 
to have any email, much less a security alert, showing as sent from an @whmcs.com address???
If you're using a 3rd party mailer, it should be configured to be coming from @whmcs.com.


Return-Path: <nicolas@critsend.com>
Received: from sender63.critsend.com (sender63.critsend.com. [95.130.9.91])
by mx.google.com with ESMTP id ba6si348854wib.21.2012.10.05.12.07.27;
Fri, 05 Oct 2012 12:07:27 -0700 (PDT)

 

So yes, they *did* change the "from" address...but it still doesn't help when you use a 3rd party mailer who's blacklisted. Just an FYI to any WHMCS employees who may read this, MailChimp runs a tight ship and fights hard to stay clean and not blacklisted.

[Soundlover]

How do I change back the first name from DISASTER?

I have still whmcs because I bought an expensive theme that cannot use with anything else.