Firefox 3 Security Blocker: Going In Deep

I've been using Mozilla's "Minefield" nightlies as my daily browser at home now for the better part of two weeks. These are the iterations that will eventually become the final Firefox 3 release.

Major issues with the Places bookmark handling, while not fully fixed, have been resolved enough to allow me to be comfortable with regular usage.

Only last night I ran into an unexpected "problem" and it is really frustrating me that I can't get more information!

Scanwith Saga

One of the websites I like to check in at periodically is ScanWith.com. This simple site provides links to freeware and $-ware software that is geared towards system-scanning. Process scanners, anti-virus scanners, malware-scanners, etc. While I don't download the files from this site (rather I go to the developer's site) it provides a great all-in-one stopping place for the latest and greatest in this class of tools.

So Friday morning I was doing a quick link-check for new utility leads before work and the site loaded great.

Friday night, after work, I fired up Firefox 3 (Nightly) and was hit in the face with this:

Apparently, someone thinks that ScanWith is an "attack site."

Bummer.

But why? And is it true or not? Have I put my own system at risk in my past visits?

Oh bother!

Claus has questions and he's out to take down names!

First Things First: Assessing the Threat

The first thing that I noticed was the frightful language and lack of options on the displayed page (Mozilla feature sample link here).

Notice there is no link here to allow you to click-through, despite your better judgement. That's normally a good thing, especially if your pc is used in a shared-user environment.

There are only two links: the "request a review" link which takes you to StopBadware.org.

Here website owners can register and plead their case to the courts.

However, despite all my attempts and searches through the "Clearinghouse" I could not locate any information on why this site got black-listed. Nothing. Nada. Zilch.

The only other link available on the site is the panic-soothing "Get me out of here!" button. That returns you to your browser's homepage.

There is no way to tell Mozilla that you wish to pass-through to the site anyway. You just can't do it.

If a site is blocked, it's blocked.

Just for kicks, I cross-checked ScanWith's URL against McAfee's SiteAdvisor for the site. It comes back clean and gets positive marks from this trusted anti-virus company. Long searches on Google turned up no additional information reported by other users.

I also left-clicked the icon to the left of the address-bar. In Firefox 3 this gets you a little drop-down mini-window that gives you some site-security information. If you want more, click the "More Information" button.

I did. While helpful, I suspect that the site's lack of supplying owner information and the fact that the connection isn't encrypted isn't what caused the alarm. This is a nice feature, but could be much more informative.

Second: The Workaround - It's all or nothin' baby!

Because there are no options at all here on the warning page to "whitelist" the site manually, there is only one option to get to the site(s) that are being blocked.

On the menu bar, go to Tools > Options... > and click on the Security icon at the top.

Now uncheck the "Tell me if the site I'm visiting is a suspected attack site."

Apply the changes and re-click.

You're in!

But now you've turned off the global-protection setting for this feature. It's not entirely clear to me, but I am pretty sure that you have also just turned off the anti-phishing feature also built into Firefox 3. So now the doors are open!

(Correction: as Asa Dotzler kindly points out in his comment, turning off the "attack warning" feature indeed does NOT turn off the anti-phishing protection in Firefox 3. In Firefox 3 there are two different control options as seen in the screenshots below. Firefox 2 only has an anti-phishing protection option so it wouldn't alert on an "attack designated" website, regardless. Thanks Asa!)

A Mozilla Dr. Jekyll and Mr. Hyde?

Actually it isn't really a "new" feature. Firefox 2.0.0.x also contains the functionality.

In Firefox 2.0.0.x menu-bar, go to Tools > Options and then click on the Security icon at the top.

Notice the options are very similar. You have the option to "Tell me if the site I'm visiting is a suspected forgery" and either "Check by using a downloaded list of suspected sites" or "Check by asking [Google] about each site I visit."

Notice below that I do have this feature enabled in Firefox 2.0.0.12 on my system.

As I do in Firefox 3.0.0.x (Minefield nightly)

However, when I load ScanWith.com in Firefox 2, it loads just fine with no interception or warning messages. When I load it with Firefox 3.0, it gets blocked.

The difference? Different download lists!

(Edit: Actually, this is true, but as Asa pointed out I'm comparing apples and oranges here. FF2's list is geared soley for checking for phishing sites. FF3's list (as I break into futher down) differentiates between "attack" sites and "phishing" sites. So this easily explains the differences in website blocking behavior I'm observing between the two, even though the lists are coming from Google.)

Because I am using the one from (presumably) Mozilla in 2.0 (and not downloading Google's) I'm cool. Because Firefox 3.0 uses Google's black-lists, "No soup for you!"

(Even more curiously, when I did toggle the "Check by asking Google..." option in Firefox 2, the site remained unblocked. Maybe I'm not waiting long-enough for the Google-list to be updated after toggling it on? I really don't know.)

More blockage screenshots: Firefox 3: Suspected Attack Site Screenshots « RJO Blog

Mozilla and Google and StopBadware...Sitting in a tree, k-i-s-s-i-n-g....

Wikipedia has some information on the relationship between Mozilla and Google for their anti-phishing/attack filtering:

The release of the anti-phishing protection in Firefox 2 especially raised controversy. By default, anti-phishing protection is enabled, based on a list that is updated by downloads to the user's computer about twice an hour from Google's server. The user cannot change the data provider within the GUI, and is not informed who the default data provider is. The browser also sends Google's cookie with each request for update. An additional, explicitly opt-in security feature has been added to recent builds by the Mozilla Foundation. This anti-phishing feature provides live protection by checking each visited URL with Google.

And the relationship between Google and StopBadware is pretty clear as well, Google's Webmaster Central blog: Better badware notifications for webmasters.

In the fight against badware, protecting Google users by showing warnings before they visit dangerous sites is only a small piece of the puzzle. It's even more important to help webmasters protect their own users, and we've been working on this with StopBadware.org. A few months ago we took the first step and integrated malware notifications into webmaster tools. I'm pleased to announce that we are now including more detailed information in these notifications, and are also sending them to webmasters via email.

The post comments were quite interesting to read and seemed to reflect some issues with getting "un-blacklisted" and the impact this event has on site-ranking.

It appears that the general process for getting tagged and blocked is this:

1. Google uses an undisclosed and proprietary method (magic?) to locate badware on websites.

2. Google passes this information to StopBadware.

3. Google adds a warning under the search-results links for sites it tags.

4. Mozilla's Firefox 3 downloads (periodically) updated lists from Google's servers and these are folded into the user's browser for protection.

Here is a current Google search result (as of this post's date) showing Google's identification on ScanWith.com.

You can still click through to their, if you are interested and not sufficiently frightened.

In theory, the website owner finds the badware on their page (embedded in the page-code or hosted for download), cleans up their webpage/website, petitions Google and StopBadware, and is released from the black-lists. More on this in a bit.

Pause for Review

So, at this stage of the game I had figured out that depending on which version of Firefox I was using, I was going to see different results. That the blacklist was provided by Google, and that any attempts by me (the public) to assess for myself what the threat was and make an informed decision to continue or not had been removed from my control in (this version) of Firefox 3 (Minefield).

So now I was OCD locked-on to this issue.

I had to know, what is going on under the Mozilla hood and could I manually "un-blacklist" a site?

Get out the Tools!

Out of curiosity, I decided to see if, perhaps, I could locate the file that was being checked against to see if a site is blacklisted.

I fired up Microsoft Sysinternal's Process Monitor and set a filter to exclude all results except the process firefox.exe.

I swapped over to Firefox and clicked the bookmark for Scanwith.com.

I then toggled back to Process Monitor and culled through the results. I pretty quickly located a file called "urlclassifier3.sqlite".

Only trouble was, it wasn't viewable in my Windows Explorer file list at the location claimed.

So back to Process Monitor. I right-clicked one of the lines with it listed and selected "jump-to."

Bam, a new Windows Explorer window opened up and I found the file buried in a super-hidden "OfflineCache" folder in my Firefox/Minefield profile folder.

Score!

Now. What goodies/baddies are in here?

The file-extension told me that it was a SQL "lite" database file.

I'm down with Access databases, but SQL isn't something I'm knowledgeable about. Some quick web-searching located what I needed: About SQLite.

Turns out it is basically an embedded SQL database engine which contains all the tables, indices, triggers and views..all self-contained in a single file. I can really see the benefits of using this from Mozilla's standpoint.

Unfortunately (for me and other prying novices) working with it demands knowledge and interaction with it from a command-line level. I'm pretty impatient. Anybody got GUI?

Yep: SQLite Database Browser. This freeware tool allows one to "to create, design and edit database files compatible with SQLite. It is meant to be used for users and developers that want to create databases, edit and search data using a familiar spreadsheet-like interface, without the need to learn complicated SQL commands." Awesome!

I downloaded the file from SourceForge and unzipped it. It is a single EXE file with no need for installation, click and run. My kind of program!

Once running I was able to dive into the blacklist files! Well, kind-of.

Welcome to urlclassifier2.sqlite versus urlclassifier3.sqlite

See, actually the first time I went hunting for urlclassifier3 on my drive, I didn't pay attention to the full path location.

I ended up first working with urlclassifier2.sqlite which I did find plainly in the root of my Firefox (Minefield) profile. This is the same location and file used by Firefox 2 builds.

When I opened it up with SQLite Database Browser I found the following structure:

goog_black_enchash - details on encrypted hash format can be found here
goog_black_url - contains blacklisted URLs of suspected phishing sites
goog_white_domain - contains whitelisted hostnames as determined by Google
goog_white_url - not currently used

When I eventually realized my mistake, I went back and found the correct one - urlclassifier3.sqlite.

moz_classifier - hashed domain and chunk_id's (my current one lists 18453 records).
moz-subs - hashed domain and chunk_id's (my current one lists 18453 records).
moz-tables - contains four names; test-malware-simple, test-phish-simple, goog-malware-shavar (with add_chunks and sub_chunks values), and goog-phish-shavar (with more add_chunks and sub_chunks values).

Turns out that with a bit of work, you can actually and easily "decode" most of the URL data in the urlclassifier2.sqlite file.

John Oberheide and the oiepoie blogs both have fantastic information on the Firefox 2 blacklist file

• jon.oberheide.org - blog - google safe browsing

• oiepoie » Sniffing Google’s blacklist for firefox

Here's how I figured out how to decrypt them using John's information:

1. Using my new friend SQLite Database Browser, I exported both the goog_black_url and goog_white_domain files as separate text files to my desktop.

2. If opened in notepad or another application, they appear to have a URL-like format, but are unreadable.

3. The trick here is knowing (from John and Mozilla's posts) that they are actually in a ROT13 encryption format.

4. If you open either of the exported files in NOTEPAD++ or EditPad Lite (thanks for the tip TinyApps!) both have a "convert ROT13" function. Run that function and there are the URL's in their normal glory! Or if you just want to pick out one or two, use an on-line ROT13 converter,

Explore to your heart's content. I ran a search for Scanwith.com against these lists, and didn't find it. Not that I expected to, as Firefox 2 doesn't block Scanwith.com, despite checking that urlclassifier2.sqlite file in my Firefox 2 profile folder was only minutes old when examined and clearly from Google's own servers.

Making Mozilla Hash

The goog_black_enchash I never could work out how to decrypt. Not that it's a big secret.

I was able to follow the steps provided in Mozilla's Wiki to create several MD5 hashes for scanwith.com. I figured, if I couldn't decode them, maybe I could encode it and compare the hash against those hashes in the list.

So I ran the following URLS through the ropes: www.scanwith.com and scanwith.com the hashing process:

1. Based on the Mozilla Wiki page I take the current database salt "oU3q.72p" and concatenate it with the sub-hostname string, and uppercase the results, thusly:

   • oU3q.72pwww.scanwith.com

   • oU3q.72pscanwith.com

2. I then computed the MD5 hash of each item:
   • 55D5ADE94D9C39E4EA9EC58E932EE62A

   • C4590E8D8FABD03ED33F3C4090DE4050

3. I then ran a search for each of these strings in the exported table, but no hits.

Again, I didn't expect any as Firefox 2 doesn't block the site, so they shouldn't be there.

I wish I could have decoded all the URL's listed in the goog_black_enchash export file. However, I'm not an advanced script-writer or a noobie cryptanalyst. So I was quickly lost following Mozilla's instructions on how to decode the data using base64 encoding, along with the "database salt, random salt, and hostname. You also have to toss some RC4 into this mix (Here is a java-based RC4 tool.) If anyone reading this has those skills and can write something to help, or provide steps for the witless like me to follow, it would be nice.

Why all the Obfuscation?

Simple. No dark secrets; it's just Mozilla's attempt to keep anti-malware scanners from alerting on the bad URL's contained in their database file.

Unfortunately, I was unable to find ANY documentation on the structure and format Mozilla is using for their revised URL checking file urlclassifier3.sql. Too bad for me.

It appears to actually be designed to function pretty coolly. From the very limited information I have, the Firefox browser is able to request small-sized database update "chunks" from Google's server to help with bandwidth utilization on the network. This way it can perform frequent "micro-updates" to avoid some earlier performance issues with this updating process in Firefox 3 (Betas). I'm guessing that the add_chunks are blacklisted URLs being added to the database while sub_chunks are coming in to remove URL's now deemed safe.

I still wonder if it would be possible, were a particular URL to be identified, if someone with enough technical knowledge could go in and, using a SQLite tool" delete the offending record out of the database or modify it. Of course, if I could, then I have no doubts that an attacker could use this as a vector to remove "true" listed phishing/attack site URLS or even falsely seed it with legitimate websites. Probably not really a useful attach vector, but certainly worth considering.

Full Circle - Back to the Beginning

So, by 2 AM this morning I decided I had squeezed all the blood I could get from this stone.

Here's where I leave off with what I know:

• Mozilla is currently using Google's malware-ranked sites in it's anti-phishing/anti-attack URL database.

• And, no (at this time) you cannot "technically" use any other such ranking website (excepting an Add-ons solution): Bug 342188 – support changing the local list data provider

Jesse Ruderman 2007-11-17 16:51:06 PST Comment #3

How should the UI for this work?  Should Firefox ship with a predefined list of
phishing/malware site list providers, like we do for search engines?  For ones
that aren't on the list, should there be a way for users to add them?

I imagine we could get one or more of McAfee SiteAdvisor, Netcraft, or GeoTrust
to provide their data in the correct format if we offered to add them as
options.

Mike Beltzner 2007-11-20 08:25:50 PST Comment #4

(In reply to comment #3)
> I imagine we could get one or more of McAfee SiteAdvisor, Netcraft, or GeoTrust
> to provide their data in the correct format if we offered to add them as
> options.

You'd imagine incorrectly. We've asked and extended offers to all of those
companies, and been rebuffed.

This doesn't block, but is wanted. The best way to do it, I'd imagine, would be
to allow alternative providers to register themselves similar to how feed
readers and other web apps do, providing the endpoints for the Safe Browsing
protocol. The UI would be a drop-down of available providers in the prefPane.

If we don't get it for Firefox 3, alternative providers can still support
themselves by creating an add-on.

• At least in the case of Scanwith.com, StopBadware and Google provide no way for the public to find what triggered the alert "attack" ranking blocking the website.
• My unprofessional review of the site's page-code doesn't find anything alarming. I am curious if any experts could verify this.
• My "gut" feeling is that there may be one or more programs available for download on the site that could be "potentially" used for ill-purposes/hacking against a system or network. Maybe the availability alone got them hit?
• StopBadware swears that a site can't be listed in error or for no good reason.
• Firefox 2's current urlclassifier2.sqlite file doesn't blacklist the site, Firefox 3's urlclassifier3.sqlite file does. Does anyone see an issue with consistency here? Hard to know who and what to trust now.
• Mozilla doesn't (currently) offer any options for Firefox 3 (Minefield) users to click-through on their own to their desired website. We can only entirely disable the protection globally to get to the site, or leave it engaged and give up or switch to a different browser.
• Mozilla's own developers have in the past openly wrestled with the philosophic issues around the Phishing/Attack protection design problem:
  

Warning Dialog UI

The hard part of a feature like this is not identifying phishing pages, but rather figuring out how to present this information to users. With this in mind, Google did usability testing of a multitude of potential UIs. In the process we learned:

• When a phishing warning is encountered, users have three primary reactions: a strong desire to continue to use the page, a near-panicked desire to get away from the page, and a desire to see the villains brought to justice. In cases where users heeded the warning, they typically also needed a sense of closure, so it's important to give it to them (hence the "report to google" option).
  
• We originally showed the warning only when the user began interacting with a problematic page, but it quickly became clear that users prefer we just disable the page as it is loading.
  
• Warnings of "suspicious" pages (pages we're not positive are phishing, but that might be) were by and large ineffective. Users either ignored the warning or said they'd prefer us to make a judgment for them, and that they'd trust us on it. Such warnings probably don't give the user enough information to go on; perhaps they'd be more effective if accompanied by a way for users to do a "background check" on the page.
  

<snip>....This is by no means final; we're still doing testing. Additionally there are obvious branding issues here that need to be considered. But this is the basic idea.

For those of you who have stuck this post out and are still with me, I'm NOT complaining or questioning either Google or StopBadware's mission and efforts to try to keep the web a safer-place for its citizens. God Bless Them for working hard and establishing some sort of review and process to help folks stay safe as they search and surf the web.

Same goes for Firefox. For the most-part, I applaud Mozilla's efforts to integrate a phishing/attack filter on their wonderful browser.

I am strongly complaining about the lack of information than both StopBadware and Google provide to help us gauge for ourselves the specific reason(s) for the blacklist rating provided. As well as the inconstancy in filtering results depending on which of their browsers I use.

I'm not trying to erroneously defend ScanWith here. They may or may not be hosting malware on their site. They may or may not just find themselves blacklisted due to the content of programs they host for download. That's my point. I can't say and no-one seems to be able to or is willing to tell me (and the public at large).

I also regret that Mozilla doesn't have some kind of mechanism in place for me to click-through anyway to get to the website in Firefox 3. Even if it would take a discouragingly painful amount of prompts to do so would be fine (like how Firefox requires me to add some Add-on developer's websites to an internal white-list first when downloading an XPI file from a non-Mozilla hosted website).

I have learned how to view many of the files blacklisted/whitelisted by Firefox 2. I really would like to be able to do the same thing for the ones in Firefox 3's urlclassifier3 file. (Nir, are you thinking what I am hoping for?)

Despite all this, I am pleased that I have now gained an deeper understanding of Mozilla's use of SQLite database files and found the SQLite Database Browser tool. Poking around in additional sqlite files in my Mozilla profiles with the utility provided some interesting information as to what and how Mozilla stores user data. Cool stuff.

More Information on the Mozilla Phishing (and Attack) Protection

• Phishing Protection - MozillaWiki
• Phishing Protection: Design Documentation - MozillaWiki
• Phishing Protection: Client Spec - MozillaWiki
• Phishing Protection: Server Spec - MozillaWiki
• Adding phishing protection data providers - Mozilla Developer Center

If anyone is more familiar with these issues and can leave clarifications or corrections, comments would be appreciated. Just in case I'm out in left-field here singing to myself.

--Claus