Firefox Malware Detection Download Monitoring: Thoughts

As is usually the case, here I was all set to toss up some linkfest posts and I got seriously sidetracked.

It started like this.

1. Boot system, go into RSS feed reader.
2. Discover new rounds of updated utilities and apps.
3. Download said updated utilities and apps using Firefox
4. Spend Saturday unpacking them and updating portable USB tool library.

Only a little issue tripped me up and down I went chasing the White Rabbit.

See, I use the Firefox Add-on Download Status Bar to help me manage and mind my downloads since the dinky little stock download indicator in Firefox isn’t helpful to me.

When I’ve got all the files addressed and stowed away, I clear the bar.

Only I had a file that wouldn’t clear off no matter what. NirSoft’s PasswordFox

Eventually I realized that the file info was reading “0K” for the size which was strange. It was almost like it didn’t actually download…thus the Download Status Bar app couldn’t remove the reference from itself…since it didn’t actually exist.

I opened up my download manager directly in Firefox and found this:

OK. Clearing that item then allowed me to successful clear the Download Status Bar item showing.

But why was Firefox suddenly blocking this download? I didn’t recall having that issue within the last few weeks.

Turns out (starting with release version 31) that Mozilla has baked in some more sheeple protection features to keep the average user safe from malware/attack sites, etc.

• Improving Malware Detection in Firefox - Mozilla Security Blog. Quoting from Sid Stamm’s post:

Until recently, we only had access to lists of reported malicious web sites, now the Safe Browsing service monitors malicious downloaded files too. The latest version of Firefox (as of July 22) will protect you from more malware by comparing files you download against these lists of malicious files, and blocking them from infecting your system.

The next version of Firefox (released in September) will prevent even more malicious downloads on Windows. When you download an application file, Firefox will verify the signature. If it is signed, Firefox then compares the signature with a list of known safe publishers. For files that are not identified by the lists as “safe” (allowed) or as “malware” (blocked), Firefox asks Google’s Safe Browsing service if the software is safe by sending it some of the download’s metadata.

• Security/Features/Application Reputation Design Doc - MozillaWiki. From that background section:

Google has offered an application reputation feature to detect malicious downloads as part of Google Safe Browsing since 2012 [1]. Although this part of the Safe Browsing API is not documented, they have offered it to us for use in Firefox. Malicious download detection is separate from detection of phishing and malware pages (present in Firefox since 2.0), though both features use some of the same mechanisms.

This document attempts to document all of the things that Google Chrome does, so that even in the absence of official API documentation from Google, we collectively have a better chance of implementing this feature correctly.

I’m not going to go into a consideration if this is a good or bad thing. Let’s just leave it at “thanks for caring” and move on to how we can tweak it for those of us daring users who like to fiddle around with Firefox.

Basically, what I would like to do is turn off just the feature that does the file download check, but leave (for now) the other safe-browsing features enabled.

OK. There are a couple of posts that tell us how to do that:

Security/Features/Application Reputation Design Doc - How to turn off this feature - MozillaWiki

Do any one of the following:

• Turn off malware detection in Preferences > Security > "Block reported attack sites." This disables all Safebrowsing malware protection, including the warning interstitial that appears when the user navigates to a malware site.
• Replace browser.safebrowsing.appRepURL in about:config with an empty string. This disables application reputation checks but leaves other Safebrowsing malware protection intact.

Download files more safely with Firefox 31 - Monica at Mozilla. Monica guides us to use the Firefox options panel --> Security preferences, and then disable either “Block reported attack sites” and/or “Block reported web forgeries”. Note that doing this will remove almost all safe-browsing protection, rather than just disabling the (download) impacting application reputation check.

So I checked by about:config and didn’t find “browser.safebrowsing.appRepURL” so I added it and set the value to blank.

Similar advice/discussions for that key found here:

• How to prevent Firefox from sending downloaded file information to Google - gHacks Tech News
• Unable to save or download files - Disable virus scanning in Firefox preferences - Windows - MozillaZine Knowledge Base
• tonysilva/firefox-about-config · GitHub
• [Fx31] browser.download.manager .scanWhenDone preference is no longer available | What to do if you can't download or save files Discussion - Knowledge Base | Mozilla Support
• Firefox adds anti-malware file reputation service - ZDNet Zero Day blog
• How to Disable Anti-malware Download Check Service in Firefox - TrishTech.com

Only it doesn’t seem to work (at least for me and my Firefox instance). Even after relaunches of Firefox after the key value is set thusly (blank), the downloads are still getting blocked.

I’ve triple checked the key to make sure no typos or white-spaced but it seems 100% accurate for the documentation, but the feature modification just doesn’t seem to work for some reason for the “blank” value.

However, when I disable the “Block reported attack sites” option I can now download the files fine.

Hmmm. All or nothing? Or new bug in 32.0/32.0.1 builds? Or “just lucky me”?

Reading this post Writing for the 98% by Monica at Mozilla I saw reference to some additional keys:

It is also interesting that fewer people disable Google SafeBrowsing checks for malware than for phishing (browser.safebrowsing.enabled and browser.safebrowsing.malware.enabled). Presumably these are disabled for privacy or performance reasons. Are users who disable one and not the other making a mistake, or do these users consider themselves phish-proof but not drive-by-download-proof? If it is a mistake, why do we allow users to construct a set of preferences that are internally inconsistent in reasoning?

Monica’s post reference above is a very interesting insight into the Mozilla development feature considerations and musings. That whole post is insightful and worth reading. I’ve added her blog to my RSS feeds. I’ve got my eye on those who look to move my Firefox cheese!

But seriously…

I found these preferences also but disabling them (value = false) seems to have the same effect as disabling the “Block reported attack sites” in the GUI options.

• Browser.safebrowsing.malware.enabled - MozillaZine Knowledge Base

So no help on just isolating & disabling the application reputation check.

If you are curious, the default value for the “browser.safebrowsing.appRepURL” key is https://sb-ssl.google.com/safebrowsing/clientreport/download?key=%GOOGLE_API_KEY%

• I update my Firefox to last version 32.0, and now i can not download anythink, can you help me - Firefox Support Forum | Mozilla Support
• Firefox adds anti-malware file reputation service - ZDNet Zero Day blog

NirSoft may not be the only site/developer hit by this headache, but he sadly appears to be one of the most well known. It is clear that due to the nature of the tools he writes, they do show up often by malicious users, but a total block is very frustrating for the many, many sysadmins who depend on them for the support work we do, as well as for Nir himself (Nirblog tag link).

• RESOLVED - Firefox v31 - New Malware idea is BAD - mozillaZine Forums
• Nirsoft - mozillaZine Forums

Sadly, unlike the advice I found from Mozilla and the mozillaZine forums, blanking out the “browser.safebrowsing.appRepURL” key value does NOT allow me to download the files successfully.

My Firefox version is 32.0.1 at the time of this posting (x32 bit Windows version).

So for now I’m stumped and will likely be resorting to downloading the files using a different web browser.

Or maybe not…

Chromium:

Well, at least with Chromium I can choose to “hurt myself” by restoring it!

Internet Explorer 11 at least isn’t too fussy:

Too bad there isn’t a feature in Firefox (like Chromium) that will allow you to allow/restore the blocked file on a case-by-case basis!  That would seem to be a nice balanced option.

1066133 – Provide a way to override application reputation checks on a per-download basis. - Bugzilla@Mozilla as filed by Wes Kocher

Currently, the only way to bypass the check is to disable the service completely. Would be nice to be able to leave the service enabled, but ignore it for a single download.

Some more thoughts/notes:

While re-attempting the download of PasswordFox after making the change to the “browser.safebrowsing.appRepURL” key, I ran a system trace using ProcessMonitor.

What I found were several of the calls to the local safebrowsing list files.

Per documentation in articles linked above in order to make the malware check process more efficient, the browser first checks individual file signature against a local list of trusted publishers. If passes, file is good, if not, then it proceeds with the online file reputation check.

Depending on your particular Firefox build they may be locate differently. I use a portable version of Firefox and the key local safebrowsing file lists were located under this path:

<drive letter>:<subfolders>\FirefoxPortable\Data\profile\safebrowsing

For Windows (Vista/7/8) users who use an installed version of Firefox I believe the location most likely would be here:

C:\Users\YourUserNameHere\AppData\Local\Mozilla\Firefox\Profiles\safebrowsing

I wondered if possibly something had been corrupted, so I deleted (moved to a diff folder) all the contents, shut down Firefox, confirmed they were still gone, then relaunched Firefox.

They rebuilt after launch but I didn’t see the previously present “classifier.hashkey” file being restored. It was dated from 11/20/2012 so maybe it was a remnant from a previous FF version.

Alas, the file download still was blocked (deleted after download).

What is really silly is that while the x32 bit version of PasswordFox gets blocked…the x64 bit “Waterfox” version is allowed to download with no fuss!

Talk about lack of a consistent reputation/rule policy!

Back to the ProcessMonitor traces for now…

One final point. I’m focusing on Nir Sofer’s PasswordFox app in this post just because I can replicate the issue/behavior with it.

I don’t think it is in anyway limited to just that file in particular.

To be clear, this is less about that particular application and all about trying to successfully disable the application reputation download protection feature via the documented work-around from Mozilla.

If I do work out anything in terms of a “resolution” to getting the application reputation check feature disabled, I’ll post an update.

Curious…

Claus Valca

Update:

I downloaded the Mozilla Firefox ESR, Portable Edition 31.x and tried it.

It does have the “browser.safebrowsing.appRepURL” key present and the “https://sb-ssl.google.com/safebrowsing/clientreport/download?key=%GOOGLE_API_KEY%” value present by default.

This Firefox version let me download the PasswordFox file with no complaints or blocks at all.

Then I downloaded a fresh install of the Mozilla Firefox, Portable Edition 32.0.1.

It is missing the “browser.safebrowsing.appRepURL” key.

However this Firefox version/download also let me let me download the PasswordFox file with no complaints or blocks at all. Even with that key missing entirely!

Maybe something is corrupt in my “production” Firefox build?

Update 2:

I shut everything down and tried the FF ESR portable version again.  This time (with no changes made) it did block the download of the PasswordFox file. Then I blanked out the required key and the download was allowed just fine. This is a FF 31.x release version. Resetting the key (it auto filled the default URL value in again by itself) and restarting FF ESR enabled the file download to block again.

I dumped my previous test install of FF Portable Edition 32.0.1 and rebuilt it from scratch. It blocked the file download on initial run now (as expected) as malicious.

Again, checking the about:config page finds it is missing the “browser.safebrowsing.appRepURL” key.

This time I manually added the key and the default string and restarted the browser.

The file download was blocked.

I then blanked out the focus string and restarted the browser.

Downloading again found the file still blocked.

I’m now leaning to there being some kind of bug (or feature change) in at least the 32.0.1 Firefox release in terms of that particular key.  Not only is it missing from the 32.0.1 release initially, once you put it in there manually…with the correct default value, and then remove it, and restart and attempt to put the “default” back, it doesn’t recognize/restore it automatically like the FF ESR release does.

Very interesting! So now I’m thinking it likely isn’t just a local problem with my FF profile/package.

Due to my testing, I can also confirm seeing this behavior: 1053645 – Downloads blocked by Application Reputation are retried on a restart - Bugzilla@Mozilla

More technical related discussions at Bugzilla@Mozilla - ”Application Reputation” Bug List

--CV