Blog: SANS Digital Forensics and Incident Response Blog

Blog: SANS Digital Forensics and Incident Response Blog

Firefox 3 History

Analysis of a browser history almost always comes up, no matter what is being investigated. And despite Firefox being one of the most popular browsers currently used there aren't many tools out there that can read and display browser history (at least in a human readable format). There are tools out there, such as f3e from FirefoxForensics.com (firefoxforensics.com) however that tool, just as others that I've found, is only distrubuted as an EXE, running on Windows (and no source code is provided).

Traditionally Firefox stored the history file as a Mork file format, which could be easily read using any standard editor. The new version, that is version 3 which has been out for quite some time now, uses a different method of storing user history. The history file is stored in a MozStorage format, as a SQLite database (see description here) in a file called places.sqlite. The tables can be easily read using sqlite3 for instance:

sqlite3 places.sqlite ".tables"
moz_anno_attributes moz_favicons moz_keywords
moz_annos moz_historyvisits moz_places
moz_bookmarks moz_inputhistory
moz_bookmarks_roots moz_items_annos

There are two tables in particular that are of interest , the moz_places and moz_historyvisits table. The schema for these tables can as well be easily extracted using sqlite3:
sqlite3 places.sqlite ".schema moz_places"
CREATE TABLE moz_places (
id INTEGER PRIMARY KEY,
url LONGVARCHAR,
title LONGVARCHAR,
rev_host LONGVARCHAR,
visit_count INTEGER DEFAULT 0,
hidden INTEGER DEFAULT 0 NOT NULL,
typed INTEGER DEFAULT 0 NOT NULL,
favicon_id INTEGER,
frecency INTEGER DEFAULT -1 NOT NULL);
CREATE INDEX moz_places_faviconindex ON moz_places (favicon_id);
CREATE INDEX moz_places_frecencyindex ON moz_places (frecency);
CREATE INDEX moz_places_hostindex ON moz_places (rev_host);
CREATE UNIQUE INDEX moz_places_url_uniqueindex ON moz_places (url);
CREATE INDEX moz_places_visitcount ON moz_places (visit_count);

And for the moz_historyvisists:
sqlite3 places.sqlite ".schema moz_historyvisits"
CREATE TABLE moz_historyvisits (
id INTEGER PRIMARY KEY,
from_visit INTEGER,
place_id INTEGER,
visit_date INTEGER,
visit_type INTEGER,
session INTEGER);
CREATE INDEX moz_historyvisits_dateindex ON moz_historyvisits (visit_date);
CREATE INDEX moz_historyvisits_fromindex ON moz_historyvisits (from_visit);
CREATE INDEX moz_historyvisits_placedateindex ON moz_historyvisits (place_id, visit_date);
CREATE TRIGGER moz_historyvisits_afterdelete_v1_trigger
AFTER DELETE ON moz_historyvisits FOR EACH ROW
WHEN OLD.visit_type NOT IN (0,4,7)
BEGIN UPDATE moz_places SET visit_count = visit_count - 1
WHERE moz_places.id = OLD.place_id
AND visit_count > 0; END;
CREATE TRIGGER moz_historyvisits_afterinsert_v1_trigger
AFTER INSERT ON moz_historyvisits FOR EACH ROW
WHEN NEW.visit_type NOT IN (0,4,7)
BEGIN UPDATE moz_places SET visit_count = visit_count + 1
WHERE moz_places.id = NEW.place_id; END;

According to the FirefoxForensics.com web site the explanation of each of keys found in the moz_places is roughly the following:
  • id INTEGER PRIMARY KEY, an integer that indicates the primary key for the database, of no real interest
  • url LONGVARCHAR, the URL that has been visited and the protocol used, something that one likes to examine.
  • title LONGVARCHAR, the title of the page as it appears in the browser
  • rev_host LONGVARCHAR, the reverse of the host name that was visited. used to ease searching and querying into hosts visited in history file.
  • visit_count INTEGER DEFAULT 0, as the variable implies a counter for the site
  • hidden INTEGER DEFAULT 0 NOT NULL,either 0 or 1. if the URL is hidden then the user did not navigate directly to it, usually indicates an embedded page using something like an iframe
  • typed INTEGER DEFAULT 0 NOT NULL,indicates whether the user typed the URL directly into the location bar
  • favicon_id INTEGER,relationship to another table containing favicon
  • frecency INTEGER DEFAULT -1 NOT NULL, combination of frequency and recency, used to calculate which sites appear at the top of the suggestion list when URL's are typed in the address bar.
Since we know the structure and the meaning of each key in the database it is simple to create a script that can parse the database and display the content as you wish. The Perl script I wrote is called ff3histview and can be found here. It reads the places.sqlite database and displays the content of it in a human readable format. The usage of the script is the following:
ff3histview [--help|-?|-help]
This screen
ff3histview [-t TIME] [-csv|-txt|-html] [-s|--show-hidden] [-o|--only-typed] [-quiet] places.sqlite
-t Defines a time scew if the places.sqlite was placed on a computer with a wrong time settings. The format of the
variable TIME is: X | Xs | Xm | Xh
where X is a integer and s represents seconds, m minutes and h hours (default behaviour is seconds)
-quiet Does not ask questions about case number and reference (default with CSV output)
-csv|-txt|-html The output of the file. TXT is the default behaviour and is chosen if none of the others is chosen
-s or --show-hidden displays the "hidden" URLs as well as others. These URL's represent URLs that the user
did not specifically navigate to.
-o or --only-typed Only show URLs that the user typed directly into the location/URL bar.

places.sqlite is the SQLITE database that contains the web history in Firefox 3. It should be located at:
[win xp] c:\Documents and Settings\USER\Application Data\Mozilla\Firefox\Profiles\PROFILE\places.sqlite
[linux] /home/USER/.mozilla/firefox/PROFILE/places.sqlite
[mac os x] /Users/USER/Library/Application Support/Firefox/Profiles/PROFILE/places.sqlite


The default behaviour of the script is to extract all URL's from moz_places that are not hidden (can be changed using parameters to the script, as well as to only see user typed in URLs). A hidden URL, according to firefoxforensics.com, is a URL "that the use did not specifically navigate to. These are comonly embedded pages, i-frames, RSS bookmarks and javascript calls."[1] So the SQL statement that the script executes is:
SELECT moz_historyvisits.id,url,title,visit_count,visit_date,from_visit,rev_host
FROM moz_places, moz_historyvisits
WHERE
moz_places.id = moz_historyvisits.place_id
AND hidden = 0

In fact a really simple SQL statement, just to extract the URLs and some other information of value. The "from_visit" that is extracted refers to a URL that the user navigated from. Relevant information about those nodes are extracted as well, giving the investigator more information about context.

The script can output both in text format as well as CSV (Comma Seperated Value) for spreadsheet manipulation as well as a HTML page. Examples of usage (standard default behavior, without asking questions about case details) :

ff3histview -q places.sqlite
Firefox 3 History Viewer
Not showing 'hidden' URLS, that is URLs that the user did not specifically navigate to, use -s to show them
Date of run (GMT): 13:7:9, Thu Jul 2, 2009
Time offset of history file: 0 s
-------------------------------------------------------
Date Count Host name URL notes
Thu Jun 25 09:16:17 2009 1 www.regripper.net http://www.regripper.net/
Wed Jun 24 20:19:53 2009 1 isc.sans.org http://isc.sans.org/
Thu Jun 25 12:43:14 2009 1 snort.org http://snort.org/
Thu Jun 25 12:43:09 2009 2 www.snort.org http://www.snort.org/
Thu Jun 25 12:53:51 2009 2 www.snort.org http://www.snort.org/ From: http://www.snort.org/news
Thu Jun 25 12:48:09 2009 1 www.snort.org http://www.snort.org/news From: http://www.snort.org/
Thu Jun 25 18:38:35 2009 1 www.groklaw.net http://www.groklaw.net/ From:
-------------------------------------------------------

And if there is any discrepancies in the time settings of the investigator's machine and the suspect's one:
ff3histview -q -t 14380 places.sqlite
Firefox 3 History Viewer
Not showing 'hidden' URLS, that is URLs that the user did not specifically navigate to, use -s to show them
Date of run (GMT): 13:8:34, Thu Jul 2, 2009
Time offset of history file: 14380 s
-------------------------------------------------------
Date Count Host name URL notes
Thu Jun 25 13:15:57 2009 1 www.regripper.net http://www.regripper.net/
Thu Jun 25 00:19:33 2009 1 isc.sans.org http://isc.sans.org/
Thu Jun 25 16:42:54 2009 1 snort.org http://snort.org/
Thu Jun 25 16:42:49 2009 2 www.snort.org http://www.snort.org/
Thu Jun 25 16:53:31 2009 2 www.snort.org http://www.snort.org/ From: http://www.snort.org/news
Thu Jun 25 16:47:49 2009 1 www.snort.org http://www.snort.org/news From: http://www.snort.org/
Thu Jun 25 22:38:15 2009 1 www.groklaw.net http://www.groklaw.net/ From:
-------------------------------------------------------

And if we want to include "hidden" URL's:
ff3histview -q -t 14380 -s places.sqlite
Firefox 3 History Viewer
Showing hidden URLs as well

Date of run (GMT): 13:14:58, Thu Jul 2, 2009
Time offset of history file: 14380 s

-------------------------------------------------------
Date Count Host name URL notes
Thu Jun 25 13:15:57 2009 1 www.regripper.net http://www.regripper.net/
Thu Jun 25 00:19:33 2009 1 isc.sans.org http://isc.sans.org/
Thu Jun 25 00:19:36 2009 0 www.sans.org http://www.sans.org/banners/isc.php From: http://isc.sans.org/
Thu Jun 25 06:14:16 2009 0 www.sans.org http://www.sans.org/banners/isc.php From: http://isc.sans.org/
Thu Jun 25 15:01:07 2009 0 www.sans.org http://www.sans.org/banners/isc.php From: http://isc.sans.org/
Thu Jun 25 06:19:18 2009 0 www.sans.org http://www.sans.org/banners/isc_ss.php From:
Thu Jun 25 15:58:20 2009 0 www.sans.org http://www.sans.org/banners/isc_ss.php From:
Thu Jun 25 13:15:57 2009 0 www.regripper.net http://www.regripper.net/links.htm From: http://www.regripper.net/
Thu Jun 25 13:15:57 2009 0 www.regripper.net http://www.regripper.net/main.htm From: http://www.regripper.net/
Thu Jun 25 13:16:13 2009 0 www.regripper.net http://www.regripper.net/RegRipper/ From: http://www.regripper.net/links.htm
Thu Jun 25 13:16:17 2009 0 www.regripper.net http://www.regripper.net/RegRipper/RegRipper/ From: http://www.regripper.net/RegRipper/
Thu Jun 25 13:16:28 2009 0 regripper.invisionplus.net http://regripper.invisionplus.net/ From: http://www.regripper.net/links.htm
Thu Jun 25 16:42:54 2009 1 snort.org http://snort.org/ From: http://www.regripper.net/links.htm
Thu Jun 25 16:42:49 2009 2 www.snort.org http://www.snort.org/ From: http://www.regripper.net/links.htm
Thu Jun 25 16:53:31 2009 2 www.snort.org http://www.snort.org/ From: http://www.snort.org/news
Thu Jun 25 16:47:49 2009 1 www.snort.org http://www.snort.org/news From: http://www.snort.org/
Thu Jun 25 22:38:15 2009 1 www.groklaw.net http://www.groklaw.net/ From:
-------------------------------------------------------


And for the HTML output:
ff3histview -t 14380 --html places.sqlite
History HTML

Kristinn Guđjónsson, GCFA #5028, works as a forensic analyst and incident handler as well as being a local mentor for SANS.

2 Comments

Posted July 16, 2009 at 4:48 AM | Permalink | Reply

Jay

What about the latest addition to Firefox's list of features "Start Private Browsing"? The only trace that can be narrowed down to a person to a particular date and time is the file's he had downloaded and also i've seen remnants of adobe flash usage. What do you think?

Posted July 18, 2009 at 9:31 AM | Permalink | Reply

kristinn

I definitely think this makes our jobs more difficult, from what I've seen there seem to be no traces left except those that you've mentioned (as well as new bookmarks that are added, but they are not linked to a particular date and quite possibly some data in memory).

In the older Firefox you could at least attempt to recover SQLite database remnants to find deleted history but if you use "Start Private Browsing" feature of FF3.5 no data is written to the disk making those attempts futile.

Other browser have provided at least similar feature as this in the past, and I guess we will see this implemented in the rest of them soon, making browser forensics more difficult.

Post a Comment






Captcha

* Indicates a required field.