3.10.09

Get Web browser history in C#

In my final year .NET project, I had to code up a module to obtain the visited URL history of at least one web browser and delete some or all of the URLs from the history. A seemingly impossible job at first, I eventually managed to write (or rather get) some neat C# code for the purpose. Here is the outcome of some very good research.

  1. Internet Explorer

  2. This was the easiest browser whose URL history can be obtained and deleted at will. My research started off with a Registry discovery - URLs typed in the address box of IE are captured in the Registry key HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\TypedURLs. But that was way short of the purpose. Then, I came across the COM interface IUrlHistory on MSDN. But it was COM and writing a C# class wrapper or something was something I was not proficient in at all. So, I reluctantly began searching for ready code. And voila!

    An article on CodeProject was all it took to serve my purpose. It provides a UrlHistoryLibrary.dll file which provides type-safe classes to access and manipulate IE's browsing history. Details on how to use the classes are given in the article itself. However, this is how I used it.




    1. public class InternetExplorer
    2. {
    3. // List of URL objects
    4. public List<URL> URLs { get; set; }
    5. public IEnumerable<URL> GetHistory()
    6. {
    7. // Initiate main object
    8. UrlHistoryWrapperClass urlhistory = new UrlHistoryWrapperClass();

    9. // Enumerate URLs in History
    10. UrlHistoryWrapperClass.STATURLEnumerator enumerator =
    11. urlhistory.GetEnumerator();

    12. // Iterate through the enumeration
    13. while (enumerator.MoveNext())
    14. {
    15. // Obtain URL and Title
    16. string url = enumerator.Current.URL.Replace('\'', ' ');
    17. // In the title, eliminate single quotes to avoid confusion
    18. string title = string.IsNullOrEmpty(enumerator.Current.Title)
    19. ? enumerator.Current.Title.Replace('\'', ' ') : "";

    20. // Create new entry
    21. URL U = new URL(url, title, "Internet Explorer");

    22. // Add entry to list
    23. URLs.Add(U);
    24. }

    25. // Optional
    26. enumerator.Reset();

    27. // Clear URL History
    28. urlhistory.ClearHistory();

    29. return URLs;
    30. }
    31. }




    Notes:
    The code is fairly self-explanatory except for a few clarifications. The URL class used above is a simple class which has three string fields - URL, Title and BrowserName - and URLs is a List<URL>. Those little string.Replace functions are to eliminate single quotes in the URLs so that there are no problems inserting them into an SQL database.

  3. Mozilla Firefox

  4. The same thing for Mozilla Firefox was way tougher. First of all, it was difficult to locate where Firefox stores the browsing history. The answer came in the form of an SQLite database which is stored in the Application Data of the Windows user. So, I had a SQLite database at hand. First, I tried FileStream to the .sqlite file. I had opened the file in Notepad++ and seen that URLs were interspersed with quirky characters, so using the FileStream, I could manage to obtain the URLs, but obtaining the Titles was impossible. Moreover, I was never sure that a URL I come across was ever visited or was it a bookmark or what. So, again some code-googling later, I came across the ADO.NET Data Provider for SQLite.The download page is here.

    What is wonderful about this Provider is that not only it provided a .NET-native namespace (System.Data.SQLite), but also it provides full Visual Studio 2008 design-time support. So, without wasting more time, let me provide you the code.




    1. public class Firefox
    2. {
    3. public List<URL> URLs { get; set; }
    4. public IEnumerable<URL> GetHistory()
    5. {
    6. // Get Current Users App Data
    7. string documentsFolder = Environment.GetFolderPath
    8. (Environment.SpecialFolder.ApplicationData);

    9. // Move to Firefox Data
    10. documentsFolder += "\\Mozilla\\Firefox\\Profiles\\";

    11. // Check if directory exists
    12. if (Directory.Exists(documentsFolder))
    13. {
    14. // Loop each Firefox Profile
    15. foreach (string folder in Directory.GetDirectories
    16. (documentsFolder))
    17. {
    18. // Fetch Profile History
    19. return ExtractUserHistory(folder);
    20. }
    21. }
    22. return null;
    23. }

    24. IEnumerable<URL> ExtractUserHistory(string folder)
    25. {
    26. // Get User history info
    27. DataTable historyDT = ExtractFromTable("moz_places", folder);

    28. // Get visit Time/Data info
    29. DataTable visitsDT = ExtractFromTable("moz_historyvisits",
    30. folder);

    31. // Loop each history entry
    32. foreach (DataRow row in historyDT.Rows)
    33. {
    34. // Select entry Date from visits
    35. var entryDate = (from dates in visitsDT.AsEnumerable()
    36. where dates["place_id"].ToString() == row["id"].ToString()
    37. select dates).LastOrDefault();
    38. // If history entry has date
    39. if (entryDate != null)
    40. {
    41. // Obtain URL and Title strings
    42. string url = row["Url"].ToString();
    43. string title = row["title"].ToString();

    44. // Create new Entry
    45. URL u = new URL(url.Replace('\'', ' '),
    46. title.Replace('\'', ' '),
    47. "Mozilla Firefox");

    48. // Add entry to list
    49. URLs.Add(u);
    50. }
    51. }
    52. // Clear URL History
    53. DeleteFromTable("moz_places", folder);
    54. DeleteFromTable("moz_historyvisits", folder);

    55. return URLs;
    56. }
    57. void DeleteFromTable(string table, string folder)
    58. {
    59. SQLiteConnection sql_con;
    60. SQLiteCommand sql_cmd;

    61. // FireFox database file
    62. string dbPath = folder + "\\places.sqlite";

    63. // If file exists
    64. if (File.Exists(dbPath))
    65. {
    66. // Data connection
    67. sql_con = new SQLiteConnection("Data Source=" + dbPath +
    68. ";Version=3;New=False;Compress=True;");

    69. // Open the Conn
    70. sql_con.Open();

    71. // Delete Query
    72. string CommandText = "delete from " + table;

    73. // Create command
    74. sql_cmd = new SQLiteCommand(CommandText, sql_con);

    75. sql_cmd.ExecuteNonQuery();

    76. // Clean up
    77. sql_con.Close();
    78. }
    79. }

    80. DataTable ExtractFromTable(string table, string folder)
    81. {
    82. SQLiteConnection sql_con;
    83. SQLiteCommand sql_cmd;
    84. SQLiteDataAdapter DB;
    85. DataTable DT = new DataTable();

    86. // FireFox database file
    87. string dbPath = folder + "\\places.sqlite";

    88. // If file exists
    89. if (File.Exists(dbPath))
    90. {
    91. // Data connection
    92. sql_con = new SQLiteConnection("Data Source=" + dbPath +
    93. ";Version=3;New=False;Compress=True;");

    94. // Open the Connection
    95. sql_con.Open();
    96. sql_cmd = sql_con.CreateCommand();

    97. // Select Query
    98. string CommandText = "select * from " + table;

    99. // Populate Data Table
    100. DB = new SQLiteDataAdapter(CommandText, sql_con);
    101. DB.Fill(DT);

    102. // Clean up
    103. sql_con.Close();
    104. }
    105. return DT;
    106. }
    107. }



    So, while some of this code was taken from the mozillazine forums, some was written on my own and the combination worked perfectly well for me in my final year project. Hope it works for you, too!

40 comments:

  1. the code is not working on VS 2010

    ReplyDelete
  2. I think you focused only on the code instead of reading the blog post entirely.
    For the IE code to work, you need to add IUrlHistory.dll to the project references. The link for the dll is provided in the blog post.
    For the Firefox code to work, you need ADO.NET Data Provider for SQLite, again whose dll link has been provided in the blog post.

    Hope this helps.

    ReplyDelete
  3. thanks for your reply!
    i have read the blog very carefully!!!
    the IE code is working fine!!!
    i have installed the SQLite as well but i m getting error for list URL "public List URLs { get; set; }"

    ReplyDelete
  4. further you have any idea how to do the same with Opera and Chrome browsers?

    ReplyDelete
  5. I have not defined the the URL class which I have used, so I think that's a mistake on my part. You can go ahead and make a class that looks like this.
    public class URL
    {
    string url;
    string title;
    string browser;
    public URL(string url, string title, string browser)
    {
    this.url = url;
    this.title = title;
    this.browser = browser;
    }
    }

    I think this should do it.

    ReplyDelete
  6. Hi, I have had a lot of trouble getting the SQLite running in C#.... can you possibly help?
    Code looks great otherwise, setting up visual studio is frustrating sometimes though.
    Thanks - falazar@yahoo.com

    ReplyDelete
  7. Hi, I was wondering if you could help, am attempting the FF version, and having troubel with SQLite and Visual Studio... thanks
    falazar@yahoo.com -
    James

    ReplyDelete
  8. Hi , I am an error as type or namespace is not found.....

    can u tell me....

    ReplyDelete
  9. Can you be more specific? If you can send me a screenshot or something or just point out the line where you are getting the error, I could help you.
    However, these steps may help.
    In case of Firefox:
    In the solution you are working on, add a reference to the SQLite dlls. If you have installed the ADO.NET provider for SQLite correctly, you will get the dlls at the following locations:
    C:\Program Files\SQLite.NET\bin\System.Data.SQLite.dll
    C:\Program Files\SQLite.NET\bin\System.Data.SQLite.Linq.dll
    Then add a "using System.Data.SQLite;" statement in the usings of your code file and you should be good to go.
    In case of Internet Explorer:
    In the article, I have pointed to a CodeProject article which contains a solution for IUrlHistory. You can get it here. (The catch is that you have to be a CodeProject member, but it's free).
    So, open the solution in Visual Studio, build it successfully once and then navigate to its bin directory. There, you will find a UrlHistoryLibrary.dll file. Copy it to a suitable location.
    In the solution you are working on, add a reference to this dll and add a "using UrlHistoryLibrary;" statement in the usings of your code file.

    Hope this helps.

    ReplyDelete
  10. Thanks Joel...
    It's Working.......

    ReplyDelete
  11. Thank you very much. That's what I'm trying to do.

    ReplyDelete
  12. Hii,
    Many thanks for sharing nice code sample. But,
    do you think the folder: "\\Mozilla\\Firefox\\Profiles\\" exists in local hard drive??

    I have installed mozila 4.0 which does not creates such folder path, now will your code work okay?

    ReplyDelete
  13. Hi,

    Currently we get an issue database is locked but when we close the firefox it works fine for me.
    Please suggest me that it works fine in the case of open firefox also??

    ReplyDelete
  14. @Karthik: Make sure you have added a project reference to System.Data.dll. If yes, you must have System.Data namespace added in your code file. Sorry for the late reply.

    ReplyDelete
  15. @Pankaj: Correct. It does not work when firefox is running. Sorry for the late reply.

    ReplyDelete
  16. @Navin C: This code is guaranteed to work for Firefox 3 series only. For Firefox 4, get started here:
    http://hacks.mozilla.org/2010/06/comparing-indexeddb-and-webdatabase/
    Sorry for the late reply.

    ReplyDelete
  17. Error I got is :
    The name directory does not exist in the current context.
    I tried in FireFox 11 . Please provide a solution .

    ReplyDelete
    Replies
    1. binary have you already sorted this issue out?

      Delete
    2. Try adding one of this directives at the top of the class:

      using System.IO;

      Delete
  18. Thanks Joel for the codes above. Very helpful. I have also added a new class to add support for Google Chrome and now it works like a charm for all 3 browsers. Cheers

    ReplyDelete
    Replies
    1. hi, i can't have any results using this code with internet explorer,( i need this in my final year project), i am new with C#, i wonder if you can help me with google chrom classes, hope that's possible
      i am having an exception at this line URLs.Add(U); VC2010 suggests to add ( new) somewhere... i really can't understand this error!!

      Delete
    2. Hi, people! For Google Chrome, I have a new post for you: http://hardcodedblog.blogspot.in/2012/01/get-web-browser-history-in-c-google.html

      Micha, I hope you have resolved the VC2010 error by now. :)

      Delete
    3. Ok, I also have a NullReferenceException for

      URLs.Add(U), any ideas?

      Delete
    4. Ok, for anyone who cannot get the IE code to work, see my blog at

      thestrangertech.blogspot.com

      for a solution.

      Delete
  19. hi, can anyone tell me how to parse History.dat file in opera location.

    ReplyDelete
  20. I am getting this error "Mixed mode assembly is built against version 'v2.0.50727' of the runtime and cannot be loaded in the 4.0 runtime without additional configuration information." when I am running code for firefox.. plz help

    swati

    ReplyDelete
  21. has any one worked on code for latest version of firefox.

    swati

    ReplyDelete
  22. I try to use y this code in web part and deploy it ti sharepoint site
    but it not work
    i have SecuritException
    can U help me please?
    thanks

    ReplyDelete
    Replies
    1. Hi Mostafa, this code gets the browser history of the machine on which it is executing. As such, it is supposed to work only in Desktop applications. Web applications do not have permission to access the local resources that this code accesses.

      Even then, can you post the complete exception that you are getting on the Sharepoint site?

      Delete
  23. Hi Joel Mathias,
    sorry i didn't see your reply before now.
    when i try to use this code in web part
    the exception was
    SecurityException was unhandled by user code
    "System.security.permissions.securitypermission"

    this appear in this section
    public UrlHistoryWrapperClass()
    {

    urlHistory = new UrlHistoryClass(); ///////Here
    obj = (IUrlHistoryStg2) urlHistory;

    }
    Thanks.

    ReplyDelete
  24. add the namespace system.data i hope it will work fine . Thanks in advance.

    ReplyDelete
  25. Hi you have nice code hre, i was try to convert the code to Vb.Net, everything is good but i have error on Private Function ExtractUserHistory (The line: Dim entryDate = (From dates In visitsDT.AsEnumerable() Where dates("place_id").ToString() = row("id").ToString() dates).LastOrDefault()

    It says ")" expected

    Can you help me with that?! Thank you

    ReplyDelete
  26. hi.. i am getting error in Directory and File. Can any one help me out.

    ReplyDelete
  27. please can you show the same in vb.net.? I need the ful working code

    ReplyDelete
  28. Hello I am using this code but when i run it threw an exception.
    can you please help me to solve this issue. ecption threw in URLs.Add(U);

    ReplyDelete
  29. HI All, I'm not able to download "UrlHistoryLibrary.dll" file from anywhere. I've spent lots of time to download this dll. Please provide me any link so that I can download this.
    Thanks in advance

    ReplyDelete
    Replies
    1. It's in the Code project page referenced in the article.

      Delete
  30. Hi All, I've to the same as described above using visual studio 2010. But, not able to get IE browsing history. Kindly suggest me about any solution.
    Thanks in advance...

    ReplyDelete
  31. Hello, I need to capture web history for Google Chrome and Firefox. can any one please help me with the tool? as i am not a experience developer, can not write high level code. you can reach me on andy010679@gmail.com. Thank you.

    ReplyDelete