Social Networking Team Solves Part of Problem

The Mediacom Social Networking team has worked with me through various channels and solved part of the problem. There are still technical challenges, but progress is being made.

Slow or No Progress

I am slowly working my way up the chain of command in search of someone who can make this right. Mediacom has no incentive to stop maliciously hijacking legitimate HTTP traffic unless customers demand monetary compensation in the form of account credits in response to this ongoing problem. Thus far, everyone I have dealt with has been extremely polite, but has stated that they cannot offer a credit for this issue as they believe they are still providing consistent service. My response is that they are only providing partial service in that they are selectively hijacking traffic and creating a situation in which the connection itself cannot be trusted. This sort of "man in the middle" attack is absolutely abhorrent and must be stopped.

On the plus side, the Mediacom Social Networking Team has once again provided consistent clear and polite communication regarding this issue. They are absolutely the best way to contact Mediacom if you have any sort of problem.

Problem Reappears

The problem appears to be reappearing every few weeks. Usually, opting in and back out is sufficient to fix it again but it is incredibly aggravating having random requests fail to produce valid results due to the standards-breaking HTTP hijacking Mediacom is perpetrating upon its customers.

To date, I have not asked for or received any sort of compensation for the aggravation this issue has caused. I have opened a new thread in the Mediacom support boards asking for the problem to be fixed and for credit to be issued to my account for the fact that I have had to deal with this problem repeatedly.

The Mediacom board team has worked hard to try to solve the problem in the past and I am hopeful that they do so again this time. It is sad that such effective customer support people are being undercut by such incompetence on the network operations side.

Update: A Mediacom representative replied that they are unable to issue credit for this issue. I have replied requesting contact information for the entity that can issue such a credit. This should raise red flags with current and potential customers - Mediacom's initial response is that they cannot issue credit when the connection is not functioning properly due to Mediacom's own choices.

Problem Appears Resolved (Round 2)

It once again appears as though the problem has been resolved. The Mediacom Social Networking Team continued to work diligently on this problem and posted a message to their forum this morning that it had been addressed. After opting in then back out of the hijacking services per Mediacom's instructions, I am unable to reproduce the redirect behavior on either a home or business account. MediacomBryan posted the following explanation as to the cause of the problem and gave me permission to quote it here:

The main database that houses the modem list for opt outs wasn't communicating properly with all the systems involved. Once the communication issue was repaired, the opt-in then opt-out process was required so that all the systems would get a new message to remove your modem from their respective list(s).

I will update this blog if the problem reappears.

Problem May Be Resolved (Update: Not resolved)

As of Sunday, January 30, 2010 at approximately 1500 hours, it appears as though the problem may be resolved. Mediacom's Social Networking Team has been working with me on this issue via the Mediacom support forums located here: Mediacom Support Forum. I will update this blog if the issue returns.

I want to publicly thank the Mediacom Social Networking team. They immediately responded to my post on their forum and have worked diligently over the past few days to get this problem addressed. I still believe it is inexcusable that these hijacking systems exist in the first place, but the team there did everything in their power to get it fixed and should be commended for that. I know it will be the first place I turn if I have future issues with my Mediacom service.

Update: A few minutes after the test, the Google hijacking returned. I will continue working with the Social Networking team to get the problem resolved and update here when it is.

Mediacom Hijacking - Initial Analysis

On January 26th and 27th, I observed that the ISP I use at home and at work, Mediacom Cable, had started hijacking HTTP requests sent from my computer and impersonating the intended recipient in order to direct me to a Mediacom search page. The purpose of this blog post is to detail exactly when I believe Mediacom is hijacking web traffic and how to avoid some of it until Mediacom discontinues the practice.


Sadly, it is fairly common for ISPs to hijack DNS requests to nonexistent domains in order to drive traffic to search portals, but Mediacom is now going even further and actually hijacking HTTP traffic directed towards legitimate and functional servers. They are doing this in two ways: they are hijacking requests which result in 404 responses from legitimate servers and, perhaps more intrusively, they are hijacking requests to google that would otherwise result in legitimate search results. Below, find a video detailing this behavior as well as captures of the HTTP streams of both kinds of hijacking.


Google Search Hijacking
Firefox (or in my case, Iceweasel) will attempt to perform a google search if a user enters an invalid or nonexistent domain into the URL bar. The browser appends a special GET parameter to this google search to notify google that the search resulted from the browser itself. As of yesterday, Mediacom Cable is hijacking these requests and instead directing the user to their own search page. If, at this very moment, I attempt to proceed to:



http://www.google.com/search?ie=UTF-8&oe=UTF-8&sourceid=navclient&gfns=1&q=testing something else


in either Iceweasel or Chrome, I am instead redirected via javascript to:


http://assist.mediacomcable.com/mediacomassist/dnsassist/main/?domain=http%3A//www.google.com/search%3Fie%3DUTF-8%26oe%3DUTF-8%26sourceid%3Dnavclient%26gfns%3D1%26q%3Dtesting%2520something%2520else


Let me be clear: Mediacom is forcing me to use their search engine (which has sponsored links and ads) rather than Google for some specifically formed valid Google search requests. This is equivalent to an ISP sending you to ford.com when you type in chevy.com because they own an interest in Ford!


I was curious how Mediacom was accomplishing this hijacking, so I fired up Wireshark and captured the contents of the HTTP traffic that occurred when I attempted to visit a google URL. Here is that traffic:

GET /search?ie=UTF-8&oe=UTF-8&sourceid=navclient&gfns=1&q=kajdfkaj+adkjfa HTTP/1.1
Host: www.google.com
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.7) Gecko/20100721 Iceweasel/3.6.7 (like Firefox/3.6.7)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Cookie: <Removed>

HTTP/1.1 200 OK

<HTML><script>window.location='http://assist.mediacomcable.com/mediacomassist/dnsassist/main/?domain='+escape(window.location);</script><body>The Search Guide redirection service has been enabled to provide helpful searches from browser queries. You entered a non-existent url and your browser attempted to redirect you with Javascript. To enable this please update your browser preferences. <a href='http://search.mediacomcable.com/prefs.php'>To turn off this feature please click this here</a></body></HTML>

As evidenced by the host header, my browser believes it is talking to google.com, yet the response is clearly not the Google page I requested. Rather, Mediacom is impersonating Google and sending back a javascript block which redirects me to their competing search engine.

But, the problem doesn't stop with Google. I noticed on the 27th that Mediacom also started hijacking requests that resulted in 404 errors from other sites. For example, when attempting to visit the url:

http://www.penny-arcade.com/intentionallycausing404onmediacom/

I was instead redirected to:

http://assist.mediacomcable.com/mediacomassist_pnf/dnsassist/main/?domain=http%3A//www.penny-arcade.com/intentionallycausing404onmediacom/

Again, I fired up Wireshark and recorded an HTTP capture. Here is the result of that capture:

GET /intentionallycausing404onmediacom/ HTTP/1.1
Host: www.penny-arcade.com
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.7) Gecko/20100721 Iceweasel/3.6.7 (like Firefox/3.6.7)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Cookie: <Removed>

HTTP/1.1 200 OK

<HTML><script>window.location='http://assist.mediacomcable.com/mediacomassist_pnf/dnsassist/main/?domain='+escape(window.location);</script><body>The Search Guide redirection service has been enabled to provide helpful searches from browser queries. You entered a non-existent url and your browser attempted to redirect you with Javascript. To enable this please update your browser preferences. <a href='http://search.mediacomcable.com/prefs.php'>To turn off this feature please click this here</a></body></HTML>

Again, you can see that my browser believes it is talking to penny-arcade.com, yet the response has been altered or replaced by the Mediacom search redirect.

Mediacom claims that all of these "helpful" services are opt-out, but I have opted out on the only preferences page I am aware of and I am still maliciously redirected. I recorded a video demonstrating this behavior in both Iceweasel (Firefox) and Chromium (Chrome) and it is available here: http://www.youtube.com/watch?v=y4vEw2eYhxY (Video best viewed full-screen)
 
I have contacted Mediacom about this issue and, after over an hour, was finally able to convince a Tier 1 supervisor that the problem really was occurring. He has escalated the issue to their Tier 2 staff and expects it to get escalated to the Network Operations Center. I will update this blog if more information becomes available.

For now, if you are a Mediacom customer and want to avoid as much hijacking as possible, you can take the following steps:

Use Google's DNS servers (8.8.8.8 and 8.8.4.4) rather than Mediacom's to avoid DNS hijacking.

Modify your firefox configuration (under about:config) to avoid Google search hijacking. I changed the keyword.URL key to use https://encrypted.google.com, but removing the sourceid parameter also seems to prevent Mediacom from hijacking the request as far as I can tell.

At the moment, I don't know of an efficient way to avoid Mediacom's 404 hijacking. Suggestions are welcome in the comments and I will update the post if someone comes up with something that works.

Once I figured out what was going on, I found a few other web posts about this issue dating back as far as 2009.

On January 28th, 2011, additional testing showed that the 404 redirect was happening on two separate Mediacom home accounts using Linux, and a Mediacom business account using Linux, Windows, and OS X. This is entirely consistent with a network-level hijacking rather than some problem with the local configuration of one network or computer.

Edited: Fixed date.