Unable to access specific website

[Nick2345]

I have a very strange and specific problem. I am unable to access one (that I know of) specific website from my home internet connection. The website is simply https://www.arnoldclark.com (i.e. the well known car/garage website). I first tried to access it last week and found that the connection attempt timed out. Since my internet was working fine in all other regards, I assumed that their website was down. However, I then tried again several times over the weekend and the same happened. I happened to speak with my brother about this on Sunday and yesterday (Monday) he emailed me and said he could get to the site fine (he is also a TalkTalk customer). I tried again and it failed again.

No matter what device I use (iOS and android phones, windows, Linux, ChromeOS laptops) I cannot reach the site via my home broadband. However, using my iOS phone in 4G I can access the site and if I use it as a hot spot, any of my other devices can then access the site. Hence, it is definitely an issue with the home broadband.

I have an Eero 6 router and I have checked and confirmed that it has no security/profile/blacklisting active that would prevent access to sites (and the website is hardly one that would be age limited etc!).

I have used a range of tools (such as ping, wget, tcptraceroute - all on Linux) that show that the connection attempt makes it as far as the general servers for the website host. However, the connection then appears to just stop (as might be the case if a server firewall was blocking and dropping the connection).

I had a 1.5 hr long "chat" with Talktalk customer services yesterday, which went through all the usual stuff (is the router turned on, etc). We eventually got a stage where (after I suggested it) apparently my router's external IP address was changed (I could not verify this, since I had not made a note before the change of what the address had been). That made no difference, as did any other change that was made (they also appeared to change the DNS servers - I knew that would make no difference, since tcptraceroute, ping, wget etc all showed me that DNS resolution was not an issue).

Hence, I am still stuck with the situation that essentially my internet connection is "working fine", but I cannot access this one specific website (I may also be unable to access other website, but they are unknown at present). This is very strange and frustrating.

The customer support person I chatted to yesterday said they would raise this as a formal complaint, but I have yet to see any notification telling me that this has been done.

I would therefore ask whether anyone else has ever seen this behaviour and/or has any idea what is causing it and how to fix it. As I stressed to the customer support person yesterday, my testing so far strongly indicates to me that the issue is not related to my router or devices (in particular, the output from tcptraceroute demonstrates that the connection request makes it all the way through to the servers associated with the website host for arnoldclark, at which point it then fails. Hence, my router, the fibre connection, DNS server and internet connections all the way to the website host servers is working).

Right now I am left totally scratching my head.

[Chris-TalkTalk]

OK thanks Nick 🙂

[Nick2345]

Hello @Chris-TalkTalk ,

Yes, James has been in touch. However, he said that they had not yet done anything. He did say that City Fibre had a red service flag on the line, but he did not know whether that was an old one. I have an open tech support ticket and can contact him back if issues reoccur. At present though it appears to be one of the frustrating "sorted itself" issues. I would rather know that something was identified and fixed, but I am happy that the issues has gone away.

[Chris-TalkTalk]

Hi Nick,

Glad to hear it working. Has someone from our network  team been in touch with you now?

Chris

[KeithFrench]

Hi @Nick2345 

You are very welcome; the more complex the issue, the more I enjoy it!

I hope that TalkTalk's networking team have resolved it. I hate faults that just go away. Let's hope it stays working.

[Nick2345]

Thanks @KeithFrench ,

You tickled my interest, so I installed it (also available for linux). On Linux it needs tcptraceroute to already be installed, which it is. I then gave it a go with google.com and got good output. I then did the same with arnoldclark.com and expected it to fail. However, it did not.

Two ping outputs:

Laptop-15-en0xxx:/usr/bin$ sudo tcpping arnoldclark.com
traceroute to arnoldclark.com (185.7.231.66), 255 hops max, 60 byte packets
seq 0: tcp response from 185.7.231.66 (185.7.231.66) <syn,ack,mss=1460,sack,window_scaling> 19.082 ms
traceroute to arnoldclark.com (185.7.231.66), 255 hops max, 60 byte packets
seq 1: tcp response from 185.7.231.66 (185.7.231.66) <syn,ack,mss=1460,sack,window_scaling> 18.717 ms

I found that rather surprising, so fired up Chrome and browsed to the site - yes, that is what I said, browsed to the site! It worked. Either the TT network team has waved a very quick magic wand or somehow the issue has resolved via another way.

Just to be certain that tcptraceroute also now succeeds, I ran that:

Laptop-15-en0xxx:/usr/bin$ sudo tcptraceroute arnoldclark.com
Running:
traceroute -T -O info arnoldclark.com
traceroute to arnoldclark.com (185.7.231.66), 30 hops max, 60 byte packets
1 _gateway (192.168.1.1) 3.397 ms 3.325 ms 3.278 ms
2 removed for privacy
3 xe-9-2-0-scr001.thn.as13285.net (78.144.0.118) 8.614 ms 7.839 ms 8.124 ms
4 ae54-scr001.msp.as13285.net (78.144.1.60) 14.189 ms 14.158 ms 14.122 ms
5 ae60-scr002.msp.as13285.net (78.144.1.131) 14.050 ms 14.018 ms 13.982 ms
6 ixmanchester.kil-man.fluency.net.uk (195.66.244.45) 13.998 ms 15.107 ms *
7 * * ae4.cr2.kil-man.fluency.net.uk (192.124.192.62) 19.333 ms
8 * ae0.cr1.cen-gla.fluency.net.uk (192.124.194.70) 19.218 ms *
9 xe-2-0-0.an1.cen-gla.fluency.net.uk (192.124.192.12) 18.860 ms * *
10 185.240.201.181 (185.240.201.181) 18.679 ms * *
11 185.7.231.66 (185.7.231.66) <syn,ack,mss=1460,sack,window_scaling> 19.567 ms * *

It completed successfully too. Hence, the issue appears to have been resolved/fixed.

Thank you once again for all your help with this.

[KeithFrench]

HI @Nick2345 

As you know, that is always a potential issue with tracert and ping, because they use ICMP. I have found a brilliant tool this morning for Windows called tcping, which pings a user-specified port number, e.g 443. You can ping by hostname or IP address, but it does not actually ping the destination at all. It tries to set up a TCP session instead, & if the distant end responds correctly, it clears down the session gracefully, thus, it does not use ICMP at all. The RTT times will vary compared to a successful ping, as they are calculated at the application layer and will be greater than those from ping, which operates at the network layer. It has some really useful options as well, including adding timestamps & outputting to a CSV.

[Chris-TalkTalk]

OK, thanks for the information 

[Nick2345]

Thanks @Chris-TalkTalk,

Just to add to what @KeithFrench  said, I used both Traceroute (uses ICMP) and tcptraceroute (uses TCP). I initially used traceroute, but was also getting problems accessing some other websites, which muddied the waters until I remembered that traceroute uses ICMP packets and some servers now block this. Switching to tcptraceroute gave consistent results whereby every site I traced the route to was accessible perfectly fine, except this one site (arnoldclark.com). 

Thank you for escalating to your networking team. I hope that they can review and figure out why this is happening.

[Chris-TalkTalk]

  No problem Keith, I've sent your diagnostics to the network team with a link to the thread 🙂

Chris

[KeithFrench]

Thanks @Chris-TalkTalk. I can't replicate it either, but the tracert done by me, produces identical results to that done by @Nick2345

[Chris-TalkTalk]

I can't replicate the problem on my home connection so just thought switching the ONT off for 30 minutes was worth a try. I'll pass it over to the network team and let you know when I receive an update or they may contact you directly 

Chris

[KeithFrench]

Hi @Chris-TalkTalk 

I do not think this is anything to do with Nick's ONT. He can access all other websites. From my diagnosis report just before passing this to you, a trace route proves the connection attempt goes beyond the TalkTalk network up to an autonomous system within the backbone BGP routing in Scotland. There is nothing further within the tracert, possibly because the destination routers have ICMP disabled. 

Therefore I can only conclude that there is some routing issue between the AS56595 & the Arnold Clark resolved IP address, which possibly only affects some of TalkTalk's customers.

[Nick2345]

Hello Chris,

I can do this, but unfortunately that is problematic during the day since people are working from home and also studying from home. Hence, that will have to be later this evening/tonight.

Please can I ask what this is expected to achieve? Keith has already identified that the issue is not in any way related to my Eero or my ONT connection. Keith's analysis, which concurs with what my own troubleshooting indicated, identifies a network routing issue somewhere else.

However, so we can say we tried the "switch it off and on again" IT solution, I will do that later today and then repost. My prediction - no change. However, I am willing to eat a lot of humble pie if it changes things.

[Chris-TalkTalk]

Thanks Keith. @Nick2345 could you switch the ONT and eero off for 30 minutes then switch back on and retest, if you are then still unable to access the website we'll pass this over to our network team as Keith has suggested 

Chris

[KeithFrench · Accepted solution]

FAO TalkTalk support

Please can you forward this to your Networks team for further investigation.

The OP is unable to reach the website https://www.arnoldclark.com/; however, other customers, such as myself, can.

Diagnostics:-

DNS correctly resolves to 185.7.231.66.

A tracert looks to be OK, bearing in mind that a BGP Autonomous System is in use here:-

sudo tcptraceroute arnoldclark.com
Running:
traceroute -T -O info arnoldclark.com
traceroute to arnoldclark.com (185.7.231.66), 30 hops max, 60 byte packets
   <first two lines removed to preserve privacy>

3 * ae50-ner001.msp.as13285.net (78.144.1.33) 11.717 ms *
4 * * ae50-scr001-msp.as13285.net (78.144.1.32) 11.591 ms
5 ae60-scr002.msp.as13285.net (78.144.1.131) 10.876 ms 10.863 ms 10.847 ms
6 ixmanchester.kil-man.fluency.net.uk (195.66.244.45) 11.455 ms * *
7 * * *
8 * * ae0.cr1.cen-gla.fluency.net.uk (192.124.194.70) 14.567 ms
9 xe-2-0-0.an1.cen-gla.fluency.net.uk (192.124.192.12) 15.905 ms * *
10 * * *
11 * * *

From here, a Google search for the last backbone router:-

an1.cen-gla.fluency.net.uk

Reveals the ASN AS56595 and one of its IPv4 Address Subnets is:-

185.7.228.0/22

The IP address range of the subnet is:-

185.7.228.0 - 185.7.231.255

The DNS resolved IP address for arnoldclark.com is 185.7.231.66 which is within the above range.

I got the OP to provide me with a Wireshark trace of his attempt to reach this website and it was clear that his PC attempts to set up a TCP session, but there is no response from the server at all.

If I try this, the TCP session is established correctly and I can access the website from my browser.

I can only conclude that there is some routing issue between the AS56595 & the Arnold Clark resolved IP address, which possibly only affects some of TalkTalk's customers.

Thanks.

[KeithFrench]

I think those emails go out periodically for no apparent reason.

I hope you get your internet back up ASAP.

[Nick2345]

Thanks Keith,

I will try to do so as soon as I get off the phone with TT technical support. My internet went all the hell this morning and they appear to have been "doing something". I tried traceroute and it failed with a DNS error (this along with general slowness and inability to load various websites was what indicated an issue with the internet).

They also sent me an email message a little while ago to tell me that my new fibre service was up and running! That would be the fibre service that has been up and running for several years presumably.... I am attempting to find out what they are doing (hence the phone call I am currently holding on).

[KeithFrench]

Hi Nick,

The trace analysis was easy. Your DNS query resolves fine & your PC tries to initialise a TCP session to that DNS-resolved IP address, however, there is no response from their server on that address. Can you please PM me the output of your last tracert?

Thanks,

Keith

[KeithFrench]

Hi Nick,

Thanks for the two files. I will try to have a look today for you. I expect it will either be very easy or rather complex; that's what most analysis is!

[Nick2345]

Hello Keith,

Please find new zip archive attached to this post. Both files should now be present.

Best wishes,

Nick