How I (Sort of) Fixed Telekom's Routing Issues in Hungary [EN]

UPDATE 22-06-2024: Please check out the project website: https://fxtelekom.org

The Problem

In Hungary, accessing Cloudflare services through Telekom’s network can be a pain due to incorrect routing. Even though both Telekom and Cloudflare are directly connected to BIX (Budapest Internet Exchange), Telekom’s selective contract likely excludes Cloudflare’s network.

As a result, all Cloudflare traffic gets routed through DTAG (Deutsche Telekom), ending up in Germany (IPv4) and Austria (IPv6).

Well… routing traffic two countries away is not exactly effective. Interestingly, 1.1.1.1 doesn’t experience such significant slowdowns, but everything else does. The worst part is that this route usually slows down even more in the evening as internet usage peaks

Getting Started

Initially, I considered creating a VPN, but it would route all traffic through the VPN, which isn’t efficient. Playing with static routes wasn’t user-friendly either. I wanted a simple solution without adding overhead to the client’s network. That’s when I came up with an idea.

My Solution

I decided to find a hosting company in Hungary connected to BIX with good routing to Telekom. By setting up an SNI Proxy and DNS server there, I could route all Cloudflare traffic through this server.

I found JZT, which seemed promising with its 10Gbit uplink and good routing to Telekom. I bought a server from them and started working on my idea.

Setting up an SNI Proxy

I set up a simple SNI proxy to forward all incoming TLS connections to the correct server based on the SNI name.

So the working principle is the following:

1. A Request Comes in for example: discord.com
2. The SNI proxy reads the SNI header from TLS
3. Resolves the domain name to IP
4. Forwards the request to that IP

This dynamic solution means the proxy will always forward the request to the correct server, no matter the SNI name. Initially, I wanted to use HAproxy due to its performance, but I ended up using OpenResty for its flexibility.

Configuring Openresty

Now lets see the configurations that I used:

    server {
        listen 443;
        listen [::]:443;
        resolver 9.9.9.9; 
        
        limit_conn conn_limit_per_ip 15;
        
        include AS5483.conf;
        deny all;

        proxy_pass $ssl_preread_server_name:443;
        ssl_preread on;
    }

It’s pretty straightforward. However, I will break it down a bit so that if you have never worked with nginx, you can understand what this configuration does:

1. Listens on port 443 and uses DNS resolver 9.9.9.9 to resolve domains from the SNI header.
2. Limits the number of connections per IP.
3. Only allows connections for AS5483, which is Telekom.
4. Denies all other connections.
5. Forwards the request to the server specified in the SNI header.

I also needed to handle HTTP traffic:

server {
        listen 80;

        location / {
            include AS5483.conf;
            deny all;
            proxy_pass http://$host$request_uri;

        }
    }

I use the Host header here as there is no SNI in plain HTTP.

You might have noticed the AS5438.conf file. This file contains all the allow rules for Telekom. I created a simple bash script to retrieve all Telekom IPs and generate an allow rule for each one:

#!/bin/bash
read -p "AS number: " ASNUM

USER_AGENT="fxtelekom"

#V4
whois -h whois.radb.net -r $USER_AGENT "-i origin $ASNUM" | grep -Eo "([0-9.]+){4}/[0-9]+" | sort -u | sed 's/^/allow /;s/$/;/' > "$ASNUM.conf"
#V6
whois -h whois.radb.net -r $USER_AGENT "-i origin $ASNUM" | grep -Eo "([0-9a-fA-F:]+){8}[\/][0-9]+" | sort -u | sed 's/^/allow /;s/$/;/' >> "$ASNUM.conf"

wait

echo "IP ranges allowed and saved to $ASNUM.conf"

This script will get all IP Ranges from whois DB for the given AS number and generate an allow rule for each one.

Configuring the DNS server

Now I reached the “hard” part: replacing all DNS answers containing Cloudflare’s IP with my own server IP.

For instance, if a DNS request comes in for discord.com and the A and AAAA records for this domain point to Cloudflare, I needed to replace those records with my server IP.

I’m using Knot Resolver because it’s secure by design and scriptable with Lua.

First, I had to obtain Cloudflare’s IP ranges. Fortunately, these ranges are public, so I simply used cURL to retrieve them.

curl https://www.cloudflare.com/ips-v6/ > cloudflare-v6.list
curl https://www.cloudflare.com/ips-v4/ > cloudflare-v4.list

Knot Resolver has a built-in function to replace DNS records. We can use it to replace A and AAAA records with our own server IP. However, what Knot Resolver does not have by default is the ability to read from a list and apply the rules from it.

Lets summarize what I needed:

1. Allow only Telekom IPs
2. Replace Cloudflare’s A and AAAA records with my server IP

Allow only Telekom IPs:

#!/bin/bash
read -p "AS number: " ASNUM

USER_AGENT="fxtelekom"

#V4
whois -h whois.radb.net -r $USER_AGENT "-i origin $ASNUM" | grep -Eo "([0-9.]+){4}/[0-9]+" | sort -u > telekom.list
#V6
whois -h whois.radb.net -r $USER_AGENT "-i origin $ASNUM" | grep -Eo "([0-9a-fA-F:]+){8}[\/][0-9]+" | sort -u >> telekom.list

wait

echo "IP ranges allowed and saved to telekom.list"

Knot Resolver allows to directly write the Lua code inside the configuration file:

local f = io.open('/etc/telekom.list', 'r')
if f then
    for line in f:lines() do
            view:addr(line, policy.all(policy.PASS))
    end
    f:close()
end

Replace Cloudflare’s A and AAAA records with my own server IP:

Next, I configured Knot Resolver to replace A and AAAA records with my own server IP.

local function rerouteIP(filePath, ip)
    local f = io.open(filePath, 'r')
    if f then
        for line in f:lines() do
            modules = {
                renumber = {
                    {line, ip},
                }
            }
        end
        f:close()
    end
end

This function takes two arguments: the path to the file and the IP address to replace. I decided to create a function here because I needed to use this code block multiple times.

As you can see I’m using the renumber module from Knot Resolver.

Now that this part is complete, I can provide a list and an IP address to the function to replace all the IPs in that list:

rerouteIP('/etc/cloudflare-v4.list', "193.188.192.47!")
rerouteIP('/etc/cloudflare-v6.list', "2a09:7ac0::1:2d4b:2dc2!")

By default, Knot Resolver replaces the entire subnet, for example, if I have the subnet 169.254.10.0/24 in the list and a query asking for a domain that has the IP 169.254.10.12, it will replace it with 193.188.192.12. This is actually very cool, but not what I wanted.

The ! at the end of the ip addresses ensures that Knot Resolver replaces the subnet with exact IP address rather than entire subnets.

Done

So the project was done, and OMG it helped a lot.

Here are some before and after results:

Before:

After:

So as you can see there is a HUGE improvement. From 90~ms ping to 10~ms

Keep in mind that the after is just the ping to the JZT server, not really cloudflare. To get the actual response time from cloudflare we need to add the response time from JZT -> Cloudflare.

So we need to add 1~ms to the response time + some processing time from NGINX. But at the end it’s still better than 90~ms.

Conclusion

Whether it is really useful I don’t know. But I was interested to see how well it could work in practice. And also it was a lot of fun and that matters the most.

If you are a Telekom Hungary customer you can try it yourself. You just need to set your DNS to 193.188.192.47 for IPV4 and 2a09:7ac0::1:2d4b:2dc2 for IPV6. But please be aware that this is more like a POC than a real solution. Not recommended in any way to use it long-term or in a production environment!

So I hope you found this article intresting. Thank you for reading!

You can find the configuration files on GitHub! You are free to use it, pull requests and ideas are welcome!