Once upon a time, there was a university president who couldn’t send an email more than 500 miles, and the wise sysadmin said that’s not possible, so the president said come to my office, and lo and behold, the emails stopped before going 500 miles. Has technology improved? Can we send an email farther than 500 miles in 2025?

There’s a lot to the story that’s obviously made up, but if we fix the details so that it can happen, we can reproduce it.

connect

We need some code to do a nonblocking connect that will timeout quickly. This is mostly copied from the examples in getaddrinfo for lookups and connect, and connect for the nonblocking check, adapted slightly to have a very short timeout and report errors.

The poll timeout is 3ms, as specified by the lore. I think this is nonsense, why would an invalid or incomplete sendmail configuration default to three milliseconds? But that’s the time you get backing a lightspeed delay out of 500 miles. We’ll see if it matters.

code

<span class=bi>#include <sys/types.h></span>
<span class=bi>#include <sys/socket.h></span>
<span class=bi>#include <sys/select.h></span>
<span class=bi>#include <poll.h></span>
<span class=bi>#include <netdb.h></span>
<span class=bi>#include <err.h></span>
<span class=bi>#include <unistd.h></span>
<span class=bi>#include <string.h></span>
<span class=bi>#include <errno.h></span>
<span class=bi>#include <stdio.h></span>

<span class=tp>int</span>
connect_wait<span class=st>(</span><span class=tp>int</span> s<span class=st>)</span>
<span class=st>{</span>
    <span class=tp>struct</span> pollfd pfd<span class=kw>[</span><span class=nm>1</span><span class=kw>]</span>;
    <span class=tp>int</span> error <span class=op>=</span> <span class=nm>0</span>;
    socklen_t len <span class=op>=</span> sizeof<span class=kw>(</span>error<span class=kw>)</span>;

    pfd<span class=kw>[</span><span class=nm>0</span><span class=kw>]</span><span class=op>.</span>fd <span class=op>=</span> s;
    pfd<span class=kw>[</span><span class=nm>0</span><span class=kw>]</span><span class=op>.</span>events <span class=op>=</span> POLLOUT;

    error <span class=op>=</span> poll<span class=kw>(</span>pfd<span class=op>,</span> <span class=nm>1</span><span class=op>,</span> <span class=nm>3</span><span class=kw>)</span>;
    <span class=kw>if</span> <span class=kw>(</span>error <span class=op>==</span> <span class=op>-</span><span class=nm>1</span><span class=kw>)</span>
        <span class=kw>return</span> <span class=op>-</span><span class=nm>1</span>;
    <span class=kw>if</span> <span class=kw>(</span>error <span class=op>==</span> <span class=nm>0</span><span class=kw>)</span> <span class=kw>{</span>
        errno <span class=op>=</span> ETIMEDOUT;
        <span class=kw>return</span> <span class=op>-</span><span class=nm>1</span>;
    <span class=kw>}</span>
    <span class=kw>if</span> <span class=kw>(</span>getsockopt<span class=cm>(</span>s<span class=op>,</span> SOL_SOCKET<span class=op>,</span> SO_ERROR<span class=op>,</span> &error<span class=op>,</span> &len<span class=cm>)</span> <span class=op>==</span> <span class=op>-</span><span class=nm>1</span><span class=kw>)</span>
        <span class=kw>return</span> <span class=op>-</span><span class=nm>1</span>;
    <span class=kw>if</span> <span class=kw>(</span>error <span class=op>!=</span> <span class=nm>0</span><span class=kw>)</span> <span class=kw>{</span>
        errno <span class=op>=</span> error;
        <span class=kw>return</span> <span class=op>-</span><span class=nm>1</span>;
    <span class=kw>}</span>
    <span class=kw>return</span> <span class=nm>0</span>;
<span class=st>}</span>
<span class=tp>int</span>
main<span class=st>(</span><span class=tp>int</span> argc<span class=op>,</span> <span class=tp>char</span> <span class=op>**</span>argv<span class=st>)</span>
<span class=st>{</span>
    <span class=tp>struct</span> addrinfo hints<span class=op>,</span> <span class=op>*</span>res<span class=op>,</span> <span class=op>*</span>res0;
    <span class=tp>int</span> error;
    <span class=tp>int</span> save_errno;
    <span class=tp>int</span> s;

    memset<span class=kw>(</span>&hints<span class=op>,</span> <span class=nm>0</span><span class=op>,</span> sizeof<span class=cm>(</span>hints<span class=cm>)</span><span class=kw>)</span>;
    hints<span class=op>.</span>ai_family <span class=op>=</span> AF_UNSPEC;
    hints<span class=op>.</span>ai_socktype <span class=op>=</span> SOCK_STREAM;
    error <span class=op>=</span> getaddrinfo<span class=kw>(</span>argv<span class=cm>[</span><span class=nm>1</span><span class=cm>]</span> ?<span class=op>:</span> <span class=st>"www.openbsd.org"</span><span class=op>,</span> <span class=st>"www"</span><span class=op>,</span> &hints<span class=op>,</span> &res0<span class=kw>)</span>;
    <span class=kw>if</span> <span class=kw>(</span>error<span class=kw>)</span>
        errx<span class=kw>(</span><span class=nm>1</span><span class=op>,</span> <span class=st>"%s"</span><span class=op>,</span> gai_strerror<span class=cm>(</span>error<span class=cm>)</span><span class=kw>)</span>;
    s <span class=op>=</span> <span class=op>-</span><span class=nm>1</span>;
    <span class=kw>for</span> <span class=kw>(</span>res <span class=op>=</span> res0; res; res <span class=op>=</span> res<span class=op>-></span>ai_next<span class=kw>)</span> <span class=kw>{</span>
        s <span class=op>=</span> socket<span class=cm>(</span>res<span class=op>-></span>ai_family<span class=op>,</span> res<span class=op>-></span>ai_socktype | SOCK_NONBLOCK<span class=op>,</span>
            res<span class=op>-></span>ai_protocol<span class=cm>)</span>;
        <span class=kw>if</span> <span class=cm>(</span>s <span class=op>==</span> <span class=op>-</span><span class=nm>1</span><span class=cm>)</span>
            <span class=kw>continue</span>;

        error <span class=op>=</span> connect<span class=cm>(</span>s<span class=op>,</span> res<span class=op>-></span>ai_addr<span class=op>,</span> res<span class=op>-></span>ai_addrlen<span class=cm>)</span>;
        <span class=kw>if</span> <span class=cm>(</span>error <span class=op>==</span> <span class=op>-</span><span class=nm>1</span> && errno <span class=op>==</span> EINPROGRESS<span class=cm>)</span>
            error <span class=op>=</span> connect_wait<span class=cm>(</span>s<span class=cm>)</span>;
        <span class=kw>if</span> <span class=cm>(</span>error <span class=op>==</span> <span class=op>-</span><span class=nm>1</span><span class=cm>)</span> <span class=cm>{</span>
            warn<span class=bi>(</span><span class=st>"connect failure"</span><span class=bi>)</span>;
            close<span class=bi>(</span>s<span class=bi>)</span>;
            s <span class=op>=</span> <span class=op>-</span><span class=nm>1</span>;
            <span class=kw>continue</span>;
        <span class=cm>}</span>
        <span class=kw>break</span>;  <span class=cm>/* okay we got one */</span>
    <span class=kw>}</span>
    <span class=kw>if</span> <span class=kw>(</span>s <span class=op>==</span> <span class=op>-</span><span class=nm>1</span><span class=kw>)</span>
        <span class=kw>return</span> <span class=op>-</span><span class=nm>1</span>;
    printf<span class=kw>(</span><span class=st>"it's a live one!\n"</span><span class=kw>)</span>;
    freeaddrinfo<span class=kw>(</span>res0<span class=kw>)</span>;
<span class=st>}</span>

The secret here is the kernel will always round 3ms up to at least one whole tick, 10ms. And maybe we get a bit extra from the current tick. So the actual timeout will be 10ms to 19ms. Enough time to make a quick roundtrip even at half c. poll or select with a zero timeout returns immediately, so not much chance of finishing the connection before that happens.

We’re not going to try sending an email with sendmail because I don’t want to wait three days while the message sits in the retry queue before failing. I hope they had some sandwiches delivered to the president’s office while the demonstration was in progress.

success

Let’s try connecting to a few schools, working our way across the country.

ix$ ./connect upenn.edu
it's a live one!
ix$ ./connect uchicago.edu
it's a live one!
ix$ ./connect ucla.edu
it's a live one!

Now I’m getting suspicious, we appear to be bending spacetime.

ix$ ping ucla.edu 
ping: Warning: ucla.edu has multiple addresses; using 3.33.167.235
PING ucla.edu (3.33.167.235): 56 data bytes
64 bytes from 3.33.167.235: icmp_seq=0 ttl=245 time=1.162 ms
64 bytes from 3.33.167.235: icmp_seq=1 ttl=245 time=1.189 ms

Okay, so we have a problem where indeed the schools are not located in spacetime the way we were expecting. Everybody is using cloud whatsit for hosting, and they may well be hosted in the same data center. We haven’t gone 500 miles. For all I know, we haven’t gone 500 feet.

At this point I realized I’m also pinging web servers, not email servers, which we’ll get back to. There’s no reason the story can’t be about a failure to load a web page farther than 500 miles. But the story is funnier if it’s an email, haha, he thought it was strapped to a pigeon leg and the bird got tired.

redo

In search of success (failure), I switch to running the experiment backwards, looking for schools with ping times that are more realistic. After a while, I found a few.

ix$ ping rutgers.edu
PING rutgers.edu (128.6.46.111): 56 data bytes
64 bytes from 128.6.46.111: icmp_seq=0 ttl=241 time=8.896 ms
64 bytes from 128.6.46.111: icmp_seq=1 ttl=241 time=5.768 ms
ix$ ./connect rutgers.edu                                                      
it's a live one!

Rutgers is pretty close, and indeed I can connect. Stretching out a bit farther, we start having some trouble.

ix$ ping cmu.edu 
PING cmu.edu (128.2.42.10): 56 data bytes
64 bytes from 128.2.42.10: icmp_seq=0 ttl=242 time=13.764 ms
64 bytes from 128.2.42.10: icmp_seq=1 ttl=242 time=13.721 ms
ix$ ./connect cmu.edu
it's a live one!
ix$ ./connect cmu.edu
connect: connect failure: Operation timed out
ix$ ping www.maine.edu
PING lv-o-wpc-prod.its.maine.edu (130.111.28.163): 56 data bytes
64 bytes from 130.111.28.163: icmp_seq=0 ttl=51 time=15.255 ms
64 bytes from 130.111.28.163: icmp_seq=1 ttl=51 time=15.310 ms
ix$ ./connect maine.edu
it's a live one!
ix$ ./connect maine.edu
connect: connect failure: Operation timed out

Maine is about 500 miles from here, so it seems we’re right on the edge again.

Dayton is 500 miles in the other direction, but the tubes are just a bit fuller.

ix$ ping udayton.edu
PING udayton.edu (131.238.73.184): 56 data bytes
64 bytes from 131.238.73.184: icmp_seq=0 ttl=235 time=25.429 ms
64 bytes from 131.238.73.184: icmp_seq=1 ttl=235 time=25.437 ms
ix$ ./connect udayton.edu
connect: connect failure: Operation timed out
ix$ ./connect udayton.edu
connect: connect failure: Operation timed out

That’ll never work. As we might have predicted, the speed of light is not significantly different in 2025.

mx

And now to make it biblically accurate, let’s consider some MX servers. This would require switching the lookup code to res_query, or we can just use some existing tools to make observations.

First, get some records for schools.

host -t mx upenn.edu
upenn.edu mail is handled by 10 mxa-00390e01.gslb.pphosted.com.
host -t mx stanford.edu
stanford.edu mail is handled by 10 mxa-00000d07.gslb.pphosted.com.

Okay, so not really much different than www servers. There’s a ton of outsourcing. The ping times are interested however.

ix$ ping mxa-00390e01.gslb.pphosted.com
PING mxa-00390e01.gslb.pphosted.com (148.163.133.158): 56 data bytes
64 bytes from 148.163.133.158: icmp_seq=0 ttl=242 time=62.538 ms
64 bytes from 148.163.133.158: icmp_seq=1 ttl=242 time=62.443 ms
ix$ ping mxa-00000d07.gslb.pphosted.com
PING mxa-00000d07.gslb.pphosted.com (67.231.157.125): 56 data bytes
64 bytes from 67.231.157.125: icmp_seq=0 ttl=246 time=23.796 ms
64 bytes from 67.231.157.125: icmp_seq=1 ttl=246 time=23.564 ms

I can walk across town and hand deliver a postcard to Penn, but I can’t send them an email. Stanford, across the country, is almost but not quite within email range.

ix$ host -t mx ucla.edu
ucla.edu mail is handled by 1 smtp.google.com.
ix$ ping smtp.google.com
ping: Warning: smtp.google.com has multiple addresses; using 142.251.167.27
PING smtp.google.com (142.251.167.27): 56 data bytes
64 bytes from 142.251.167.27: icmp_seq=0 ttl=109 time=6.533 ms
64 bytes from 142.251.167.27: icmp_seq=1 ttl=109 time=6.556 ms

Ha, there we go. I can send an email 3000 miles to UCLA without fear of timeout.

conclusion

The 500 mile limit for perfectly misconfigured servers is still in place, but good luck trying to determine which domains are accessible using a roadmap.