I’ve been using HTTP servers for a long time. The first one I ever touched was Apache. Then I mostly used Nginx with Gunicorn. Although I use them every day I’ve never gone into details of their implementation. They just work and I’m fine with this. Nevertheless, last week I decided to write a simple HTTP server in Python with asyncio. Here is what I did.

Asyncio – but why?

In the Python standard library, you will find a library called socket. It provides a simple API to create and use TCP/UDP sockets. It’s enough to create an HTTP server but there is one problem. You can handle one request at the time. No one can connect to the socket until the current request is finished. A server like that is fine for development but you can’t use it in production. When using the NodeJS server you don’t have to worry about this. So how should we do it with Python? I could use threads but there are too many limitations. At first glance, asyncio library seemed promising. Therefore, I’ve decided to use it.

asyncio is a library to write concurrent code using the async/await syntax.

https://docs.python.org/3/library/asyncio.html

Asyncio HTTP server – where to start?

Firstly, I’ve read this blog to learn the basics. It helped me to get a basic understanding of the library. Then I’ve researched the socket library. After that, I was prepared to really write some code.

At the beginning I’ve just created a non-blocking TCP server.

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(('localhost', PORT))
server.listen(1)
server.setblocking(False)

I’ve set family to the socket.AF_INET to use IPv4. When using TCP you are reading a continuous stream of bytes. So I’ve set the type of socket to socket.SOCK_STREAM. Then the server is bind to the selected port on the localhost. I’ve used non-blocking sockets. Therefore, there can be multiple of them active at once.

After that, I’ve added main task named run_server.

async def run_server(selected_server):
    while True:
        client, _ = await loop.sock_accept(selected_server)
        loop.create_task(handle_client(client))

It is added to event loop where it runs until it is completed.

try:
    loop.run_until_complete(run_server(server))
except KeyboardInterrupt:
    server.close()

Asyncio HTTP server – Handle the client

As we can see, the server can run but the requests are not handled yet. So I added handle_client function. It is reading a request from client chunks and returning a response back.

async def handle_client(client):
    request = await read_request(client)
    response = await build_response(request)
    await loop.sock_sendall(client, response)
    client.close()

Read a request

Request from the client is arriving as bytes through the socket. A server needs to read them until there are bytes to read. They are read in small chunks until chunk size is less than a limit.

async def read_request(client):
    request = ''
    while True:
        chunk = (await loop.sock_recv(client, CHUNK_LIMIT)).decode('utf8')
        request += chunk
        if len(chunk) < CHUNK_LIMIT:
            break

    return request

Build a response

After the request is fully read, the response must be created according to received data. The server only serves static files based on the requested URL. So it parses URL and read HTML file from the static folder. If there is no file on the requested path it returns not_found.html. Status and status message are defined when

def url_to_path(url):
    if url in ('/index.html', '/'):
        path = 'index.html'
    else:
        path = re.sub(r'(^/)(.+)(/$)', r'\2', url)

    return path


def parse_request(request_str):
    part_one, part_two = request_str.split('\r\n\r\n')
    http_lines = part_one.split('\r\n')
    method, url, _ = http_lines[0].split(' ')
    if method != 'GET':
        status, status_msg = 405, 'Not allowed'
    else:
        status, status_msg = 200, 'OK'

    return status, status_msg, url


@alru_cache(maxsize=512)
async def load_response(path):
    try:
        async with AIOFile(f"{STATIC_DIR}/{path}", 'r') as afp:
            html = await afp.read()
    except FileNotFoundError:
        async with AIOFile(f"{STATIC_DIR}/not_found.html", 'r') as afp:
            html = await afp.read()

    return html


async def build_response(request):
    status, status_msg, url = parse_request(request)
    html = await load_response(url_to_path(url))
    response = RESPONSE.format(
        status=status,
        status_msg=status_msg,
        html=html
    ).encode('utf-8')

    return response

To improve performance I’ve used alru_cache – LRU cache for asyncio.

Improvements for Aysncio HTTP server

As you can see from the code, it was built just for fun. I didn’t plan it well. Neither did I optimize the code.

You will notice that when a resource is not found, status 200 is returned with not_found.html. Status and message are defined in parse_request. It shouldn’t be there. Nevertheless, it does its job for the purpose.

Conclusion

It was fun building it. As a result, I’ve learned more about sockets and async programming. Therefore, I suggest you build something on your own too.

Github repository

Happy coding!

Read more:

ReDoc code examples in DRF with drf-yasg


0 Comments

Leave a Reply