Modern software development largely relies on concurrent programming. When applications become composite, the luxury of handling one at a time is not sustainable. asyncio library in Python contains powerful and flexible tools that allow the creation of complex concurrent applications.
This blog by Technoligent – A Python Development Company India, gives an extensive coverage of `asyncio` from basics, best practises, more advanced features and real-world examples.
Understanding Asyncio: Core Concepts
‘asyncio’ is an application added to Python so that the implementation of asynchronous programming Walden 8 This is made possible by implementing concurrency with activities being done simultaneously with the help of a Parallel Processing environment but devoid of the overhead that accompanies multithreaded application.
1. The Event Loop
The event loop is at the centre and is basic to asyncio. It is supposed to be in charge of monitoring and organising the performance of asynchronous operations. The event loop exists in a single thread which means that tasks do not have chances of interfering with each other.
– How It Works: The event loop checks whether there is a task that is, set up for processing, and puts it through the process. Whenever a task is performing an I/O operation (for example, a network request), the control returns to the loop, and other tasks may be executed.
– Example:
“`python
import asyncio
async def say_hello():
print(“Hello”)
await asyncio.sleep(1)
print(“World”)
asyncio.run(say_hello())
“`
2. Async/Await Syntax
The `async` and `await` keywords are fundamental to writing `asyncio` programmes.
– `async`: Marks a function to be an asynchronous function enabling the use of the keyword await.
– `await`: Suspends a coroutine until the expected action has been completed.
3. Coroutines and Tasks
A Coroutine is a Python function which is created using the keyword async def. Tasks are a higher abstraction level, the event loop schedules coroutines thus created for further execution.
– Creating Tasks:
“`python
async def main():
task = asyncio.create_task(say_hello())
await task
“`
Advanced Techniques with Asyncio
We will also show that `asyncio` looks even more powerful when used together with the threading and multiprocessing paradigms. With these forms of advanced techniques it becomes easy for developers to put up systems that are very efficient.
1. The integration of Asyncio with Threading
Threading is appropriate for running tasks that involve much computation or that block I/O operations. Here you can learn how to run blocking functions in a separate thread with the help of asyncio.to_thread without the event loop freezing.
– Example:
“`python
import asyncio
import time
def blocking_task():
time.sleep(5)
print(“Blocking task completed”)
async def main():
await asyncio.to_thread(blocking_task)
asyncio.run(main())
“`
2. Integrating with Multiprocessing
Multiprocessing is better than threading for CPU tasks because it involves use of multiple CPU cores. With `asyncio.run_in_executor()` the opportunities are provided to transfer tasks to a process pool.
– Example:
“`python
import asyncio
from concurrent.futures import ProcessPoolExecutor
def cpu_bound_task(n):
return sum(i i for i in range(n))
async def main():
with ProcessPoolExecutor() as executor:
result = await asyncio.get_event_loop().run_in_executor(executor, cpu_bound_task, 10_000_000)
print(f”Result: {result}”)
asyncio.run(main())
“`
Building Robust Asynchronous Applications
`asyncio’ stands out in real-life applications for such processes as web scraping, server-side programming. It is time to focus on what it can do with it is strengths, so let us consider some examples.
1. Web Scraper with Asyncio
An asynchronous web scraper for example using aiohttp will be able to scrape multiple web pages at the same time thereby bringing down the total time taken considerably.
– Example:
“`python
import asyncio
import aiohttp
async def fetch(url):
async with aiohttp.ClientSession() as session:
async with session.get(url) as response:
return await response.text()
async def main():
urls = [“https://example.com” for _ in range(5)]
tasks = [fetch(url) for url in urls]
results = await asyncio.gather(tasks)
print(“Fetched”, len(results), “pages”)
asyncio.run(main())
“`
2. Server-Side Applications
`asyncio` is ideal for building lightweight servers that handle multiple client connections simultaneously.
– Example:
“`python
import asyncio
async def handle_client(reader, writer):
data = await reader.read(100)
message = data.decode()
print(f”Received: {message}”)
writer.write(data)
await writer.drain()
writer.close()
async def main():
server = await asyncio.start_server(handle_client, “127.0.0.1”, 8888)
async with server:
await server.serve_forever()
asyncio.run(main())
“`
Performance Considerations and Best Practices
Before proceeding let us outline some of the pros and cons, as well as some of the guidelines while dealing with `asyncio`.
1. Avoid Blocking Calls
Many operations that are blocking to the event loop are the calls, for example, `time.sleep` or any a strong computational burden. They are to utilise the asynchronous equivalents of those locks such as `asyncio.sleep`.
2. Limit Concurrency
When too many tasks are entered into the system, they can overload it. To control the concurrency level use `asyncio.Semaphore`
– Example:
“`python
semaphore = asyncio.Semaphore(10)
async def limited_task():
async with semaphore:
await asyncio.sleep(1)
“`
3. Profile and Monitor
To profile your asynchronous code, the recommended profiling tools are things like `aiomonitor` to analyse performance issues.
4. Error Handling
Use try-except to enclose tasks to reduce chances of getting exceptions and know how to handle them.
– Example:
“`python
async def safe_task():
try:
await risky_task()
except Exception as e:
print(f”Error: {e}”)
“`
Interactive Coding Exercise
Let us implement a common scenario: downloading multiple files concurrently.
Exercise: File Downloader
“`python
import asyncio
import aiohttp
async def download_file(url, filename):
async with aiohttp.ClientSession() as session:
async with session.get(url) as response:
content = await response.read()
with open(filename, “wb”) as file:
file.write(content)
async def main():
urls = [“https://example.com/file1”, “https://example.com/file2″]
tasks = [download_file(url, f”file{i}.txt”) for i, url in enumerate(urls)]
await asyncio.gather(tasks)
asyncio.run(main())
“`
Pitfalls to Avoid
– Skipping Exception Handling: Always handle exceptions in individual tasks to avoid crashes.
– Unbounded Concurrency: Use semaphores or controlling mechanisms.
FAQ
What is the difference between `asyncio` and threading?
asyncio: It has a singular thread for running tasks; whereas, threading synchronises multiple threads concurrently.
Can `asyncio` run CPU-bound tasks?
The `asyncio` library is great for programmes that will primarily consist of I/O operations. For I/O bound tasks you should use threading or else run the task in a separate process.
How do I debug `asyncio` applications?
Some tools are used for debugging; for example, `asyncio.run(debug=True)` and `aiomonitor` is the library for that purpose.
What are common pitfalls in `asyncio`?
Specific examples of bad practises are: Blocking calls, unbounded concurrency, ignoring exception handling.
How does `asyncio` improve performance?
`asyncio` helps to minimise the idle time resource, bring in the order, and latency of tasks.
Conclusion
The asyncio library has been written in Python and it will have definitive roles in concurrent programming. It allows the developer to create efficient and scalable application with little or no additional complication. If you learn about asynchronous programming from its most basic level, understand as well as learn general concepts and advanced techniques, and make sure to practise most important asynchronous programming techniques, you would be able to use all features that are offered by this style of programming. In case of developing web scrapers, servers or any other I/O-bound system, It is worth mastering ‘asyncio’.
