Multi-threading in python
Multi-Threading or writing Asynchonous requests occurs in python
A thread is a piece in a process within Python, a process has three main components:
An executable Program
The data needed by the program whether it is a buffer or variable
The programs status/ the execution of the program
A thread is basically a subdivision of a process within python, one of the three above. While a process is its own execution unit. Threads within a process will share resources like memory, open files, and network connections. While a process will have it’s own resources allocated.
WHEN IS MUILTI THREADING USED?
You need to perform multiple tasks concurrently within a single process. For example, In applications that require real-time responsiveness, such as games or video conferencing, using threads can help ensure that tasks like rendering graphics, processing audio, or handling user input are executed more efficiently. Or in asynchronous programming threads are often used to implement asynchronous programming patterns, where tasks can be started and then paused or resumed without blocking the main thread of execution. This can make your application more responsive and efficient.
You’re willing to share resources for multiple processes. A web server is a good example of this. It uses threads to handle multiple concurrent requests. When a user requests a web page, the server creates a new thread to process that request. This allows the server to handle multiple requests simultaneously, improving performance and responsiveness.
THREADING ARCHITECTURE
image from geekforgeeks.com
In the single threaded process
each Instruction is executed one after another.
All parts of the process share the same code, data, files, registers, and stack.
In the Multi-Threaded process
It contains multiple threads of execution within the same process.
Different threads can execute instructions simultaneously.
All threads share the same code, data, and files, but each thread has its own registers and stack.
HOW TO THREAD MANUALLY IN PYTHON
The threading module is part of the standard Python Library so you don’t really need to do much. To import it we are going to make the header for our code ‘import threading’
code explanation
‘import threading’ imports the threading module from the python library which provides tools for creating and managing threads in python.
define functions (def) defines the function blocks for printsquare and print_cube to calculate and print the square and the cube for the value that we give it
create threads two threads are created t1 and t2 for threading.Thread one for printsquare and print_cube respectively. As well as an argument (args) for our value which is 100.
Start threads start() which executes both threads
Join threads join() which waits for both threads to finish before .py is executed
print(“done”) obv print that it is done once the code has been executed
Our finished result should provide us with the square and the cube of the argument we have given it at the same time.
python threadpool
Python Threadpool is a collection of pre-created threads thats used execute tasks concurrently. It's a pretty common pattern in multi-threaded programming, especially when dealing with I/O-bound tasks or tasks that require a large number of concurrent operations.
A thread pool is created with a specified number of threads. These threads are typically kept idle, waiting for tasks to be assigned.
When a new task (e.g., a function to be executed) is submitted to the thread pool, it's assigned to an available thread.
The thread executes the task.
Once a thread finishes its task, it returns to the pool and can be reused to execute another task.
We can use Python’s ThreadPoolExecutor class to create and manage threads which is part of the concurrent.features module.
code explanation
import concurrent.features imports concurrent.features module, which provides tools for asynchronous execution (multi-threading) in Python. To create thread pools and submit tasks for execution.
The process_data function defines a function that will be executed by the threads in the pool and takes a data argument as its input. The function performs some processing on the data ( multiplying it by 2) and returns the result.
Define main function which is the main entry point of the program. It creates a list containing the data elements that will be processed. And creates a ThreadPoolExecutor object with a maximum of 4 worker threads. This object manages a pool of threads that can execute tasks concurrently.
The Map method to submit the Process_data function to the thread pool for each element in the data_list. This distributes the tasks among the available threads for parallel execution.
Then stores the results of the processed data in the results variable.
the main function ensures that the main function is only executed when the script is run directly, not when it's imported as a module.
important notes
GIL global interpreter lock- Python's GIL prevents multiple threads from executing Python bytecode simultaneously. This means that even with multiple threads, only one thread can execute Python code at a time
Ensure that the libraries you're using are thread-safe. Some libraries may not be designed to handle concurrent access from multiple threads.
In some cases, asynchronous programming (e.g., using asyncio) might be a better alternative to multithreading, especially for I/O-bound tasks.