Les crashs dev d'un passionné
Python -Discovering GIL
22 janv. 2020
The Global Intercept Lock is a mechanism called Mutex that allows you to execute only 1 python thread at a given time. It is simply a bit that blocks an area of memory, because the threads in the same process share the same memory area. The execution order of the threads being managed by the CPU itself.
In summary, only one python thread can run in a single CPU at the same time.
The CPython interpreter uses this mechanism in this core because object management is not thread safe. This mechanism is set up to make thread safe the execution of python code.
Behind every python process there is a GIL that handles the threads.
There are other interpreters like Jython which do not have this GIL problem but it is not the most chosen implementation (usually CPython). Be careful if you use another interpreter, some libraries may not work properly.
During I/O operation the GIL is released (lock release), so other available threads can be executed. In this case the multi thread is interesting in python.
During the CPU solicitation (important calculation, video processing, ...) the GIL periodically checks (every 100 tick) which thread is eligible for execution.
In the other cases it is not necessarily interesting to do the threading because the threads are launched in turn by the master thread.
Indeed, it is the master thread that implements the GIL and manages the other threads of the process.
But then why set up the GIL? This is historic because previously the servers were equipped with a single processor and multi threading was not relevant.
The philosophy of python is to do simple things, the language uses a general lock so the developer does not worry about competition problems.
But with the development of multi-core within a CPU the GIL has added a complexity in the management of multi-threaded programs. This is the heart of python, it is difficult to simplify it.
However in version 3.9 of python an evolution is going to take place. Indeed a new library governed by the PEP 554 would launch multiple interpreters within the
same process and therefore potentially launch multiple threads in the same process. It would be the interpreter that would carry the GIL.
The release of this python version 3.9 is scheduled for the end of 2020.In the meantime, check that our program has an interest in using the thread management libraries.
And it is important to know the difference between parallelizable code and concurrency code. Concurrency running code refers to a shared resource unlike parallel code that executes tasks completely independently.
Concurrency code will perform tasks alternatively for example on a machine with only 1 core.
Parallel code will execute the different tasks at the same time in different CPU cores.
Here is a comparison of performance
blog comments powered by Disqus
In summary, only one python thread can run in a single CPU at the same time.
The CPython interpreter uses this mechanism in this core because object management is not thread safe. This mechanism is set up to make thread safe the execution of python code.
Behind every python process there is a GIL that handles the threads.
There are other interpreters like Jython which do not have this GIL problem but it is not the most chosen implementation (usually CPython). Be careful if you use another interpreter, some libraries may not work properly.
During I/O operation the GIL is released (lock release), so other available threads can be executed. In this case the multi thread is interesting in python.
During the CPU solicitation (important calculation, video processing, ...) the GIL periodically checks (every 100 tick) which thread is eligible for execution.
In the other cases it is not necessarily interesting to do the threading because the threads are launched in turn by the master thread.
Indeed, it is the master thread that implements the GIL and manages the other threads of the process.
But then why set up the GIL? This is historic because previously the servers were equipped with a single processor and multi threading was not relevant.
The philosophy of python is to do simple things, the language uses a general lock so the developer does not worry about competition problems.
But with the development of multi-core within a CPU the GIL has added a complexity in the management of multi-threaded programs. This is the heart of python, it is difficult to simplify it.
However in version 3.9 of python an evolution is going to take place. Indeed a new library governed by the PEP 554 would launch multiple interpreters within the
same process and therefore potentially launch multiple threads in the same process. It would be the interpreter that would carry the GIL.
The release of this python version 3.9 is scheduled for the end of 2020.In the meantime, check that our program has an interest in using the thread management libraries.
And it is important to know the difference between parallelizable code and concurrency code. Concurrency running code refers to a shared resource unlike parallel code that executes tasks completely independently.
Concurrency code will perform tasks alternatively for example on a machine with only 1 core.
Parallel code will execute the different tasks at the same time in different CPU cores.
Here is a comparison of performance