Compilers vs. Interpreters
Computers don’t know how to execute high-level programming languages like C++, at least not directly. In order to translate human-readable code into something your CPU can run, it must be converted. A computer usually does this through one of two methods—compilation or interpretation.
Compilation involves running a compiler, a program that takes source code and converts it into binary machine code, before running the application. The compilation takes place on the dev’s computer before being packaged and sent out. Most executables that you download are compiled at some level to run on your machine, and compiled code is usually pretty fast because the compiler can make optimizations for your particular machine.
However, compilers have a few downsides. Programs must be compiled for specific CPU instructions sets (like x86-64 or ARM). On top of that, even operating systems that share instruction sets (like Windows and Linux, which both run on Intel CPUs) must have programs compiled separately due to the many differences in how they work.
This has the great benefit of not requiring the program to be built for any particular machine; as long as the machine can run the Python interpreter, it can run any Python scripts.
What Does JIT Compilation Do?
Just-In-Time compilation is essentially a compiler that procrastinates, and only compiles the code for each function whenever it is needed. Whenever you call a function, if the JIT compiler hasn’t seen it yet, it will compile that function (applying any optimizations for the machine it’s running on), and run it. The next time you call the function, it already has the machine code on hand, so it just needs to look it up from the cache.
Just-In-Time refers to the fact that the interpreter doesn’t have to compile the whole app all at once. It certainly could, but for a large app it would lead to very high startup times. It’s better for performance to only do compilation when it’s needed (i.e., just in time).
Does JIT Compilation Have a Performance Hit?
JIT is literally just a performance improvement over regular interpreters, so compared to not doing it at all, it’s much faster. Compiling code isn’t particularly fast though, so obviously doing compilation at runtime comes with a performance hit compared to if it was compiled directly to bytecode beforehand.
However, because JIT compilation usually only has to be ran the first time a function is invoked, commonly used functions will only really see a performance hit on the first invocation. You can test this in C# with StopWatches—they pick up .NET “background noise,” including the time spent doing JIT compilation on the first run of a function.
The primary downside of JIT compiled applications is a high cold startup time, as thousands of functions called at startup must be compiled right at the start. Some ASP.NET web applications can take over a minute to bootstrap, in part due to high stress on the JIT compiler at the start.