Java Virtual Threads - Part 1

This is a small write up on how java virtual threads work.

How threads work in java

Threads in java are just a wrapper around the actual operating system threads. The JVM requests the operating system to create a thread. Each thread in java has its own components like program counter, call stack etc. There are two types of threads, in the JVM one is the native thread and the java thread. The native thread is the thread that the OS controls to perform various types of operations like controlling internal hardware operations etc.

Are threads expensive ?
Each thread in Java has its own call stack, which is used to store the state of the thread as it executes. When a thread is created, a new call stack is also created for that thread, and this can consume a significant amount of memory, especially if the stack frames are large

What is the solution

Java 16 came up with the solution of virtual threads. A virtual thread stores the stack memory on the heap instead of the stack, allowing it to allocate more memory.

Let us see an example on how to create a virtual thread.

public static void main(String[] args) {
        ExecutorService executorService = Executors.newVirtualThreadExecutor(new ThreadFactory() {
            @Override
            public Thread newThread(Runnable r) {
                return Thread.ofVirtual().start();
            }
        });

        executorService.submit(() -> {
            System.out.println("Hello from virtual thread!");
        });

        System.out.println("Hello from main thread!");

        executorService.shutdown();
    }

In this example, we create a new ExecutorService that is configured to use virtual threads. We define a ThreadFactory that creates new virtual threads using the Thread.ofVirtual() method, which returns a new virtual thread that is ready to be started.

We then submit a new task to the executor service, which will be executed on a virtual thread. The task simply prints a message to the console.

How virtual threads work internally ?

The JVM maintains a pool of platform threads. This is known as the forkJoinPool. This pool contains a pool of around 256 threads which is the maximum limit. It usually depends on the number of cpu cores. When a virtual thread encounters a blocking operation, such as waiting for I/O or a lock, it will block and suspend its execution until the operation completes.

When this happens, the virtual thread's stack chunk (the portion of the stack that the virtual thread is currently using) is copied back to the heap, which frees up the carrier thread to execute other virtual threads that are ready to run. The carrier thread is the underlying thread that is executing the virtual thread's code, and it is responsible for running multiple virtual threads concurrently.

Once the blocked virtual thread finishes the blocking operation, the scheduler will schedule it again for execution. This can happen on the same carrier thread or a different one, depending on the scheduling policy of the virtual thread pool.

In the next part we will take a look at scheduling and how virtual threads are pooled together.