Virtual thread: Difference between revisions

Revision as of 23:06, 8 February 2024

Revision as of 23:14, 8 February 2024

Line 5:

Virtual threads allows for tens of millions of preemptive tasks and events on a 2021 consumer-grade computer,<ref name="javaworld2">{{cite web |last1=Rudell |first1=Harald |date=2022-03-19 |title=massivevirtualparallelism |url=https://codeberg.org/haraldrudell/massivevirtualparallelism/src/branch/main/README.md |website= |quote= |df= |accessdate=}}</ref> compared to low thousands of operating system threads.<ref>{{Cite web |last=baeldung |date=2022-01-02 |title=Maximum Number of Threads Per Process in Linux {{!}} Baeldung on Linux |url=https://www.baeldung.com/linux/max-threads-per-process |access-date=2022-03-30 |website=www.baeldung.com |language=en-US}}</ref> Preemptive execution<ref>{{Cite web |title=Go 1.14 Release Notes - The Go Programming Language |url=https://go.dev/doc/go1.14#runtime |access-date=2022-03-30 |website=go.dev}}</ref> is important to performance gains through parallelism and fast preemptive response times for tens of millions of events.

Earlier constructs that are not or not always preemptive, such as [[coroutine]]s, [[green thread]]s or the largely single-threaded [[Node.js]], introduce delays in responding to asynchronous events such as every incoming request in a server application.<ref>{{Cite web |last=Node.js |title=Don't Block the Event Loop (or the Worker Pool) |url=https://nodejs.org/en/docs/guides/dont-block-the-event-loop/ |access-date=2022-03-30 |website=Node.js |language=en}}</ref>

== Definition ==

{{More citations needed|section|date=April 2022}}

Virtual threads were commercialized with Google's Chrome browser in 2008<ref>{{Cite web |title=Threading and Tasks in Chrome |url=https://chromium.googlesource.com/chromium/src/+/HEAD/docs/threading_and_tasks.md#core-concepts |access-date=2022-04-05 |website=chromium.googlesource.com}}</ref> where virtual threads may hop physical threads. Virtual threads are truly virtual, created in user-space software.

* Virtual threads are preemptive

Line 36:

== Underlying reasons ==

Java servers have featured extensive and memory consuming software constructs allowing dozens of pooled operating system threads to preemptively execute thousands of requests per second without the use of virtual threads. Key to performance here is to reduce the initial latency in thread processing and minimize the time operating system threads are blocked.<ref>{{Cite web |title=Principles to Handle Thousands of Connections in Java Using Netty - DZone Performance |url=https://dzone.com/articles/thousands-of-socket-connections-in-java-practical |access-date=2022-03-30 |website=dzone.com |language=en}}</ref>

Virtual threads increase possible concurrency by many orders of magnitudes while the actual [[Parallel computing|parallelism]] achieved is limited by available execution units and pipelining offered by present processors and processor cores. In 2021, a consumer grade computers typically offer a parallelism of tens of concurrent execution units.<ref>{{Cite web |title=MacBook Pro 14-inch and MacBook Pro 16-inch |url=https://www.apple.com/macbook-pro-14-and-16/ |access-date=2022-03-30 |website=Apple |language=en-US}}</ref> For increased performance through parallelism, the language runtime need to use all present hardware,<ref>{{Cite web |title=Frequently Asked Questions (FAQ) - The Go Programming Language |url=https://go.dev/doc/faq#number_cpus |access-date=2022-03-30 |website=go.dev}}</ref> not be single-threaded or feature global synchronization such as [[global interpreter lock]].

The many magnitudes of increase in possible preemptive items offered by virtual threads is achieved by the language runtime managing resizable thread stacks.<ref>{{Cite web |title=JEP draft: Virtual Threads (Preview) |url=https://openjdk.java.net/jeps/8277131#Memory-use-and-interaction-with-garbage-collection |access-date=2022-03-30 |website=openjdk.java.net}}</ref> Those stacks are smaller in size than those of operating system threads. The maximum number of threads possible without swapping is proportional to the amount of main memory.<ref>{{Cite web |last=Rudell |first=Harald |date=2022-03-22 |title=Maximum number of virtual threads in Go |url=https://www.reddit.com/r/golang/comments/tifbow/comment/i1owqo9}}</ref>

In order to support virtual threads efficiently, the language runtime has to be largely rewritten to prevent blocking calls from holding up an operating system thread assigned to execute a virtual thread<ref>{{Cite web |last=Szczukocki |first=Denis |date=2020-03-18 |title=Difference Between Thread and Virtual Thread in Java {{!}} Baeldung |url=https://www.baeldung.com/java-virtual-thread-vs-thread |access-date=2022-03-30 |website=www.baeldung.com |language=en-US}}</ref> and to manage thread stacks.<ref>{{Cite web |date=2018-04-12 |title=Why you can have millions of Goroutines but only thousands of Java Threads |url=https://rcoh.me/posts/why-you-can-have-a-million-go-routines-but-only-1000-java-threads/ |access-date=2022-03-30 |website=rcoh.me |language=en-us}}</ref> An example of a retrofit of virtual threads is Java Loom.<ref>{{Cite web |title=Main - Main - OpenJDK Wiki |url=https://wiki.openjdk.java.net/display/loom/Main |access-date=2022-03-30 |website=wiki.openjdk.java.net}}</ref> An example of a new language designed for virtual threads is Go.<ref>{{Cite web |last= |first= |date=2022-03-22 |title=The Go Programming Language |url=https://go.dev/ |access-date=2022-03-30 |website=go.dev}}</ref>

== Complexity ==

Because virtual threads offer parallelism, the programmer needs to be skilled in multi-threaded programming and synchronization.

Because a blocked virtual thread would block the OS thread it occupies at the moment, much effort must be taken in the runtime to handle blocking system calls. Typically, a thread from an pool of spare OS threads is used to execute the blocking call for the virtual thread so that the initially executing OS thread is not blocked.

Management of the virtual thread stack requires care in the linker and short predictions of additional stack space requirements.

== Implementations ==

=== Google Chrome Browser ===

Virtual threads are used to serialize singleton input/output activities. When a virtual thread is executing, it can hop on different OS thread. The Chrome browser first appeared in 2008. Chrome's virtual threads are available to developers extending the browser.

=== Go ===

goroutines became preemptive with go1.4 in 2014 and is since the most prominent application of virtual threads. A goroutine is a virtual thread. Go first appeared in 2009.

=== Java Project Loom ===

[https://openjdk.java.net/projects/loom/ Project Loom]: ''Virtual threads'' is a lightweight user-mode scheduled alternative to standard OS managed threads. Virtual threads are mapped to OS threads in a many-to-many relationship. Work on project loom by Oracle started in 2017. Loom has the goal of implementing virtual threads for performance, at the same time simplifying thread handling across OS threads, concurrent threads and virtual threads. As of 2022, Project Loom is available as early-access using JDK 19.

== Other uses of the term ==