Virtual thread: Difference between revisions

Revision as of 19:06, 5 April 2022

Latest revision as of 14:57, 11 April 2025

(36 intermediate revisions by 19 users not shown)

Line 1:

{{TOC limit|2}}

{{AFC submission|d|exists|Green threads|u=Haraldrudell|ns=118|decliner=Robert McClenon|declinets=20220405062331|ts=20220405015607}} <!-- Do not remove this line! -->

{{Short description|Computational threads scheduled by a run-time library}}

{{AFC comment|1=You may edit the existing article to add any material that is not already present. If this is a request to rename, please discuss via a [[WP:MR|Move Request]], [[User:Robert McClenon|Robert McClenon]] ([[User talk:Robert McClenon|talk]]) 06:23, 5 April 2022 (UTC)}}

{{Distinguish|green thread}}

In [[computer programming]], a '''virtual thread''' is a [[Thread (computing)|thread]] that is managed by a [[runtime library]] or [[virtual machine]] (VM) and made to resemble "real" operating system thread to code executing on it, while requiring substantially fewer resources than the latter.

In [[computer programming]], '''virtual threads''' are [[Thread (computing)|threads]] that are scheduled by a [[runtime library]] instead of natively by the underlying [[operating system]] (OS). Virtual threads allows for tens of millions of preemptive tasks and events without swapping on a 2021 consumer-grade computer.,<ref name="javaworld2">{{cite web |last1=Rudell |first1=Harald |date=2022-03-19 |title=massivevirtualparallelism |url=https://codeberg.org/haraldrudell/massivevirtualparallelism/src/branch/main/README.md |website= |quote= |df= |accessdate=}}</ref>, compared to low thousands of operating system threads.<ref>{{Cite web |last=baeldung |date=2022-01-02 |title=Maximum Number of Threads Per Process in Linux {{!}} Baeldung on Linux |url=https://www.baeldung.com/linux/max-threads-per-process |access-date=2022-03-30 |website=www.baeldung.com |language=en-US}}</ref>. Preemptive execution<ref>{{Cite web |title=Go 1.14 Release Notes - The Go Programming Language |url=https://go.dev/doc/go1.14#runtime |access-date=2022-03-30 |website=go.dev}}</ref> is important to performance sincegains constructsthrough thatparallelism areand notfast preemptive, suchresponse astimes [[Coroutine|coroutines]]for ortens theof largelymillions single-threaded [[Node.js]], introduce delays in responding to asynchronousof events such as every incoming request in a server application<ref>{{Cite web |last=Node.js |title=Don't Block the Event Loop (or the Worker Pool) |url=https://nodejs.org/en/docs/guides/dont-block-the-event-loop/ |access-date=2022-03-30 |website=Node.js |language=en}}</ref>.

----

Earlier constructs that are not or not always preemptive, such as [[coroutine]]s, [[green thread]]s or the largely single-threaded [[Node.js]], introduce delays in responding to asynchronous events such as every incoming request in a server application.<ref>{{Cite web |last=Node.js |title=Don't Block the Event Loop (or the Worker Pool) |url=https://nodejs.org/en/docs/guides/dont-block-the-event-loop/ |access-date=2022-03-30 |website=Node.js |language=en}}</ref>

{{Draft topics|software|computing|technology}}

{{AfC topic|stem}}

In [[computer programming]], '''virtual threads''' are [[Thread (computing)|threads]] that are scheduled by a [[runtime library]] instead of natively by the underlying [[operating system]] (OS). Virtual threads allows for tens of millions of preemptive tasks and events without swapping on a 2021 consumer-grade computer.<ref name="javaworld2">{{cite web |last1=Rudell |first1=Harald |date=2022-03-19 |title=massivevirtualparallelism |url=https://codeberg.org/haraldrudell/massivevirtualparallelism/src/branch/main/README.md |website= |quote= |df= |accessdate=}}</ref>, compared to low thousands of operating system threads<ref>{{Cite web |last=baeldung |date=2022-01-02 |title=Maximum Number of Threads Per Process in Linux {{!}} Baeldung on Linux |url=https://www.baeldung.com/linux/max-threads-per-process |access-date=2022-03-30 |website=www.baeldung.com |language=en-US}}</ref>. Preemptive execution<ref>{{Cite web |title=Go 1.14 Release Notes - The Go Programming Language |url=https://go.dev/doc/go1.14#runtime |access-date=2022-03-30 |website=go.dev}}</ref> is important to performance since constructs that are not preemptive, such as [[Coroutine|coroutines]] or the largely single-threaded [[Node.js]], introduce delays in responding to asynchronous events such as every incoming request in a server application<ref>{{Cite web |last=Node.js |title=Don't Block the Event Loop (or the Worker Pool) |url=https://nodejs.org/en/docs/guides/dont-block-the-event-loop/ |access-date=2022-03-30 |website=Node.js |language=en}}</ref>.

== Definition ==

{{DraftsMore movedcitations from mainspaceneeded|section|date=April 2022}}

The term Virtual threads has been used ambiguously over the years. For example Intel<ref>{{Cite web |title=Intel Technology Journal |url=https://www.intel.com/content/dam/www/public/us/en/documents/research/2007-vol11-iss-4-intel-technology-journal.pdf}}</ref> in 2007 spoke about their hyper-threading hardware as virtual threads. Virtual threads were commercialized with Google’s Chrome browser in 2008<ref>{{Cite web |title=Threading and Tasks in Chrome |url=https://chromium.googlesource.com/chromium/src/+/HEAD/docs/threading_and_tasks.md#core-concepts |access-date=2022-04-05 |website=chromium.googlesource.com}}</ref> where virtual threads may hop physical threads. Virtual threads are truly virtual, created in user-space software

* '''Virtual Threadsthreads are preemptive'''

** This is importantImportant for response performance, that thea virtual thread can react to events without programmer intervention or before concluding a current task.

* Preemption requires knowledge of multi-threaded programming to avoid torn writes, data races, and invisible writes by other threads.

* '''Virtual Threadsthreads can hop over the execution units of all processors and cores'''

** This allows forbetter usingutilisation allof available hardware, a 10x increase on today’s computers.

** InGo the(since go1version 1.18) implementation, there areuses virtual thread queues per execution unit. There are additional virtual threads not allocated to an execution unit and an execution unit can steal virtual threads from another execution unit.<ref>{{Cite web |last=Lu |first=Genchi |date=2021-07-22 |title=Java’sJava's Thread Model and Golang Goroutine |url=https://medium.com/@genchilu/javas-thread-model-and-golang-goroutine-f1325ca2df0c |access-date=2022-04-05 |website=Medium |language=en}}</ref>

* '''Virtual threads require no yield or similar interventions by the programmer'''

** Virtual threads appear to execute continuously until they return or stop at a synchronization lock .

** Unlike coroutines, if a virtual thread is in an infinite loop, it does not block the program. Execution continues at a higher cpuCPU load, even if there are more looping threads than available execution units.

* '''Virtual Threadsthreads can existsnumber in the tens of millions by featuring small often managed stacks'''

** This allows for manyseveral magnitudes more threads than fromit would be possible using OSoperating system threads.

** Go 1.18 can havelaunch approximately15 350,000million virtual threads withon lessa than2021 3consumer-grade KiBcomputer, stacki.e. about 350,000 per gigabyte of main memory,. ieThis atis leastenabled 15by milliongoroutines onhaving a 2021resizable, consumer-gradeless computerthan 3 KiB stack.

** A consumer grade computer typically supports 3,000 OS threads an can through system configuration offer maybe 15,000

* '''Virtual Threadsthreads can be allocated quickly, similar to the rate of memory allocations'''

** Because allocatingallocation of a virtual thread ishas akinlittle tooverhead allocateon top of allocating memory structures, they can be allocated very quickly, like 600,000 per second. This is not possible for OS threads that would crash the host long before this rate

** The quicker ramp-up lessens the need for thread-pools of pre-launched threads to cater for sudden increases in traffic.

** In Go, a virtual thread is allocated using a function call preceded by the keyword go. The function call provides a closure of variable values guaranteed to be visible to the to the new goroutine. goroutines have no return value, so a goroutine that returns just disappears

* '''Virtual threads share memory maplike likeoperating OSsystem threads'''

** Like OS threads, virtual threads share the memory across the process and can therefore freely share and access memory objects subject to synchronization.

** Some single-threaded architectures, such as the V8 ECMAScript engine used by Node.js, do not readily accept data that the particular thread did not allocate, requiring special zero-copy data types to be used when sharing data between threads .

* '''Virtual threads offer parallelism like OSoperating system threads'''

** Parallelism means that multiple instructions are executed truly at the same time which typically leads to a magnitude of faster performance.

** TheThis is different from the simpler concurrency, in which a single resourceexecution isunit executes multiple threads shared in small time increments. soThe thattime-slicing itmakes appearseach alwaysthread available,appear to be continuously executing. While concurrency is easier to implement and program, butit dodoes not offer any gains in performance.

== Underlying reasonsMotivation ==

Java servers have featured extensive and memory consuming software constructs allowing dozens of pooled operating system threads to preemptively execute thousands of requests per second without the use of virtual threads. Key to performance here is to reduce the initial latency in thread processing and minimize the time operating system threads are blocked.<ref>{{Cite web |title=Principles to Handle Thousands of Connections in Java Using Netty - DZone Performance |url=https://dzone.com/articles/thousands-of-socket-connections-in-java-practical |access-date=2022-03-30 |website=dzone.com |language=en}}</ref>

Virtual threads increase possible concurrency by many orders of magnitudes while the actual [[Parallel computing|parallelism]] achieved is limited by available execution units and pipelining offered by present processors and processor cores. In 2021, a consumer grade computers typically offer a parallelism of tens of concurrent execution units.<ref>{{Cite web |title=MacBook Pro 14-inch and MacBook Pro 16-inch |url=https://www.apple.com/macbook-pro-14-and-16/ |access-date=2022-03-30 |website=Apple |language=en-US}}</ref>. For increased performance through parallelism, the language runtime need to use all present hardware,<ref>{{Cite web |title=Frequently Asked Questions (FAQ) - The Go Programming Language |url=https://go.dev/doc/faq#number_cpus |access-date=2022-03-30 |website=go.dev}}</ref>, not be single-threaded or feature global synchronization such as [[global interpreter lock]].

The many magnitudes of increase in possible preemptive items offered by virtual threads is achieved by the language runtime managing resizable thread stacks.<ref>{{Cite web |title=JEP draft: Virtual Threads (Preview) |url=https://openjdk.java.net/jeps/8277131#Memory-use-and-interaction-with-garbage-collection |access-date=2022-03-30 |website=openjdk.java.net}}</ref>. Those stacks are smaller in size than those of operating system threads. The maximum number of threads possible without swapping is proportional to the amount of main memory.<ref>{{Cite web |last=Rudell |first=Harald |date=2022-03-22 |title=Maximum number of virtual threads in Go |url=https://www.reddit.com/r/golang/comments/tifbow/comment/i1owqo9}}</ref>

In order to support virtual threads efficiently, the language runtime has to be largely rewritten to prevent blocking calls from holding up an operating system thread assigned to execute a virtual thread<ref>{{Cite web |last=Szczukocki |first=Denis |date=2020-03-18 |title=Difference Between Thread and Virtual Thread in Java {{!}} Baeldung |url=https://www.baeldung.com/java-virtual-thread-vs-thread |access-date=2022-03-30 |website=www.baeldung.com |language=en-US}}</ref> and to manage thread stacks.<ref>{{Cite web |date=2018-04-12 |title=Why you can have millions of Goroutines but only thousands of Java Threads |url=https://rcoh.me/posts/why-you-can-have-a-million-go-routines-but-only-1000-java-threads/ |access-date=2022-03-30 |website=rcoh.me |language=en-us}}</ref>. An example of a retrofit of an existing runtime with virtual threads is Java's ''Project Loom''.<ref>{{Cite web |title=Main - Main - OpenJDK Wiki |url=https://wiki.openjdk.java.net/display/loom/Main |access-date=2022-03-30 |website=wiki.openjdk.java.net}}</ref>. An example of a new language designed for virtual threads is Go.<ref>{{Cite web |last= |first= |date=2022-03-22 |title=The Go Programming Language |url=https://go.dev/ |access-date=2022-03-30 |website=go.dev}}</ref>

== Complexity ==

Because virtual threads offer parallelism, the programmer needs to be skilled in multi-threaded programming and synchronization.

Because a blocked virtual thread would block the OS thread it occupies at the moment, much effort must be taken in the runtime to handle blocking system calls. Typically, a thread from ana pool of spare OS threads is used to execute the blocking call for the virtual thread so that the initially executing OS thread is not blocked.

Management of the virtual thread stack requires care in the linker and short predictions of additional stack space requirements.

== Implementations ==

=== Google Chrome Browser ===

Virtual threads are used to serialize singleton input/output activities and available to developers extending the browser. When a virtual thread is executing, it can hop on a different OS thread.<ref>{{Cite Theweb Chrome|title=Threading browserand first appearedTasks in 2008Chrome |url=https://chromium.googlesource.com/chromium/src/+/HEAD/docs/threading_and_tasks.md#core-concepts |access-date=2022-04-05 |website=chromium.googlesource.com}}</ref>

=== Go ===

Go's ''goroutines'' became preemptive with go1Go 1.4 in 2014 and isare since the mosta prominent application of virtual threads. A goroutine is a virtual thread. Go first appeared in 2009

=== Java Project Loom ===

Java introduced virtual threads in 2023 with Java 21, with the limitation that any code running on a virtual thread which uses ''synchronised'' blocks or native calls will become pinned to its carrier OS thread.<ref>{{Cite web |title=Virtual Threads |url=https://docs.oracle.com/en/java/javase/21/core/virtual-threads.html |access-date=2024-09-10 |website=Oracle Help Center |language=en-US}}</ref> The former limitation was fixed in Java 24.<ref>{{Cite web |title=JEP 491: Synchronize Virtual Threads without Pinning |url=https://openjdk.org/jeps/491 |access-date=2025-03-30 |website=OpenJDK |language=en-US}}</ref>

== Other uses of the term ==

Intel<ref>{{Cite web |title=Intel Technology Journal |url=https://www.intel.com/content/dam/www/public/us/en/documents/research/2007-vol11-iss-4-intel-technology-journal.pdf}}</ref> in 2007 referred to an Intel compiler specific optimization technique as virtual threads.

=== Java Project Loom ===

[https://openjdk.java.net/projects/loom/ Project Loom]: ''Virtual threads'' is a lightweight user-mode scheduled alternative to standard OS managed threads. Virtual threads are mapped to OS threads in a many-to-many relationship. Work on project loom by Oracle started in 2017. Loom has the goal of implementing virtual threads for performance, at the same time simplifying thread handling across OS threads, concurrent threads and virtual threads. As of 2022, Project Loom is available as early-access using JDK 19

== See also ==

Line 85 ⟶ 87:

* [https://codeberg.org/haraldrudell/massivevirtualparallelism/src/branch/main/README.md massivevirtualparallelism Go program testing limits on virtual threads]

[[Category:Threads (computing)]]

__FORCETOC__

[[Category:Virtualization]]

{{Drafts moved from mainspace|date=April 2022}}