Talk:Load testing: Difference between revisions

Revision as of 14:41, 15 June 2007

Latest revision as of 13:05, 5 February 2024

(34 intermediate revisions by 16 users not shown)

Line 1:

{{WikiProject banner shell|class=Start|

{{WikiProject Technology}}

}}

The link to Grinder is wrong. It should be http://grinder.sourceforge.net/ <small><span class="autosigned">—Preceding [[Wikipedia:Signatures|unsigned]] comment added by [[User:ToddCostella|ToddCostella]] ([[User talk:ToddCostella|talk]] • [[Special:Contributions/ToddCostella|contribs]]) 17:40, 22 January 2010 (UTC)</span></small><!-- Template:Unsigned --> <!--Autosigned by SineBot-->

]]]]]=='''''VUser'''''==

What is a VUser? As someone who knows little about this topic, a sentence or two on its definition would be helpful. [[User:Ssnseawolf|Ssnseawolf]] ([[User talk:Ssnseawolf|talk]]) 15:39, 25 February 2009 (UTC)

== '''''helfulTool''''' ==

What is the criteria for listing vendors on the tools section? Marketshare? Annual sales? Popularity?

Line 5 ⟶ 13:

What is the criteria for being listed in the tools section? The first document, for example, is not about the load testing process, but is a software manual, while an article about how to do load testing on .NET applications is removed as "spam". [[User:Czei|Czei]] 14:43, 13 March 2007 (UTC)

The two pages should be merged since the separate "tools" section can't possibly work given the guidelines: "Promotional articles about yourself, your friends, your company or products, or articles created as part of a marketing or promotional campaign, will be deleted in accordance with our deletion policies." Any listing of "tools" would have to include products, and therefore would be categorized as "marketing". To be fair, any listing of "tools" which have to include all tools or none, otherwise you'd have to get into the criteria why some are listed others are not. ([[User:Czei|Czei]] 14:41, 15 June 2007 (UTC))

. <small>—Preceding [[Wikipedia:Signatures|unsigned]] comment added by [[Special:Contributions/121.243.192.105|121.243.192.105]] ([[User talk:121.243.192.105|talk]]) 06:15, 21 November 2007 (UTC)</small><!-- Template:UnsignedIP --> <!--Autosigned by SineBot-->

== Software Load Testing is Misleading ==

The world of software load testing is changing and this article (as well as the list of load testing tools) is out of date, incomplete, and misleading. Disclaimer: I'm the founder of a new startup, [http://browsermob.com BrowserMob], and I have a very clear conflict of interest and therefore cannot make edits to any of these pages.

This article on claims that software load testing shares very little similarity with functional testing tools, primarily because load testing operates on the protocol level whereas functional testing tools work at the GUI level. I have several concerns with this:

* This paragraph would much more informational if it explained why most load testing tools work at the protocol level. Ex: because GUI-level testing for even 1 VU consumes much more memory that is practical to use for large scale load testing.

* This is no longer true. In November 2008, I launched BrowserMob, which provides load testing via [[Selenium_(software)]], a functional testing tool. It uses Selenium for both record and playback. Therefore, this is a load testing tool which does not operate at the protocol level and instead does indeed work just like a functional testing tool.

* The technique of protocol level playback is becoming more of a problem as web applications evolve and become more complex. I think this topic needs to be discussed somewhere. I recently published an article on http://ajaxian.com that discusses [http://ajaxian.com/archives/why-load-testing-ajax-is-hard why load testing Ajax applications is hard].

BrowserMob works by leveraging cloud computing, which makes it affordable to actually use real browsers for GUI-based interaction in a load test. This technique has also been [http://developer.amazonwebservices.com/connect/thread.jspa?messageID=77249 referenced independently] by people on the Amazon EC2 forums.

I also agree that the load testing tools section seems a bit odd. Some of the tools referenced are just not accurate ([[HtmlUnit]] and Selenium are almost never used for load testing, except in the case of Selenium with BrowserMob). Also, some commercial tools are referenced while some are not, which seems to violate the neutrality of Wikipedia.

I think a better approach would be to eliminate the category page and roll it up in to this article, which a more in-depth discussion of the different approaches and techniques. I would also remove HtmlUnit and Selenium entirely and put them in the proper category. I would love to do this myself (I'm the founder of OpenQA and SeleniumHQ, so I have the expertise) but I clearly have a conflict of interest.

Any thoughts would be appreciated!

[[User:Plightbo|Plightbo]] ([[User talk:Plightbo|talk]]) 18:17, 3 January 2009 (UTC)

==Market Share==

A comment left by a recent editor reads: "none of these tools have 5% market share (unless you only include Fortune 500)". I would argue that a tools survey, such as the one done by Better Software Magazine and StickyMinds.com is sufficient for determining market share. --[[User:Walter Görlitz|Walter Görlitz]] ([[User talk:Walter Görlitz|talk]]) 01:12, 16 January 2010 (UTC)

=== Poor Criteria ===

It is interesting to note that this article does not even mention Apache Bench, yet Apache Bench is distributed with every instance of Apache Web Server and I would contend that gives them at least 60%. Arbitrary and Capricious seems to be the watch word for this article. <small><span class="autosigned">— Preceding [[Wikipedia:Signatures|unsigned]] comment added by [[User:AreYouFreakingKidding|AreYouFreakingKidding]] ([[User talk:AreYouFreakingKidding|talk]] • [[Special:Contributions/AreYouFreakingKidding|contribs]]) 17:46, 6 December 2012 (UTC)</span></small><!-- Template:Unsigned --> <!--Autosigned by SineBot-->

: Feel free to provide a reference to support that statement. ''Use'' of JMeter is common while ''use'' of ApacheBench isn't. The tools surveys that I have seen from ''Better Software'' and other locations support that. While it may be distributed, it whether it's used or not that counts for market share. We don't just add thinks because someone thinks they should be added. --[[User:Walter Görlitz|Walter Görlitz]] ([[User talk:Walter Görlitz|talk]]) 18:00, 6 December 2012 (UTC)

==Computational Load Testing==

I think there should be something that ties this subject into the computer world with computer stress tests and servers/workstations. It would be usefull to try to link to the [[Stress_testing#Computer_processors|CPU or computer system]] stress test page. This gives details and allows for more central ___location of the computer aspect of load testing. If others agree that it is relevant we can promote it to the main page.

There is a clear difference between load testing and steress testing. Load testing tests a system for behavior at normal and peak loads. The expectation is to ensure it behaves within an acceptable range established by requirements. Stress tests on the other hand test a system until it fails to ensure the system degrades and recovers gracefully. <span style="font-size: smaller;" class="autosigned">— Preceding [[Wikipedia:Signatures|unsigned]] comment added by [[Special:Contributions/149.101.1.120|149.101.1.120]] ([[User talk:149.101.1.120|talk]]) 14:26, 12 February 2014 (UTC)</span><!-- Template:Unsigned IP --> <!--Autosigned by SineBot-->

: There is no unified nomenclature. What one glossary, certifying body or company calls load testing could be called stress testing or performance testing or something else in a different glossary, certifying body or company. [[User:Walter Görlitz|Walter Görlitz]] ([[User talk:Walter Görlitz|talk]]) 15:22, 12 February 2014 (UTC)

== Concurrency testing ==

Is concurrency testing another synonym for load testing? Can anyone verify this? -- [[User:AnonymousDDoS|AnonymousDDoS]] ([[User talk:AnonymousDDoS|talk]]) 02:53, 26 June 2012 (UTC)

: Not really. Usually when testing is done, it is done by one or at most a few individuals and this would be on larger systems and on different parts of the system. Concurrency testing is when multiple people are attempt to access the same part of the system at the same time. Perhaps one is writing a record while another tries to read it. The following article has a short section describing it: http://www.stickyminds.com/s.asp?F=S6920_ART_2 . Whether it may incorrectly be used synonymously with the term load testing isn't really clear. --[[User:Walter Görlitz|Walter Görlitz]] ([[User talk:Walter Görlitz|talk]]) 04:08, 26 June 2012 (UTC)

:: Perhaps we could add a section to load testing, or create a totally new article and provide a red link? -- [[User:AnonymousDDoS|AnonymousDDoS]] ([[User talk:AnonymousDDoS|talk]]) 13:58, 26 June 2012 (UTC)

== This Article seems to be incredibly biased ==

The listing of load testing tools on this page seems to be arbitrary and capricious. Two of the most popular and readily available tools aren't even listed here. Why is one person granted the power to decide what does and does not go on these pages. I thought this was a community project. <small><span class="autosigned">— Preceding [[Wikipedia:Signatures|unsigned]] comment added by [[User:AreYouFreakingKidding|AreYouFreakingKidding]] ([[User talk:AreYouFreakingKidding|talk]] • [[Special:Contributions/AreYouFreakingKidding|contribs]]) 17:40, 6 December 2012 (UTC)</span></small><!-- Template:Unsigned --> <!--Autosigned by SineBot-->

: I totally agree with this. <span style="font-size: smaller;" class="autosigned">— Preceding [[Wikipedia:Signatures|unsigned]] comment added by [[Special:Contributions/50.193.56.110|50.193.56.110]] ([[User talk:50.193.56.110|talk]]) 00:41, 13 February 2013 (UTC)</span><!-- Template:Unsigned IP --> <!--Autosigned by SineBot-->

:: Neither arbitrary nor capricious. The basis for inclusion is common on Wikipedia lists: notability. If there is an article, it may be included here. Not all lists follow this guideline, but many do. If it were not enforced we would have an inordinately long list of small projects that aren't useful for the broad public. This isn't a stand-alone list either since that is simply a short section listing the tools. All of the most popular and readily available tools, according to the tools surveys I have seen, are listed here. Which tools do you think are missing? [[User:Walter Görlitz|Walter Görlitz]] ([[User talk:Walter Görlitz|talk]]) 14:32, 2 May 2013 (UTC)

::: Wrk, Tsung, k6, Eggplant Performance, and Artillery would definitely be a good start. This list is currently incredibly outdated and every attempt to rectify that seems to lead to an instant revert. [[Special:Contributions/217.210.164.244|217.210.164.244]] ([[User talk:217.210.164.244|talk]]) 16:25, 1 April 2021 (UTC)

:::: Any attempt to add a product that has no article to the table should be reverted. There's no reason that they cannot be added, with [[:WP:SECONDARY]] sources in the prose though. [[User:Walter Görlitz|Walter Görlitz]] ([[User talk:Walter Görlitz|talk]]) 00:04, 2 April 2021 (UTC)

== Undid revision 1083961456 {{Reply to|Walter Görlitz}}==

With SoapUI you can perform load tests <ref>https://www.soapui.org/docs/load-testing/getting-started/</ref>. [[User:LeFelix|LeFelix]] ([[User talk:LeFelix|talk]]) 08:41, 22 April 2022 (UTC)

: Thanks. I had not seen that feature before, but I haven't used it in about 12 years. Thanks again. Would you mind updating the SoapUI article with that detail as well? [[User:Walter Görlitz|Walter Görlitz]] ([[User talk:Walter Görlitz|talk]]) 15:27, 22 April 2022 (UTC)

::No problem and sorry for the late reply. As you have already seen, I only added a link to the SoapUI article, since it was already noted that this tool can run load tests. I will add SoapUI to the list again. :) [[User:LeFelix|LeFelix]] ([[User talk:LeFelix|talk]]) 15:37, 5 May 2022 (UTC)

{{reflist-talk}}